Push It To the Limit #3

If you’re considering trying out autocross, I say go for it. I’m very new, and have found people at the two events I’ve attended (WOSCA #1, and PITL #3) to be friendly and extremely helpful.

Additionally, you can do it with your own car. You don’t need some sort of special race-spec track beast. Both events have also had loaner helmets available (although I spent $200 and bought my own helmet meeting the appropriate standards).

I’ve posted a video of the PITL #3 event on youtube. Additionally, I’ve created a playlist, which also contains my WOSCA videos.

Video!

When I left off after my third adventure with ffmpeg, I decided that I wanted to attempt video transitions next time. Well, it’s next time.

Individual Clip Preparation

Like always, I concatenated my dashcam video chunks, then trimmed them as appropriate. This was done like my previous posts, using -codec copy to preserve quality.

Timelapse

I decided to attempt a timelapse intro/outro of my drive to the event. 32 times seemed to be a good balance of not-too-long, while also being somewhat fluid.

ffmpeg -i 0-commute.mov -filter:v "setpts=0.03125*PTS" -an -strict -2 0-commute-timex32.mov;

That command is the trimmed commute clip as input. The filter is basically speeding up the playback. 1*PTS would be normal speed, and 1/32 = 0.03125.

I did not timelapse the audio, although that is possible too.

Other prep

I also made title images in Gimp, like before.

Performance considerations

I went through several iterations to reach the final video I wanted. The best resource I had was this post by Mulvya on Stack Overflow.

Initially, I had five video streams, configured to fade in/out as required. However, I very quickly hit a performance issue related to ffmpeg’s overlay filter. Previously I used concat, but performing a transition requires the videos to actually overlay.

To illustrate the issue, assume you have two one-minute videos, and you want a two-second transition between them. Your final video will be 1:98. ffmpeg will stretch the 1 minute video’s last frame to the length of the video, which means it is now compositing 98 additional seconds that could technically be preserved unchanged. Attempting to work around this (by fading out the source video, trimming it, etc) seem to basically mean you’re compositing a transparency. The performance is still affected.

Then remember I’m compositing five videos together, each one building on the previous. This very quickly went from real-time to 5fps processing time.

The “solution” is to chunk each clip into three parts: A A short beginning and ending clip a few seconds long, and a middle clip that requires no modifications. Now you can overlay the two 2 second clips to create your transition, and concat the other 1:96.

The performance benefit is substantial.

The Video Build Command

The command I ended up with is basically a big mess, so I’ll just get start with it, then pick apart each section:

 4 time ffmpeg \
 5 -i 0-commute-timex32.mov \
 6 -i 1-Run1-trim.mov \
 7 -i 2-Run2-trim.mov \
 8 -i 3-Run3-trim.mov \
 9 -i 4-Run4-trim.mov \
10 -i 5-commute-timex32.mov \
11 -loop 1 -i ../../../PITL_logo.png \
12 -loop 1 -i ../../../0-PITL.png \
13 -loop 1 -i ../../../1-Run1.png \
14 -loop 1 -i ../../../2-Run2.png \
15 -loop 1 -i ../../../3-Run3.png \
16 -loop 1 -i ../../../4-Run4.png \
17 -loop 1 -i ../../../5-Run5.png \
18 -filter_complex "
19   [0:v]  trim=start=0:end=15,setpts=PTS-STARTPTS                                [clip0start];
20   [0:v]  trim=start=15:end=170,setpts=PTS-STARTPTS                              [clip0];
21   [0:v]  trim=start=170:end=172,fade=out:st=170:d=1:alpha=1,setpts=PTS-STARTPTS [clip0end];
22 
23   [1:v]  trim=start=0:end=10,setpts=PTS-STARTPTS                                [clip1start];
24   [1:v]  trim=start=10:end=88,setpts=PTS-STARTPTS                               [clip1];
25   [1:v]  trim=start=88:end=90,fade=out:st=88:d=1:alpha=1,setpts=PTS-STARTPTS    [clip1end];
26 
27   [2:v]  trim=start=0:end=10,setpts=PTS-STARTPTS                                [clip2start];
28   [2:v]  trim=start=10:end=106,setpts=PTS-STARTPTS                              [clip2];
29   [2:v]  trim=start=106:end=108,fade=out:st=106:d=1:alpha=1,setpts=PTS-STARTPTS [clip2end];
30 
31   [3:v]  trim=start=0:end=10,setpts=PTS-STARTPTS                                [clip3start];
32   [3:v]  trim=start=10:end=106,setpts=PTS-STARTPTS                              [clip3];
33   [3:v]  trim=start=106:end=108,fade=out:st=106:d=1:alpha=1,setpts=PTS-STARTPTS [clip3end];
34 
35   [4:v]  trim=start=0:end=10,setpts=PTS-STARTPTS                                [clip4start];
36   [4:v]  trim=start=10:end=98,setpts=PTS-STARTPTS                               [clip4];
37   [4:v]  trim=start=98:end=100,fade=out:st=98:d=1:alpha=1,setpts=PTS-STARTPTS   [clip4end];
38 
39   [5:v]  trim=start=0:end=10,setpts=PTS-STARTPTS                                [clip5start];
40   [5:v]  trim=start=10,setpts=PTS-STARTPTS                                      [clip5];
41 
42   [6:v]  trim=start=0:end=15,fade=out:st=3:d=1:alpha=1,setpts=PTS-STARTPTS                           [logo];
43   [7:v]  trim=start=0:end=15,fade=in:st=5:d=1:alpha=1,fade=out:st=12:d=1:alpha=1,setpts=PTS-STARTPTS [title0];
44   [8:v]  trim=start=0:end=9,fade=in:st=3:d=1:alpha=1,fade=out:st=7:d=1:alpha=1,setpts=PTS-STARTPTS   [title1];
45   [9:v]  trim=start=0:end=9,fade=in:st=3:d=1:alpha=1,fade=out:st=7:d=1:alpha=1,setpts=PTS-STARTPTS   [title2];
46   [10:v] trim=start=0:end=9,fade=in:st=3:d=1:alpha=1,fade=out:st=7:d=1:alpha=1,setpts=PTS-STARTPTS   [title3];
47   [11:v] trim=start=0:end=9,fade=in:st=3:d=1:alpha=1,fade=out:st=7:d=1:alpha=1,setpts=PTS-STARTPTS   [title4];
48   [12:v] trim=start=0:end=9,fade=in:st=3:d=1:alpha=1,fade=out:st=7:d=1:alpha=1,setpts=PTS-STARTPTS   [title5];
49 
50   [clip0start][logo]     overlay [clip0logo];
51   [clip0logo] [title0]   overlay [clip0transition];
52 
53   [clip1start][title1]   overlay [clip1title];
54   [clip1title][clip0end] overlay [clip1transition];
55 
56   [clip2start][title2]   overlay [clip2title];
57   [clip2title][clip1end] overlay [clip2transition];
58 
59   [clip3start][title3]   overlay [clip3title];
60   [clip3title][clip2end] overlay [clip3transition];
61 
62   [clip4start][title4]   overlay [clip4title];
63   [clip4title][clip3end] overlay [clip4transition];
64 
65   [clip5start][title5]   overlay [clip5title];
66   [clip5title][clip4end] overlay [clip5transition];
67 
68  [clip0transition] [clip0]
69  [clip1transition] [clip1]
70  [clip2transition] [clip2]
71  [clip3transition] [clip3]
72  [clip4transition] [clip4]
73  [clip5transition] [clip5] concat=n=12 [vout]
74 " \
75 -map "[vout]" \
76 -aspect '16:9' \
77 -codec:v libx264 -crf 21 -bf 2 -flags +cgop -pix_fmt yuv420p \
78 -movflags faststart \
79 -strict -2 \
80 00_final_video.mp4

Yeah…

  • Lines 5-17 are my source input files. All visual files are here (video, overlays, etc).

  • Lines 19-21 are the chunking of the first video into start/main/end segments. This is then repeated for each clip through to Line 40

  • setpts to guarantee every video segment starts its counters at 0 (instead of their pre-existing timestamps).
  • trim is used to cut the video. Values calculated by hand.
  • The end of each video has a fade-out. You could do a fade-in on the start segment, but that has additional quirks I’ll discuss shortly.
  • Lines 42-48 are the title cards (png files from Gimp)
  • Each configured with a fade-in, and fade-out, and trim values.
  • Line 53/54 are the short overlay transitions.
  • Line 53 overlays the title card on the starting video
  • Line 54 overlays the fade-out transition of the ending video on-top of the output from Line 53

It is important to note here that you could use a fade-in on the start video instead. However, you’ll need your fade-out to be the exact correct length, otherwise your fade-in will be truncated, and/or your fade-in won’t transition seamlessly to the rest of the clip. This way was easier.

  • Lines 50/51 are very similar, but for the Logo and main title.

  • Lines 68-73 simply concatenate the short transition overlays we’ve created, with the untouched video clips inbetween.

This saves soooo much processing time.

  • Line 75 maps the final concat output to the file

  • Line 76 fixes the aspect ratio

  • Line 77-79 sets the youtube video codec stuff

  • Line 80 is the output file name.

Now, that’s pretty complex.

The Audio Build Command

You also might notice there’s no sound. Attempting to do sound at the same time caused a number of buffer issues, video stutter, sound not lining up. Furthermore, while this video renders “faster” than a pure-overlay approach, we’re still talking >10 minutes, so trial-and-error to fix sound was very time consuming. I eventually decided to just do the sound separately, then merge them afterwards.

100 time ffmpeg \
101 -itsoffset 0 -i ../../../Alternate.mp3 \
102 -i 1-Run1-trim.mov \
103 -i 2-Run2-trim.mov \
104 -i 3-Run3-trim.mov \
105 -i 4-Run4-trim.mov \
106 -itsoffset 0 -i ../../../Drifting_2.mp3 \
107 -filter_complex "
108   [0:a]  atrim=start=0:end=171,asetpts=PTS-STARTPTS [clip0sound];
109   [1:a]  atrim=start=0:end=89,asetpts=PTS-STARTPTS  [clip1sound];
110   [2:a]  atrim=start=0:end=107,asetpts=PTS-STARTPTS [clip2sound];
111   [3:a]  atrim=start=0:end=107,asetpts=PTS-STARTPTS [clip3sound];
112   [4:a]  atrim=start=0:end=99,asetpts=PTS-STARTPTS  [clip4sound];
113   [5:a]  atrim=start=0:end=246,asetpts=PTS-STARTPTS [clip5sound];
114 
115  [clip0sound] [clip1sound]  acrossfade=d=1  [a01];
116  [a01]        [clip2sound]  acrossfade=d=1  [a012];
117  [a012]       [clip3sound]  acrossfade=d=1  [a0123];
118  [a0123]      [clip4sound]  acrossfade=d=1  [a01234];
119  [a01234]     [clip5sound]  acrossfade=d=1  [aout]
120 
121 " \
122 -map "[aout]" \
123 -vn \
124 -codec:a aac -strict -2 -b:a 384k -r:a 48000 \
125 -movflags faststart \
126 -strict -2 \
127 00_final_audio.mp4

This command is actually very similar. Interestingly, there is a crossfade functionality for audio. Why this doesn’t exist for video, I’ll never know.

  • Lines 101-106 are the input files. Note that I used mp3 files for the timelapses. These are from the Youtube Audio Library (attributed appropriately on the youtube video)

  • Lines 108-113 are similar time clipping, using values similar to the source videos (but all 1 second off, due to the two-second overlap I used for the videos)

  • Lines 115-119 are stringing the audio together using the acrossfade filter.

Merging Audio and Video

ffmpeg -i 00_final_video.mp4 -i 00_final_audio.mp4 -codec copy 00_final_merged.mp4

Not much too it. You may need to do a few rounds of Audio & Merging to ensure the sound lines up with the video.