FSCONS Video processing

13 Apr 2011

FSCONS video post processing

By popular demand, here's how we did our video post processing. We focused on getting everything up on Vimeo, so the videos contain only the sound track from the camera. We'll make raw format videos and surrounding sound available via bittorrent when we have figured out where we have enough disk space.

This description is very specific to our setup and file name and directory layout. So, the scripts used will assume that the naming and directory layout as described below is in place.

Raw video and sound file information

The video format from the cam memory cards are in "MTS" format: Input #0, mpegts, from 'Sessions/101-RepRapOpenHardware/00001.MTS': Duration: 00:34:48.96, start: 0.481989, bitrate: 7277 kb/s Program 1 Stream #0.0[0x1011]: Video: h264, yuv420p, 1440x1080 [PAR 4:3 DAR 16:9], 50 tbr, 90k tbn, 50 tbc Stream #0.1[0x1100]: Audio: ac3, 48000 Hz, stereo, s16, 256 kb/s

And the external sound is plain WAV, for instance: Sessions/101-RepRapOpenHardware/ |-- 00001.MTS |-- 4CH002I.wav `-- 4CH002M.wav

The sound in the video file is from the camera mic, and the sound on 4CH002I.wav is from the (??) mic, and the sound on 4CH002M.wav is from the (??) mic.

What format and sources we decided on for publishing on Vimeo

First of all, the MTS files are huge and not easy to handle. Each file is between 1 and 4 GB in size. Second, it is timely and tedious to match and align the surrounding sound files with the video (and sync it with the sound from the camera mic). So we decided to go for only the original internal camera sound for publishing on Vimeo.

Vimeo is a bit picky with what formats it accepts for uploads, and the ffmpeg and mencoder software are not so forgiving either. Rikard found that, since we are to prepend each published video with an FFKP crafted credits page (FSCONS and thanks to .SE and Nordisk Kulturfond) and a CC license, a way forward was to make everything videos in an AVI container (using the same codecs for video and sound in both the prepend files and the AVI from the session MTS).

This way, mencoder could concatenate credits->license->session into an AVI with the whole shebang.

This is an example of the format for a Session AVI: Input #0, avi, from 'Sessions/encoding/encoded_vids/101-RepRapOpenHardware-1.avi': Duration: 00:34:48.99, start: 0.000000, bitrate: 2283 kb/s Stream #0.0: Video: h264, yuv420p, 1280x720, PAR 1:1 DAR 16:9, 25 tbr, 25 tbn, 50 tbc Stream #0.1: Audio: ac3, 48000 Hz, stereo, s16, 256 kb/s and this is the cc license AVI: Input #0, avi, from 'Sessions/encoding/prepends/cc-1920.avi': Duration: 00:00:02.01, start: 0.000000, bitrate: 634 kb/s Stream #0.0: Video: h264, yuv420p, 1280x720, PAR 1:1 DAR 16:9, 50 tbr, 50 tbn, 100 tbc Stream #0.1: Audio: ac3, 48000 Hz, stereo, s16, 256 kb/s

In order to create the license AVI you need to add a fake sound track in the ac3, 48000 Hz, stereo, s16, 256 kb/s format for it to match the AVI created from the MTS file.

Strategy for the encoding scripts

We decided to create and upload around one session per day. In order to keep the original files safe, Rikard created a working directory on our NAS where the input files to be encoded by the scripts, and the prepend files and the resulting AVI reside: $ ls -l Sessions/encoding total 1 drwx------ 1 rikard rikard 0 2010-11-26 14:01 114-ConflictingValues drwx------ 1 rikard rikard 0 2010-11-26 14:02 118-OneWireSensors drwx------ 1 rikard rikard 0 2010-11-26 14:13 120-EmbeddingLinux drwx------ 1 rikard rikard 0 2010-11-26 13:05 130-SocialResponsibleNetworks -rwx------ 1 rikard rikard 122 2011-03-14 10:14 DoneAndUploadedSessions.txt drwx------ 1 rikard rikard 0 2011-03-14 10:21 encoded_vids drwx------ 1 rikard rikard 0 2011-03-08 11:57 prepends

The directories with names starting with a number are the sessions to be encoded by one round of the encoding scripts. The results go into the encoded_vids directory, and the credits and license AVIs are in the prepends directory. The textfile is just a list to keep track of what has been uploaded to Vimeo and that one is manually maintained ;-)

The encoded AVIs end up in the encoding/encoded_vids directory.

Scripts and commands used to start a round of encoding

There is a directory on our NAS with all the scripts needed for a round of encoding: ~$ tree Sessions/bin/ Sessions/bin/ |-- encode_all.sh `-- encode.sh

The encode_all.sh script must be run from the Sessions directory (or it won't run). It assumes you have put all sessions directories you want to encode, for instance 114-ConflictingValues 118-OneWireSensors 120-EmbeddingLinux and 130-SocialResponsibleNetworks in the encoding working directory (which then works as a queue for encoding). This is best accomplished by logging into the nas (in this case the nas is also the place for the working directory "encoding") via ssh and rsync the directories to be encoded from Sessions to Sessions/encoding: /home/rikard$ cd Sessions/ Sessions$ rsync -va --progress 114-ConflictingValues 118-OneWireSensors 120-EmbeddingLinux 130-SocialResponsibleNetworks encoding/

Note that it is, of course, imperative to leave out the trailing slashes on the session directory names, or otherwise rsync will only copy the files within the directories into the encoding directory, and not the whole directories themselves.

After that, the encoding directory will have, in this case, four video directories whose names start with a number: 114-ConflictingValues 118-OneWireSensors 120-EmbeddingLinux 130-SocialResponsibleNetworks.

This is how the encode_all.sh finds them (it treats all directories in encoding that have names starting with a number as video directories to be encoded). This is of cource specific to our setup, and that is exploited by the encoding scripts.

The encode.sh script is just used by encode_all.sh as an auxiliary script, wrapping the creation of AVI from MTS (using ffmpeg).

The resulting encoded AVIs end up in encoding/encoded_vids/ and if there are more than one MTS video per session, they are named with a sequence number starting at 1 like so:[SessionName]-1-all.avi, e.g.: 108-FutureTransports-1-all.avi 108-FutureTransports-2-all.avi 108-FutureTransports-3-all.avi

To start an encoding round, these are the steps:

  1. Make sure last run has finished
  2. Move or delete last run's video directories from the encoding directory
  3. Log in to the nas via ssh and rsync the session directories to be encoded so that they are copied from Sessions/ into Sessions/encoding/ (leave out trailing slashes on video direcory names so that the complete directories are copied)
  4. cd into Sessions/
  5. Run: encode_all.sh 2>&1 | tee -a txt-files/encoding.log (-a flag is for "append") Leave this shell open (or start it in a Screen session)
  6. If you'd like to watch what's going on, do a tail -200f txt-files/encoding.log (if you want to see what it's doing from a different shell)
  7. When the script is finished, verify that the videos are OK with mplayer, e.g.: Sessions$ mplayer encoding/encoded_vids/101-RepRapOpenHardware-1-all.avi
  8. Log in on Vimeo and upload the file named SessionName-1-all.avi (the -all part simply means it's got the credits and license prepended and included in it).
  9. Look up the session on FSCONS 2010 schedule and name the video SpeakerFirstName SpeakerLastName - Session Name
  10. In the description, start with the title: SpeakerFirstName SpeakerLastName - Session Name [(pt 1/n)] (if it's one of many parts), then copy the abstract from FSCONS schedule page (sessions in the schedule are links to the abstract page)
  11. Tag the video with appropriate tags, FSCONS, FSCONS 2010, and save the video.
  12. Do the same for each newly encoded file (named with -all.avi at the end) and then delete the Session directories from the encoding working directory to save up space if needed.
  13. Mark the session as uploaded in encoding/DoneAndUploadedSessions.txt

How does it work?

The encode_all.sh script must be run from the Sessions directory. It loops over each directory in Sessions/encoding starting with a number. For each such directory, it loops over each .MTS video file and encodes it to AVI naming it SessionDirectoryName-N.avi (where N is the sequence number, starting with 1 - e.g. the three MTS files in 108-FutureTransports/ will be named: 108-FutureTransports-1.avi 108-FutureTransports-2.avi 108-FutureTransports-3.avi). Next, it prepends each one of them with the credits and licens AVI into an AVI named e.g. 108-FutureTransports-1-all.avi . This means that it is the files named SessionDirectoryName-N-all.avi that are the full files, credits and license and session, that are interesting and to be uploaded to Vimeo.

encode_all.sh is available here and is provided "as is" and use at own risk etc.

The encoding into AVI is done with help of the Session/bin/encode.sh script. It uses ffmpeg with the following flags and options: ffmpeg -i encoding/SessionDirectoryName/RawVideoFromCamera.MTS -acodec copy -vcodec libx264 -s 1280x720 -vpre default -b 2000000 -r 25 \ encoding/encoded_vids/SessionDirectoryName-N.avi

The encode.sh script is available here and is provided "as is" and use at own risk etc.

The concatenation of the credits and license AVIs to the beginning of the end result AVI before the session video is done with mencoder: mencoder -oac copy -ovc copy -o encoding/encoded_vids/SessionDirectoryName-N-all.avi \ encoding/prepends/1920-title.avi encoding/prepends/cc-1920.avi \ encoding/encoded_vids/SessionDirectoryName-N.avi


In order to run the encoding scripts, a number of tools, libraries and codecs need to be installed.

The video transformation tools are:

  • ffmpeg
  • mencoder

The codecs are:

  • libavcodec-extra-52
  • libaften0 - Not required if you have the license and title AVIs with AC3 sound! (for creating the title and license AVI, we needed it to have fake silent audio as AC3 to match the AVIs created from the Session MTS files, or mencoder would protest)

To test the results:

  • mplayer

Apart from this, you need a fairly speedy PC in order to get a timely encoding. And of course, you need to have mounted the NAS directories on that PC.

What about the raw files and surround audio WAVs?

Creating title frames with silent audio

If you'd like to create a - say 4 seconds - banner to show first in the video with no audio, you can do that from a PNG image.

This is how I did it: ffmpeg -loop_input -t 4 -i TitleNeutral.png -ar 44100 -acodec pcm_s16le -f s16le -ac 2 -i /dev/zero -acodec ac3 -ar 48000 -ab 256k -vcodec libx264 -s 1280x720 -vpre default -b 5000000 -r 50 titleNeutral.avi

The PNG was TitleNeutral.png and just a screen saying "FSCONS" more or less. I needed to create an AVI with AC3 sound (silent, but still AC3) so that mencoder could concatenate it with the AVI from a session. mencoder needs the video AND audio streams to be exactly the same, in order to concatenate the AVIs (as far as I've discovered, anyway).

Short re-cap

The Sessions directories all have names starting with a number, such as 101-RepRapOpenHardware . Then cd into Sessions/ and run bin/encode_all.sh 2>&1 | tee -a txt-files/encoding.log (which will append output to the logfile).

Note that bin/encode_all.sh will encode every MTS file from every directory in Sessions/encoding starting with a number (such as 101-RepRapOpenHardware ) so clean up after you are done if file space is an issue;-)

The resulting file(s) to be uploaded to Vimeo end up in Sessions/encoding/encoded_vids/ and will have the name(s) in this form: SessionDirectoryName-N-all.avi (where N is the sequence number, starting with 1). The -all part of the name, is there to separate it from the AVI without credits and license. It is the "-all" file you upload to Vimeo. You can tag and describe the video on Vimeo according to our standard as described above.

Of course, and obviously, the scripts are not intended as best practise or even good practise for bash or video programming. I will assume all shame and blame for lousy coding, but the scripts are provided in the hope that they will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE, as the saying goes ;-)