Open Source Video Processing Tools - MPlayer, FFMpeg, AviSynth, MKVToolnix, MP4Box

From Texas Instruments Embedded Processors Wiki

Contents 1 Downloads 2 Codecs, Containers, Pixel formats 3 MPlayer 4 FFMpeg 5 YUV Processing 6 Elementary stream processing 7 FOURCC codes 8 Encoding videos using FFMPEG 9 Compiling MPlayer/FFMpeg sources 10 Using MKV files 11 Using Transport stream containers 12 Some basic audio conversion/encoding options 13 Playing in PS3 14 Using AVISynth

Downloads

• MPlayer and FFMpeg for Windows:
  
  • http://oss.netfarm.it/mplayer-win32.php
  • Installation: Just extract and use.
    
• MPlayer codecs:
  
  • Codecs Directory
  • Latest Windows package
  • Installation: Extract the files to <MPLAYER_INSTALL_DIR\codecs>
    
• MP4Box for Windows:
  
  • Pick up latest version from kurtnoise
    • Installation: Just extract and use.
  • GUI options:
    
    • YAMB
      
    • MY-MP4BOX-GUI
      
• x264:
  • x264 Homepage
    
  • Pickup appropriate version for your system. GUI options are also available at same location.
    

• K-Lite Mega Codec Pack for Windows:
  
  • K-Lite Download Page
  • Download the MEGA pack. This shall enable your PC to play all videos files. Even in Windows Media Player if you still use it.
  • Installation notes:
    • Select the "Lots of stuff with player" option during installation.
      
    • Enable DXVA option for both H.264 and VC1 during installation. This will reduce CPU usage drastically on newer systems.
      
• AVISynth
  • Download
    
• MKV utils:
  • MKVToolnix Download Page: Download and install the windows package.
  • MKVExtractGUI Download Page: Download latest version and extract into MKVTOOLNIX install directory.
  • MKVExtractGUI-2 Download Page: Download latest version and extract into MKVTOOLNIX install directory. This is an alternative to the above one.
    
• TS Muxer
  • Download the latest version of TsMuxer with GUI. Extract and add to system PATH
    
• LAME
  • Download the latest version of LAME. Extract and add to system PATH.
    

Codecs, Containers, Pixel formats

• Codecs
  
• Containers
  
• Pixels formats:
  • YUV
  • RGB
    
• To get a list of these, try ffmpeg -codecs, ffmpeg -formats, ffmpeg -pix_fmts

MPlayer

• MPlayer is primarily a media player. It supports practically all audio/video codecs/containers in the market.
  Besides, it can also be used to play RAW YUVs and elementary codec streams as well.
  It provides a host of audio filters (eg: karaoke, surround, scaletempo) and video filters (eg: smartblur, rotate, crop) to post-process the output.
  
• Get help:
  
  • Manpage
    
• Basic command:
  
  • mplayer <filename>
  • mplayer -fps 30 <filename> for elementary streams
    
• Modifying playback FPS:
  
  • mplayer -fps 24 <filename>
    
• Resize during playback:
  
  • mplayer -x <NewWidth> -y <NewHeight> <filename>
    
• De-interlace:
  
  • Linear interpolation:
    
    • mplayer <filename> -vf pp=li
  • Cubic interpolation::
    
    • mplayer <filename> -vf pp=ci
  • Other options:
    
    • mplayer -pphelp
      
• Loop playback:
  
  • mplayer <filename> -loop <loopCount>
    
  • loopCount = 0, means infinite
    
• Giving start offsets while processing:
  • mplayer -ss HH:MM:SS <input>
  • "-ss" sets offset in seconds. You can give HH:MM:SS as well.
  • FPS value is provided by the container. Alternately, -fps <val> can be used to for manual control.
    
• Dumping Elementary streams:
  
  • mplayer <filename> -dumpvideo -dumpfile <videoStream.dump>
    
  • mplayer <filename> -dumpaudio -dumpfile <audioStream.dump>
    
• Playing RAW YUVs:
  
  • mplayer <input_320x240_i420.yuv> -demuxer rawvideo -rawvideo w=320:h=240:fps=30:format=i420
    
  • To see supported formats use "format=help"
    

• Video Filters:
  • Get list of available filters:
    • mplayer -vf help
  • Example: Using the "crop" filter. You can follow similar steps to use other filters.
    • Get help:
      • mplayer -vf crop=help
    • Example 1:
      • mplayer input_1280x720.avi -vf crop=w=640:h=360
      • This shall crop the center 640x360 region of the original 1280x720 video.
    • Example 2:
      • mplayer input_1280x720.avi -vf crop=w=640:h=480:x=0:y=360
        
      • This shall crop the top-right 640x360 region of the original 1280x720 video.
  • Getting Screenshots:
    • mplayer <inputFile> -vf screenshot. Hit "s" to take a screenshot when you want.
    • mplayer <inputFile> -vo png. This will dump .png for all frames.
      

• Controlling window location and output modes:
  • Specifying window location (x,y) in pixels:
    • mplayer <inputFile> -geometry <x>:<y>
  • Playing multiple videos simultaneously:
    • mplayer <inputFile> -vo direct3d
    • If the default value of "directx" is used for "vo", one video will show only green frames
    • This also works better with geometry option explained above

FFMpeg

• FFMpeg is primarily a transcoder. It supports practically all audio/video codecs/containers as well as elementary stream formats in the market.
  It provides a host of audio filters (eg: resampling, downmix channels) and video filters (eg: crop, pad, etc) to use during transcoding.
  
• Get help:
  
  • ffmpeg -h
  • HOW-TO
    
• List of supported containers:
  
  • ffmpeg -formats
    
• List of supported codecs:
  
  • ffmpeg -codecs
    
• List of supported YUV formats:
  
  • ffmpeg -pix_fmts
    
• Typical usage:
  
  • ffmpeg <inputInfo> -i <inputFile> <encodeOptions> <outputFile>
    
• Controlling number of frames to process:
  
  • ffmpeg -vframes 100 <inputInfo> -i <inputFile> <encodeOptions> <outputFile>
    
• Get codec information for a file:
  
  • ffmpeg -i <inputFile>
    
• Extracting YUV:
  
  • ffmpeg -i <inputFile> -vcodec rawvideo <output.yuv>
    
• Resizing YUV:
  
  • ffmpeg -s 320x240 -i <input_320x240.yuv> -s 640x480 <output_640x480.yuv>
    
• Changing YUV formats:
  
  • ffmpeg -s 320x240 -pix_fmt nv12 -i <input_320x240_nv12.yuv> -s 320x240 -pix_fmt yuv420p <output_320x240_yuv420p.yuv>
    
  • "-s" option can be used to resize YUV in same command
    
• Giving start offsets while processing:
  • ffmpeg -ss HH:MM:SS -i <input> -vcodec copy <output>
  • "-ss" sets offset in seconds.
  • "-r <val>" can be used to change the fps.
    
• Extracting elementary streams:
  
  • Identify required elementary stream format from supported formats using
    
    • ffmpeg -formats
      
  • To extract:
    
    • ffmpeg -i <input> -vcodec copy -f <format> <output.format>
      
  • Example 1:
    
    • Extracting elementary H.264 from an AVI
      
      • ffmpeg -formats gives "format = h264"
        
      • ffmpeg -i input.avi -vcodec copy -vbsf h264_mp4toannexb -f h264 output.h264
        
  • Example 2:
    
    • Extracting elementary MPEG4 from an AVI
      
      • ffmpeg -formats gives "format = m4v"
        
      • ffmpeg -i input.avi -vcodec copy -f m4v output.m4v
        
        
• Get bmp/png/jpg for frames:
  
  • ffmpeg -ss 00:10:00 -vframes 10 -i <inputFile> output?d.bmp
    
  • This will dump bitmaps for ten frames from the time stamp 00:10:00 into files name output01.bmp, output02.bmp and so on....
  • You can get .png, .jpg, .gif similarly.
  • Note that lossy codecs like jpeg will involve re-encoding and loss in quality. You can add "qmax=10" or any appropriate value to control the quality.
  • Hack: To handle raw YUV similarly, rename all yuv to bmp. Specify -vcodec rawvideo, -s wxh and -pix_fmt yuv420p before -i filename.
    

• Filters:
  • The following applies to all codecs/YUVs/elementary streams/containers as long as the resultant data is supported by the same.
    
  • Video Cropping:
    • Older versions:
      • ffmpeg -i input_320x240.avi -cropright 10 -cropleft 10 -croptop 10 -cropbottom 10 output_300x220.avi
    • Newer versions:
      • ffmpeg -i input_320x240.avi -vf crop=<x>:<y>:<w>:<h> output_300x220.avi
      • w = Output width
      • h = Output height
      • x = X co-ordinate of input image in the output image
      • y = Y co-ordinate of input image in the output image
        
  • Video Padding:
    • Older versions:
      • ffmpeg -i input_320x240.avi -padcolor <RRGGBB> -padright 10 -padleft 10 -padtop 10 -padbottom 10 output_340x260.avi
      • <RRGGBB[AA]> is in HEX. FF0000 = RED, 0000FF = BLUE, etc...
    • Newer versions:
      • ffmpeg -i input_320x240.avi -vf pad=<w>:<h>:<x>:<y>:<c> output_340x260.avi
      • w = Output width
      • h = Output height
      • x = X co-ordinate of input image in the output image
      • y = Y co-ordinate of input image in the output image
      • c = Padding color in <RRGGBB[AA]> format. FF0000 = RED, 0000FF = BLUE, etc...
        
        
• Also see Encoding using FFMpeg
  

YUV Processing

• Decoding compressed streams:
  • FFMPEG:
    
    • ffmpeg -i <inputFile> -vcodec rawvideo <output.yuv>
      
  • MPlayer:
    
    • mplayer <inputFile> -vo yuv4mpeg
    • This generates "stream.yuv" which is raw data inside a YUV4MPEG header which is used by MJPEG tools.
    • To convert this into raw data only, use ffmpeg as mentioned above.
      
• Playing RAW YUVs:
  
  • mplayer <input_320x240_i420.yuv> -demuxer rawvideo -rawvideo w=320:h=240:fps=30:format=i420
    
  • To see supported formats use "format=help"
    
• Resizing YUV:
  
  • ffmpeg -s 320x240 -i <input_320x240.yuv> -s 640x480 <output_640x480.yuv>
    
• Changing YUV formats:
  
  • ffmpeg -vframes 100 -s 320x240 -pix_fmt nv12 -i <input_320x240_nv12.yuv> -s 320x240 -pix_fmt yuv420p <output_320x240_yuv420p.yuv>
    
  • "-s" option can be used to resize YUV in same command
  • "-vframes" option can be used to control the number of frames to process
    
• Giving start offsets while processing:
  • ffmpeg -r 1 -ss 10 -i input.yuv -vcodec copy output.yuv
  • "-r" sets input fps.
  • "-ss" sets offset in seconds. You can give HH:MM:SS as well.
  • In above example, since fps=1 and offset=10, processing shall start from 10th frame.
    
• Interlaced YUVs
  • Converting field interleaved to field separate format
    • mplayer input.yuv -demuxer rawvideo -rawvideo w=352:h=288 -vf tfields=0 -vo yuv4mpeg -fps 1000000. Note that stream.yuv generated here contains a header.
    • ffmpeg -i stream.yuv output.yuv
    • 100 frames of 352x288 will give 200 frames of 352x144
      
  • Merging field separate to field interleaved format
    • mplayer input.yuv -demuxer rawvideo -rawvideo w=352:h=144 -vf tinterlace -vo yuv4mpeg -fps 1000000. Note that stream.yuv generated here contains a header.
    • ffmpeg -i stream.yuv output.yuv.
    • 200 frames of 352x144 will give 100 frames of 352x288
      
  • Extracting individual fields using mplayer
    • mplayer input.yuv -demuxer rawvideo -rawvideo w=352:h=288 -vf field=<n> -vo yuv4mpeg -fps 1000000. Note that stream.yuv generated here contains a header.
    • Even value for n gives even fields. Odd value for n gives odd fields.
    • ffmpeg -i stream.yuv output.yuv.
    • 100 frames of 352x288 will give 100 frames of 352x144
      
  • Extracting individual fields using ffmpeg - Single step
    • Give input width as twice the actual width and crop appropriately. Check ffmpeg filters (FFMpeg Video Filters).
      
      
• Also see the ffmpeg filters (FFMpeg Video Filters) for padding/cropping methods.
  
• Overlay frame numbers on YUV
  • Create test.srt in MicroDVD subtitle format.
    • Sample line in test.srt: {1}{1}1. This means the subtitle for frame 1 is "1".
    • Sample line in test.srt: {1}{4}1. This means the subtitle for frames 1 to 4 is "1".
    • Create test.srt with entries for sufficient number of frames.
  • mplayer input352x288.yuv -demuxer rawvideo -rawvideo w=352:h=288:fps=1 -fps 100000 -sub test.srt -subdelay 0 -vo yuv4mpeg . Note that stream.yuv generated here contains a header.
  • ffmpeg -y -i stream.yuv output352x288_withFrameNumOverlay.yuv
  • First couple of frame numbers do not get overlayed. Send me a solution if you have one.
  • The fps and subdelay options seemed to reduce the number of frames not getting overlayed at the beginning. Not really sure why.
    

Elementary stream processing

• Tools and supported formats:
  • FFMPEG
    • YUV formats: ffmpeg -pix_fmts
    • Codecs: ffmpeg -codecs
    • Containers: ffmpeg -formats
      
  • MPlayer
    • Available RAW/compressed video codecs: mplayer -vc help
      
  • MP4Box
    • RAW formats and containers: mp4box -h format
      
• Extracting elementary streams:
  • Suggested tools for different formats:
    • .avi with any codec: FFMpeg
    • .mov, .3gp , .mp4 with MPEG4/H.264: MP4Box
    • For FFMPEG, use "-vbsf h264_mp4toannexb -f h264 -vcodec copy" to dump elementary stream with start codes.
      • ffmpeg -i <inputFileWith264> -vbsf h264_mp4toannexb -f h264 -vcodec copy output.264
    • In case the above suggestions don't work, try out other tools. eg: For .3gp with H.264, START CODES are not dumped. However, MP4Box dumps proper elementary stream.
    • Another workaround is to convert the input into an AVI using FFMPEG (see Creating AVI) and then extract elementary using FFMPEG.
      
  • FFMpeg
    • Identify required elementary stream format from supported formats using
      
      • ffmpeg -formats
        
    • To extract:
      
      • ffmpeg -i <input> -vcodec copy -f <format> <output.format>
        
    • Example 1:
      
      • Extracting elementary H.264 from an AVI
        
        • ffmpeg -formats gives "format = h264"
          
        • ffmpeg -i input.avi -vcodec copy -vbsf h264_mp4toannexb -f h264 output.h264
          
    • Example 2:
      
      • Extracting elementary MPEG4 from an AVI
        
        • ffmpeg -formats gives "format = m4v"
          
        • ffmpeg -i input.avi -vcodec copy -f m4v output.m4v
          
  • MPlayer:
    
    • mplayer <filename> -dumpvideo -dumpfile <videoStream.dump>
      
    • mplayer <filename> -dumpaudio -dumpfile <audioStream.dump>
      
  • MP4Box:
    
    • Get track-IDs to dump using mp4box -info input.mp4
      
    • Dump required track using mp4box -raw <TrackId> input.mp4
    • GUI Alternatives: YAMB, MY-MP4BOX-GUI
      
      

• Creating AVI/MP4/other container format files from elementary stream
  • FFMpeg
    • ffmpeg -i <elementaryStream> -vcodec copy output.avi
    • ffmpeg -i <elementaryStream> -vcodec copy output.mp4
      
  • MP4Box
    • mp4box -add input.m4v output.mp4
    • mp4box -add input.m4v input.aac output.mp4
    • Remember to delete the output file before running the command. MP4Box appends to existing files by default.
      

FOURCC codes

• (FOURCC == FOUR Character Code) is used to identify a video codec inside an AVI container.
  
• Get help:
  • fourcc.org
  • Wikipedia
    
• Changing FOURCC:
  • ffmpeg -i <inputFile> -vcodec copy -acodec copy -vtag <XXXX> output_FOURCC_XXXX.avi
  • <XXXX> should be the required FOURCC code in the avi.
  • Modify FOURCC tags to force PC to use different decoders.
    • Example 1:
      • In case of MPEG4 stream, FOURCC can be XVID or DIVX. Default is FMP4.
      • If FOURCC=XVID, and you have an XVID decoder installed, and you try to play the avi in Windows Media Player, the XVID decoder is used.
        
      • If FOURCC=DIVX, and you have an DIVX decoder installed, and you try to play the avi in Windows Media Player, the DIVX decoder is used.
        
      • If FOURCC=FMP4, and you have installed K-Lite Codec Pack, and you try to play the avi in Windows Media Player, ffdshow decoder is used.
  • GUI alternative: Nic's Mini AviC FourCC Changer - comes with K-Lite Codec Pack
    

Encoding videos using FFMPEG

• Read about FFMpeg first in FFMpeg section.
  
• Using preset files
  • Get presets files from the ffpresets directory in the FFMPEG source package.
  • The preset file naming convention is self-explantory.
  • The parameters in the preset files are explained in the output of ffmpeg -h
    
• Specifying bitrates for encoding
  • Example: To encode at 1mbps (million bits per second) with deviation of upto 0.5mbps:
    • ffmpeg -i <input> -vb 1000000 -bt 500000 -fpre <PathToPresetFile> <output>
      
• Specifying codecs
  • Default video codec: MPEG4.
  • Default audio codec: MP2.
  • Example: To encode using h264 and aac:
    • ffmpeg -i <input> -vcodec libx264 -acodec ac3 <output>
      
• Faster encoding speeds
  • FFMPEG supports upto 8 parallel threads whenever possible.
  • Example: Typically keep number of threads equal to number of CPU cores you have.
    • ffmpeg -i <input> -threads 8 <output>
      
• Single Pass Encoding
  • ffmpeg -i <input> -vcodec libx264 -acodec ac3 -fpre <PathToPresetFile> <output>
    
• Two Pass Encoding
  • 1st Pass: ffmpeg -pass 1 -i <input> -vcodec libx264 -acodec ac3 -fpre <PathToFirstPassPresetFile> <output>
  • 2nd Pass: ffmpeg -pass 2 -i <input> -vcodec libx264 -acodec ac3 -fpre <PathToNormalPresetFile> <output>
    

Compiling MPlayer/FFMpeg sources

• Linux
  • Extract sources
    • MPlayer: Checkout, Other options
    • FFMpeg: Checkout, Other options
  • ./configure --help /* List options for compiling*/
  • ./configure /* Auto-detect features to enable, manually add as required */
  • make /* Compile the code */
  • make install /* Copy binaries to $PATH */
• Windows
  • Get MINGW build environment from BE gcc 4.2.5/MinGW-full-gcc-4.2.5-Mar-2010.7z/download Sherpya
  • Wiki, forums, etc...
    

Using MKV files

• MKV stands for Matroska Video. Other related extensions are .mka and .mks for audio and subtitles. This is a multimedia container format just like .AVI or .MP4.
  
• Install MKV utils
  • MKVToolnix Download Page: Download and install the windows package.
  • MKVExtractGUI Download Page: Download latest version and extract into MKVTOOLNIX install directory.
  • MKVExtractGUI-2 Download Page: Download latest version and extract into MKVTOOLNIX install directory. This is an alternative to the above one.
    
• Extracting individual tracks:
  • Exectuable: mkvextract.exe
  • GUI: MKVextractGUI.exe
    
• Creating MKV:
  • Exectuable: mkvmerge.exe
  • GUI: mmg.exe
  • Things you can do:
    • These utils are just muxers/demuxers. They do not re-encode any data. Refer to FFMpeg section
      to do so.
    • Add .avi and .srt (subtitles) files and create a single .mkv containing the subtitles.
    • Options are available to split the file into multiple files of smaller sizes.
    • Create DVD like Chapters in the mkv.
      

Using Transport stream containers

• Download the latest version of TsMuxer with GUI.
• Supported formats: .TS, .M2TS, Blu-ray, AVCHD. Demuxing is also supported
• Most input file formats (avi, mkv, etc...) are supported.
• Note that there is some issue with very long file names. If your file is not detected, try reducing the length of the file path.
  

Some basic audio conversion/encoding options

• Converting any video/audio file into MP3
  • ffmpeg -i <inputFile> -ac 2 -ab 128k output.mp3
    • "ac" is used to convert input to 2 channels which is supported by mp3.
    • "ab" is the bitrate to be used for encoding.
  • If input already contains mp3 data use: ffmpeg -i <inputFile> -acodec copy output.mp3
  • Better encoding:
    • Get raw audio: ffmpeg -i <inputFile> -ac 2 -ab 128k output.wav
    • Use LAME: lame --preset insane output.wav output.mp3
      
• Other codecs
  • Supported codec list: ffmpeg -codecs
  • Supported format list: ffmpeg -formats
  • Above set of commands remain valid for other formats like .aac, .mp2, .ac3, etc...
  • Output file extension has to be appropriately used.
    
• Changing audio only in a file
  • ffmpeg -i <inputFile> -ac 2 -ab 128k -vcodec copy <outputFile>
    
• Removing audio in a file
  • ffmpeg -i <inputFile> -an -vcodec copy <outputFile>

Playing in PS3

• PS3 Media Server
  • This shall create a media server on you PC which is on same network as your PS3.
  • This effectively removes file size limitation of 4GB (due to FAT32).
  • Also all containers avi, mp4, flv, mkv can be played without re-encoding. The application will re-mux the stream into a compatible container.
  • If a file is not supported by PS3, it has a transcode option which will stream media files in PS3 compatible formats.
    
• If you still want to re-encode yourself: PS3 H.264 Conversion Guide
• Some known limitations of PS3 (when playing without the media server):
  • H.264 is supported only upto Level 4.1
  • .mp4 files generated by FFMPEG are not playable directly. Use MP4Box to generate the .mp4 files.

Using AVISynth

• Overview with example scripts File:AviSynth overview.pdf
• AviSynth is a powerful tool for video post-production. It provides ways of editing and processing videos. AviSynth works as a frameserver, providing instant editing without the need for temporary files.
  
• AVISynth handles only RAW picture data. However, you can use compressed videos as source in an .avs script using certain filters.
  
• AVISynth in itself has no GUI and provides only a DLL that other applications can call. After installing avisynth, you can play .avs scripts in MPlayer/VLC/any other media player. You can also see the script as an input to FFMpeg.
  
• AVISynth filters
  • A filter here refers to a processing functionality provided by AVISynth using its Internal filters or External filters/plugins.
  • To install external filters, just extract the provided DLLs into the plugins directory under AVISYNTH's install directory.
  • Note that avisynth.dll is kept in C:\WINDOWS\SYSTEM32\ directory, but plugins are present in <AVISYNTH_INSTALL_DIR>\plugins.
    
• Sample script (Separating odd and even fields and displaying separately):
  • Create a sample.avs file with the following lines. Change F:\sample.avi to a valid filename.
    • V=DirectShowSource("F:\sample.avi")
    • V=AssumeFrameBased(V)
    • V=AssumeTFF(V)
    • VS=SeparateFields(V)
    • VTop=SelectEven(VS)
    • VBot=SelectOdd(VS)
    • stackvertical(VTop,VBot)
  • You can play this file in MPlayer using the command mplayer sample.avs
  • The output will show the original video in the top 2/3 of the output. The bottom 1/3 will show the video one field at a time. As a result the lower part will take twice as long to complete. The AssumeTFF filter shall indicate top field to be displayed first.
  • You can use encode this output using ffmpeg by using the command ffmpeg -i sample.avs output.avi. You might need to add other options to control the codec/bitrate of output.avi.
  • The compressed sample.avi is decoded using DirectShow codecs installed in the system. These will get installed when you install the K-Lite Codec Pack (K-Lite Codec Pack). Using DirectShow almost all codecs can be supported in .avs scripts.
    
• Sample script (Displaying two videos side by side):
  • Create a sample.avs file with the following lines. Change F:\sample1.avi & F:\sample2.avi to valid filenames.
    • V1=DirectShowSource("F:\sample1.avi")
    • V2=DirectShowSource("F:\sample2.avi")
    • stackhorizontal(V1,V2)
• Commonly useful filters:
  • RawSource - Read raw data
  • DirectShowSource - Read compressed data
  • FFmpegSource2 - Read compressed data (alternative)
  • ShowSMPTE, ShowFrameNumber - show timestamps
  • AssumeFieldBased, AssumeFieldBased - Use data as frames/fields
  • SelectOdd, SelectEven - Select fields and separately process
  • SeparateFields, Weave - Interleave/De-interleave fields
  • StackVertical, StackHorizontal - Create mosaics
  • Resize, AddBorders, Crop, LetterBox
    

Leave a Comment