DM81xx Gstreamer Plugin

From Texas Instruments Wiki
Jump to: navigation, search

Overview

The TI DM81xx GStreamer plug-in uses TI implemented OpenMax development framework to simplify applications use HDVPSS, C674x DSP, and HDVICP2 hardware accelerators on TI's DM816x/DM814x DaVinci™ Digital Media Processors. GStreamer is run as an application on the ARM processor and through the use of OpenMax IL APIs is able to interact with the HDVPSS, C674x DSP and HDVICP2 to acquire, render or transform audio and video streams.

High level features and usecases

  • Supported Video decoders – MPEG-2, H.264 BP/MP/HP, MPEG-4 SP/ASP, VC1, MJPEG
  • Video Post processing/Display/Capture – Scalar & Color Conversion, De-interlacer, V4L2 display, V4L2 capture
  • Audio decoders/encoders
    • on C674x DSP - MP3, AAC-LC, AC3 decoders, AAC-LC encoder
    • on ARM Cortex A8 - Several open-source audio codecs
  • Support for decode and display pipelines
    • Playback pipelines for containers (MP4, MOV, TS, PS, VOB, ASF) with above listed video codecs and wide range of audio codecs
  • Support for A/V capture + encode pipelines with H.264 video and AAC-LC audio
  • Support for transcode pipelines
    • H264 to H264 transcode (different resolution/framerate etc)
    • MPEG-2 to H264 transcode
  • Support for multi-channel network video receiver pipelines (with Mosaic display)
    • 4 x RTP Streams =>Decode => Video Mixer (4=>1 Mosaic) => Display


DM81xx Gstreamer System Architecture

DM81xx Gstreamer architecture 1.jpg

Components

GStreamer is a media-processing library that provides an abstract model of a transformation that is based on a pipeline. Media flows in a defined direction from input to output. GStreamer has gained wide popularity in the digital video programming community through its ability to abstract the manipulation of different media; thus, simplifying the programming process. GStreamer makes it possible to write a general video or music player that can support many different formats and networks. Most operations are performed by plugins, rather than by the GStreamer core. Figure 2 displays this media pipeline concept:

Dm81xx gstreamer decode pipeline 2.jpg


The TI DM81xx GStreamer package contains the following directories, under the top level directory "gstreamer_ti_dm81xx".

opensource_build

This directory contains the open source packages used to build GStreamer and the TI plugin. This includes packages such as glib as well as the GStreamer core libraries and plugin libraries. These packages are included in the GStreamer release for convenience so there is no need to download them from various locations, and this way developers know they have all dependent packages that were used during system testing.

There is also a directory called patchfiles in this directory which contains various patches that are used when building the open source packages for TI processors. These patches may disable features that impact performance on embedded processors or work around build/configuration issues when cross-compiling for the ARM processor. Some patches fix issues or enhance opensource elements to function well in the system along with TI Gstreamer elements.

ti_build

This directory contains the TI GStreamer plugin code. The code for DM81xx gstreamer plugin is located at ti_build/gst-openmax. This plugin contains multiple elements which allow for encoding/decoding of audio/video/imaging files.

gstreamer_demo

This directory contains a series of scripts that can be used to test the functionality of the various TI GStreamer elements. There is a directory for each supported processor. These scripts would be installed on the target file system during the build process at /opt/gstreamer_demo/<platform>.

How to run a gstreamer pipeline on DM81xx EVM in 7 steps

  1. Get a DM8148/DM8168 EVM
  2. Download EZSDK from here
  3. Install EZSDK, Installation FAQ
  4. Pull Gstreamer for DM81xx platform from http://gstreamer.ti.com. Details here.
  5. Build and Install Gstreamer – Details here
  6. Setup the EVM with newly built EZSDK filesystem, that includes gstreamer binaries from the previous step
  7. Run the gstreamer pipelines – example pipelines are availble at /opt/gstreamer_demo/<platform> on target filesystem

Not able to get it working.... maybe you missed something, refer FAQ here!

Still not able to get it working... maybe we missed something, report the issues here

Download

The TI GStreamer project is hosted at http://gstreamer.ti.com. From that page you can checkout the source code from the SVN repository.

Note that the website http://gstreamer.ti.com also hosts the other TI Gstreamer project, the DMAI Gstreamer Plugin. The source code for this project (DM81xx plugin) is hosted in the same SVN repository under the directory trunk/gstreamer_ti_dm81xx. Whereas DMAI gstreamer project is available at trunk/gstreamer_ti.


Gstremer for DM81xx depends on EZSDK, so based on the version of EZSDK used, this table show the tested versions of gstreamer on the SVN repository that should be used. Some EZSDK releases come with pre-built gstreamer binaries and source code in the package. The exsting gstreamer package in the EZSDK can be upgraded with newer gstreamer from SVN if desired.

EZSDK version
Gstreamer version packaged with EZSDK
SVN revision number on trunk
Latest release TAG (tested on this EZSDK version)
     5_02_02_60
           Not present
                            974

     5_03_00_09
           Not present
                            983

        TAG_DM81XX_RELEASE_0_03

     5_03_01_15
TAG_DM81XX_RELEASE_0_03
1026
TAG_GST_DM81XX_00_04_00_00

To checkout (download) source code using revision number from the trunk in SVN:

svn checkout -r <revision> --username <username> https://gstreamer.ti.com/svn/gstreamer_ti/trunk/gstreamer_ti_dm81xx 

To checkout (download) source code from a Tagged version in SVN:

svn checkout --username <username> https://gstreamer.ti.com/svn/gstreamer_ti/tags/<TAG Name> 


Upgrading gstreamer in EZSDK

EZSDK versions 5_03_01_15 onwards includes:

  1. Complete set of pre-built gstreamer core libraries and opensource plugins. The source code is not included in the standard EZSDK installation, could be obtained in DVD form if required.
  2. Source code for TI gst-openmax plugin - located at $(EZSDK)/component-sources

Upgrading the above requires different procedures. Since gst-openmax source code and build is already part of EZSDK standard package, it is relatively easy to replace the source package with a new one. For gst core libraries and opensource plugins, the suggested method is to build them using make system of gstreamer.ti.com and copy the required binaries (.so files) into the target filesystem.

  • Upgrading gst-openmax plugin

The steps below outlines how to download and build gst-openmax plugin from the EZSDK installation.

  1. Download the gst-openmax release from here
  2. The gst-openmax tarball file is named gst-openmax-<version>.tar.gz. Untar this file at any location.
  3. Alternatively, <svn_checkout_path>/gstreamer_ti_dm81xx/ti-build/gst-openmax could be used. In this case, "./autogen.sh"should be run before the next step.
  4. In $(EZSDK)/Rules.mk, Update the GST_OMX_INSTALL_DIR variable to the location of the gst-openmax package.
  5. Now from EZSDK base path, Make sure the components are compiled for Linux by executing "make components_linux".
  6. Build and install gst-openmax using "make gstomx gstomx_install"
  • Upgrading other opensource plugins or gstreamer core libraries
  1. Pull Gstreamer for DM81xx platform from http://gstreamer.ti.com. Details here.
  2. Build the package and install into a temporary location – Details here
  3. Copy the upgraded plugins of interest to EZSDK target filesystem's /usr/lib/gstreamer-0.10/

System Requirements

In order to build/install the TI GStreamer package you will need the following:

  • Host PC running Linux running Ubuntu 10.04 LTS 32-bit
  • EZSDK software for your processor
  • A cross-compile toolchain. This is CodeSourcery tools for ARM Cortex A8.
  • A target file system. You can use the sample NFS file system provided with the EVM.

How to Run the Demo on the Target

The previous build steps were done on the host file system. Now you can run the demo on the target file system.

  • Boot the hardware EVM. It is assumed that you have the file system mounted over NFS in U-Boot settings.
  • Once you are at the login prompt, type root and enter.
  • Go to the /opt/gstreamer_demo/<processor> directory where <processor> is the processor you have build the GStreamer package for (i.e. dm814x). This directory contains:
    • example_pipeline.sh - Contains example gstreamer pipelines pipelines and environment variable settings
  • Copy your media files (TS, AVI, AAC, MP3, MP4, etc) to the target file system.
  • Create a gstreamer pipeline and run it using gst-launch command.

Information about general GStreamer usage can be found at http://gstreamer.freedesktop.org

DM81xx gstreamer elements [status as of release 0.4]

Elements that are part of gst-openmax plugin (source code at ti_build/gst-openmax)

gstreamer

element

input
output
OpenMax component name
Description
Current Status
omx_mpeg4dec
MPEG-4 video bitstream
YUV 4:2:0 SP
OMX.TI.DUCATI.VIDDEC
Decode MPEG4 video
Available (alpha)
omx_h264dec
H.264 video bitstream
YUV 4:2:0 SP
OMX.TI.DUCATI.VIDDEC
Decode H.264 video
Available (alpha)
omx_mpeg2dec
MPEG-2 video bitstream
YUV 4:2:0 SP
OMX.TI.DUCATI.VIDDEC
Decode MPEG-2 video
Available (alpha)
omx_mjpegdec
MJPEG video bitstream
YUV 4:2:0 SP
OMX.TI.DUCATI.VIDDEC
Decode MJPEG video
Available (alpha)
omx_h264enc
YUV 4:2:0 SP
H.264 video bitstream
OMX.TI.DUCATI.VIDENC
Encodes H.264 video
Available (alpha)
omx_vc1dec
VC-1/WMV video bitstream
YUV 4:2:0 SP
OMX.TI.DUCATI.VIDDEC
Decodes VC-1/WMV video
Available (alpha)
omx_videosink
YUV 4:2:2 I

OMX.TI.VPSSM3.VFDC
Displays video on one of the outputs configured
Available (alpha)
omx_scaler
YUV 4:2:0 SP
YUV 4:2:2 I
OMX.TI.VPSSM3.VFPC.INDTXSCWB
Resizes and color converts video frames
Available (alpha)
omx_mdeiscaler, omx_hdeiscaler
YUV 4:2:0 SP
YUV 4:2:2 I and YUV 4:2:0 SP
OMX.TI.VPSSM3.VFPC.DEIMDUALOUT, OMX.TI.VPSSM3.VFPC.DEIHDUALOUT
De-interlaces, resizes and color converts video frames produces 2 outputs simultaneously - one for display, another for encoder
Available (alpha)
omx_noisefilter
YUV 4:2:2 I
YUV 4:2:0 SP
OMX.TI.VPSSM3.VFPC.NF
Used to filter camera input noise for video capture
Not tested
omx_ctrl


OMX.TI.VPSSM3.CTRL.TVP/DC
Configure external TVP capture and display controllers
Available (alpha)
omx_videomixer
YUV 4:2:0 SP
YUV 4:2:2 I
OMX.TI.VPSSM3.VFPC.INDTXSCWB
Mosaic N video inputs into 1.
Available (alpha)
omxbufferalloc



Buffer allocater element used to allocate buffers that can be passed to OMX elements. E.g. V4L2 capture uses this to allocate buffers, which then can be passed to omx_h264enc without incurring a buffer copy.
Available (alpha)
gstperf



Display video framerate, cpu utilization and other statistics
Available (alpha)


Other opensource elements frequently used for typical usecases such as decode and display, tested on DM81xx platforms.

gstreamer element name
function
Status on DM81xx platforms
alsasink
Audio output
Working
alsasrc
Audio Capture
Working
v4l2sink
Video display
Patched to add user pointer support, working.
v4l2src
Video capture
Patched to add user pointer support, working.
qtdemux
MP4, MOV demuxer
Working

mpegtsdemux, mpegpsdemux,

ffdemux_mpegts, ffdemux_mpeg

MPEG2 TS/PS demuxer
Working
h264parse
H.264 video frame chunker
Patched to work well with TI decoder (omx_h264dec) component, working
mpeg2parse
MPEG-2 video frame chunker
Patched to work well with TI decoder (omx_mpeg2dec) component, working
nal2bytestream_h264
H.264 NAL to Bytestream converter

New element to adapt H.264 from containers such as MP4 to be understood by TI Decoder element (omx_h264dec)

Obsoleted from 0.4 release, use h264parse instead.

ffdec_ac3
AC-3 audio decoder
Working
ffdec_mp3
MP3 audio decode
Working
faad
AAC audio decoder
Working
rtspsrc, rtph264depay, rtpmp2tpay, udpsrc, udpsink
Network media streaming/control protocol handlers Working


Licensing

  • The TI GStreamer plugin is licensed under the terms of the GNU LGPL v2.1. The licensing terms are:
This program is free software; you can redistribute it and/or modify it under the terms of the  
GNU Lesser General Public License as published by the Free Software Foundation version 2.1 of the License. 
  • The EZSDK software package used by the TI GStreamer port is covered under a seperate license. Please refer to the EZSDK click-wrapped license for more information on the terms of use for the EZSDK software.


NOTE: This licensing information only covers the TI GStreamer plugin, for information about the licensing of other GStreamer components please see licensing information for that component.

Roadmap

Release Version Planned elements Expected timeline Status SVN Revision Works with EZSDK release
0.2 Refer to current status 21st Oct 2011  Available      974
     5_02_02_60
0.3

MPEG-4 Decoder, VC-1 decoder, More test on DM8168

2nd Week Nov 2011 Available
     5_03_00_xx
0.4   MJPEG decoder, V4L2 Capture, De-interlacer 2nd Week December 2011 Available

     5_03_01_15
0.5 DSP audio, performance optimizations End of feb, 2012  Planned

More Information