DM81xx Gstreamer Plugin

Overview

The TI DM81xx GStreamer plug-in uses TI implemented OpenMax development framework to simplify applications use HDVPSS, C674x DSP, and HDVICP2 hardware accelerators on TI's DM816x/DM814x DaVinci™ Digital Media Processors. GStreamer is run as an application on the ARM processor and through the use of OpenMax IL APIs is able to interact with the HDVPSS, C674x DSP and HDVICP2 to acquire, render or transform audio and video streams.

High level features and usecases

• Supported Video decoders – MPEG-2, H.264 BP/MP/HP, MPEG-4 SP/ASP, VC1, MJPEG
• Video Post processing/Display/Capture – Scalar & Color Conversion, De-interlacer, V4L2 display, V4L2 capture
• Audio decoders/encoders
  • on C674x DSP - MP3, AAC-LC, AC3 decoders, AAC-LC encoder
    
  • on ARM Cortex A8 - Several open-source audio codecs
• Support for decode and display pipelines
  • Playback pipelines for containers (MP4, MOV, TS, PS, VOB, ASF) with above listed video codecs and wide range of audio codecs
• Support for A/V capture + encode pipelines with H.264 video and AAC-LC audio
• Support for transcode pipelines
  • H264 to H264 transcode (different resolution/framerate etc)
    
  • MPEG-2 to H264 transcode
• Support for multi-channel network video receiver pipelines (with Mosaic display)
  • 4 x RTP Streams =>Decode => Video Mixer (4=>1 Mosaic) => Display

DM81xx Gstreamer System Architecture

Components

GStreamer is a media-processing library that provides an abstract model of a transformation that is based on a pipeline. Media flows in a defined direction from input to output. GStreamer has gained wide popularity in the digital video programming community through its ability to abstract the manipulation of different media; thus, simplifying the programming process. GStreamer makes it possible to write a general video or music player that can support many different formats and networks. Most operations are performed by plugins, rather than by the GStreamer core. Figure 2 displays this media pipeline concept:

The TI DM81xx GStreamer package contains the following directories, under the top level directory "gstreamer_ti_dm81xx".

opensource_build

This directory contains the open source packages used to build GStreamer and the TI plugin. This includes packages such as glib as well as the GStreamer core libraries and plugin libraries. These packages are included in the GStreamer release for convenience so there is no need to download them from various locations, and this way developers know they have all dependent packages that were used during system testing.

There is also a directory called patchfiles in this directory which contains various patches that are used when building the open source packages for TI processors. These patches may disable features that impact performance on embedded processors or work around build/configuration issues when cross-compiling for the ARM processor. Some patches fix issues or enhance opensource elements to function well in the system along with TI Gstreamer elements.

ti_build

This directory contains the TI GStreamer plugin code. The code for DM81xx gstreamer plugin is located at ti_build/gst-openmax. This plugin contains multiple elements which allow for encoding/decoding of audio/video/imaging files.

gstreamer_demo

This directory contains a series of scripts that can be used to test the functionality of the various TI GStreamer elements. There is a directory for each supported processor. These scripts would be installed on the target file system during the build process at /opt/gstreamer_demo/<platform>.

How to run a gstreamer pipeline on DM81xx EVM in 7 steps

1. Get a DM8148/DM8168 EVM
   
2. Download EZSDK from here
   
3. Install EZSDK, Installation FAQ
   
4. Pull Gstreamer for DM81xx platform from http://gstreamer.ti.com. Details here.
   
5. Build and Install Gstreamer – Details here
   
6. Setup the EVM with newly built EZSDK filesystem, that includes gstreamer binaries from the previous step
   
7. Run the gstreamer pipelines – example pipelines are availble at /opt/gstreamer_demo/<platform> on target filesystem
   

Not able to get it working.... maybe you missed something, refer FAQ here!

Still not able to get it working... maybe we missed something, report the issues here

Download

The TI GStreamer project is hosted at http://gstreamer.ti.com. From that page you can checkout the source code from the SVN repository.

Note that the website http://gstreamer.ti.com also hosts the other TI Gstreamer project, the DMAI Gstreamer Plugin. The source code for this project (DM81xx plugin) is hosted in the same SVN repository under the directory trunk/gstreamer_ti_dm81xx. Whereas DMAI gstreamer project is available at trunk/gstreamer_ti.

Gstremer for DM81xx depends on EZSDK, so based on the version of EZSDK used, this table show the tested versions of gstreamer on the SVN repository that should be used. Some EZSDK releases come with pre-built gstreamer binaries and source code in the package. The exsting gstreamer package in the EZSDK can be upgraded with newer gstreamer from SVN if desired.

EZSDK version	Gstreamer version packaged with EZSDK	SVN revision number on trunk	Latest release TAG (tested on this EZSDK version)
5_02_02_60	Not present	974
5_03_00_09	Not present	983	TAG_DM81XX_RELEASE_0_03
5_03_01_15	TAG_DM81XX_RELEASE_0_03	1026	TAG_GST_DM81XX_00_04_00_00

To checkout (download) source code using revision number from the trunk in SVN:

svn checkout -r <revision> --username <username> https://gstreamer.ti.com/svn/gstreamer_ti/trunk/gstreamer_ti_dm81xx 

To checkout (download) source code from a Tagged version in SVN:

svn checkout --username <username> https://gstreamer.ti.com/svn/gstreamer_ti/tags/<TAG Name> 

Upgrading gstreamer in EZSDK

EZSDK versions 5_03_01_15 onwards includes:

1. Complete set of pre-built gstreamer core libraries and opensource plugins. The source code is not included in the standard EZSDK installation, could be obtained in DVD form if required.
   
2. Source code for TI gst-openmax plugin - located at $(EZSDK)/component-sources
   

Upgrading the above requires different procedures. Since gst-openmax source code and build is already part of EZSDK standard package, it is relatively easy to replace the source package with a new one. For gst core libraries and opensource plugins, the suggested method is to build them using make system of gstreamer.ti.com and copy the required binaries (.so files) into the target filesystem.

• Upgrading gst-openmax plugin

The steps below outlines how to download and build gst-openmax plugin from the EZSDK installation.

1. Download the gst-openmax release from here
   
2. The gst-openmax tarball file is named gst-openmax-<version>.tar.gz. Untar this file at any location.
   
3. Alternatively, <svn_checkout_path>/gstreamer_ti_dm81xx/ti-build/gst-openmax could be used. In this case, "./autogen.sh"should be run before the next step.
4. In $(EZSDK)/Rules.mk, Update the GST_OMX_INSTALL_DIR variable to the location of the gst-openmax package.
   
5. Now from EZSDK base path, Make sure the components are compiled for Linux by executing "make components_linux".
   
6. Build and install gst-openmax using "make gstomx gstomx_install"
   

• Upgrading other opensource plugins or gstreamer core libraries

1. Pull Gstreamer for DM81xx platform from http://gstreamer.ti.com. Details here.
   
2. Build the package and install into a temporary location – Details here
   
3. Copy the upgraded plugins of interest to EZSDK target filesystem's /usr/lib/gstreamer-0.10/
   

System Requirements

In order to build/install the TI GStreamer package you will need the following:

• Host PC running Linux running Ubuntu 10.04 LTS 32-bit
  
• EZSDK software for your processor
• A cross-compile toolchain. This is CodeSourcery tools for ARM Cortex A8.
• A target file system. You can use the sample NFS file system provided with the EVM.

How to Run the Demo on the Target

The previous build steps were done on the host file system. Now you can run the demo on the target file system.

• Boot the hardware EVM. It is assumed that you have the file system mounted over NFS in U-Boot settings.
• Once you are at the login prompt, type root and enter.
• Go to the /opt/gstreamer_demo/<processor> directory where <processor> is the processor you have build the GStreamer package for (i.e. dm814x). This directory contains:
  
  • example_pipeline.sh - Contains example gstreamer pipelines pipelines and environment variable settings
    
• Copy your media files (TS, AVI, AAC, MP3, MP4, etc) to the target file system.
• Create a gstreamer pipeline and run it using gst-launch command.

Information about general GStreamer usage can be found at http://gstreamer.freedesktop.org

DM81xx gstreamer elements [status as of release 0.4]

Elements that are part of gst-openmax plugin (source code at ti_build/gst-openmax)

gstreamer element	input	output	OpenMax component name	Description	Current Status
omx_mpeg4dec	MPEG-4 video bitstream	YUV 4:2:0 SP	OMX.TI.DUCATI.VIDDEC	Decode MPEG4 video	Available (alpha)
omx_h264dec	H.264 video bitstream	YUV 4:2:0 SP	OMX.TI.DUCATI.VIDDEC	Decode H.264 video	Available (alpha)
omx_mpeg2dec	MPEG-2 video bitstream	YUV 4:2:0 SP	OMX.TI.DUCATI.VIDDEC	Decode MPEG-2 video	Available (alpha)
omx_mjpegdec	MJPEG video bitstream	YUV 4:2:0 SP	OMX.TI.DUCATI.VIDDEC	Decode MJPEG video	Available (alpha)
omx_h264enc	YUV 4:2:0 SP	H.264 video bitstream	OMX.TI.DUCATI.VIDENC	Encodes H.264 video	Available (alpha)
omx_vc1dec	VC-1/WMV video bitstream	YUV 4:2:0 SP	OMX.TI.DUCATI.VIDDEC	Decodes VC-1/WMV video	Available (alpha)
omx_videosink	YUV 4:2:2 I		OMX.TI.VPSSM3.VFDC	Displays video on one of the outputs configured	Available (alpha)
omx_scaler	YUV 4:2:0 SP	YUV 4:2:2 I	OMX.TI.VPSSM3.VFPC.INDTXSCWB	Resizes and color converts video frames	Available (alpha)
omx_mdeiscaler, omx_hdeiscaler	YUV 4:2:0 SP	YUV 4:2:2 I and YUV 4:2:0 SP	OMX.TI.VPSSM3.VFPC.DEIMDUALOUT, OMX.TI.VPSSM3.VFPC.DEIHDUALOUT	De-interlaces, resizes and color converts video frames produces 2 outputs simultaneously - one for display, another for encoder	Available (alpha)
omx_noisefilter	YUV 4:2:2 I	YUV 4:2:0 SP	OMX.TI.VPSSM3.VFPC.NF	Used to filter camera input noise for video capture	Not tested
omx_ctrl			OMX.TI.VPSSM3.CTRL.TVP/DC	Configure external TVP capture and display controllers	Available (alpha)
omx_videomixer	YUV 4:2:0 SP	YUV 4:2:2 I	OMX.TI.VPSSM3.VFPC.INDTXSCWB	Mosaic N video inputs into 1.	Available (alpha)
omxbufferalloc				Buffer allocater element used to allocate buffers that can be passed to OMX elements. E.g. V4L2 capture uses this to allocate buffers, which then can be passed to omx_h264enc without incurring a buffer copy.	Available (alpha)
gstperf				Display video framerate, cpu utilization and other statistics	Available (alpha)

Other opensource elements frequently used for typical usecases such as decode and display, tested on DM81xx platforms.

gstreamer element name	function	Status on DM81xx platforms
alsasink	Audio output	Working
alsasrc	Audio Capture	Working
v4l2sink	Video display	Patched to add user pointer support, working.
v4l2src	Video capture	Patched to add user pointer support, working.
qtdemux	MP4, MOV demuxer	Working
mpegtsdemux, mpegpsdemux, ffdemux_mpegts, ffdemux_mpeg	MPEG2 TS/PS demuxer	Working
h264parse	H.264 video frame chunker	Patched to work well with TI decoder (omx_h264dec) component, working
mpeg2parse	MPEG-2 video frame chunker	Patched to work well with TI decoder (omx_mpeg2dec) component, working
nal2bytestream_h264	H.264 NAL to Bytestream converter	New element to adapt H.264 from containers such as MP4 to be understood by TI Decoder element (omx_h264dec) Obsoleted from 0.4 release, use h264parse instead.
ffdec_ac3	AC-3 audio decoder	Working
ffdec_mp3	MP3 audio decode	Working
faad	AAC audio decoder	Working
rtspsrc, rtph264depay, rtpmp2tpay, udpsrc, udpsink	Network media streaming/control protocol handlers	Working

Licensing

• The TI GStreamer plugin is licensed under the terms of the GNU LGPL v2.1. The licensing terms are:

This program is free software; you can redistribute it and/or modify it under the terms of the  
GNU Lesser General Public License as published by the Free Software Foundation version 2.1 of the License. 

• The EZSDK software package used by the TI GStreamer port is covered under a seperate license. Please refer to the EZSDK click-wrapped license for more information on the terms of use for the EZSDK software.

NOTE: This licensing information only covers the TI GStreamer plugin, for information about the licensing of other GStreamer components please see licensing information for that component.

Roadmap

Release Version	Planned elements	Expected timeline	Status	SVN Revision	Works with EZSDK release
0.2	Refer to current status	21st Oct 2011	Available	974	5_02_02_60
0.3	MPEG-4 Decoder, VC-1 decoder, More test on DM8168	2nd Week Nov 2011	Available		5_03_00_xx
0.4	MJPEG decoder, V4L2 Capture, De-interlacer	2nd Week December 2011	Available		5_03_01_15
0.5	DSP audio, performance optimizations	End of feb, 2012	Planned

More Information

• OpenMax development Guide
  
• Go to http://gstreamer.ti.com for all project-related information, including:
  • Current Status
  • TODOs
  • HOWTOs
  • Support
  • Issue Tracking
    
• DM81xx Gstreamer FAQ