MCSDK VIDEO 2.1 CODEC TEST FW User Guide

MCSDK Video

Version 2.x

Codec Integration and Test User Guide

Last updated: 11/08/2013

---

Introduction

Codec integration & test application is one of the builds provided by the MCSDK Video [Getting Started Guide]. It has been developed to facilitate the development and testing of stand-alone Codecs (including video, audio, and image codecs). It provides TFTP support over Ethernet for data I/O instead of slow freads and fwrites over JTAG. Standardized XDM compliant wrappers are provided to exercise codec APIs, with several pre-integrated video codecs as the example. It supports batch process regression test suite of codecs for multiple test vectors in a row. It supports codecs running on single core and multiple cores.

Specifically, this Codec Integration and Test User Guide provides information about the call flow of Codec Test Application, how to rebuild it (named as sv04 in the MCSDK Video), how to use the Codec Test Application on TI C66x multi-core DSPs, and how to plug-in a new XDM compliant codec into the framework.

Recommended Hardware Emulator:

1. Blackhawk XDS 560 or
2. Spectrum Digital XDS 560 or
3. Shannon XDS100 on board

Useful Tip 1. Please visit Getting Started Guide for general information on MCSDK Video, including product download and installation. 2. After running this Codec Integration and Test Application, it is recommended the user to experience other video demos provided by MCSDK Video, which includes packet based real-time video transcoding demos and PCIe based video demos. Please visit MCSDK Video Demo Guide for additional information on that.

Call Flow of Codec Integration and Test Application

The call flow of Codec Test Application is as follows:

• Platform initialization
• Network interface initialization

1. Read network configuration file via JTAG file I/O
2. Configure network channel for streaming, and local/remote network endpoints;

• Read test cases configuration file via JTAG file I/O
• Loop through all test cases

1. Parse test case configuration file, and set static and dynamic parameters
2. Codec initialization
3. Loop through all frames until end of file reached

• TFTP Get input file into DDR
• Process frames (decode or encode)
• TFTP Put generated bytes to TFTP server

In the above call flow, only codec initialization and frame processing are specific for individual codecs. The rest is common for all codecs.

Framework Folders and Make Instructions

Framework Folders

The following table lists the folders required for building Codec Codec Framework. It also highlights the folders which users expect to change when plugging in a new XDM compliant codec in the framework.

Top Level Folder	Brief Description	Users Expected to Change?
\dsp
	Build related perl files	NO
	Bootloader shared information with the application	NO
	Build framework files (ggcfg)	YES
	Hardware Abstraction Layer	NO
	Memory management and utilities	NO
	Main Makefile (mkrel)	YES for setupenvMsys.sh
	Network Driver Interface	NO
	Codec integration framework (siu)	YES
	Build files (ggcfg\build\sv04)	YES
\components	Video components	NO
\inc	Include files to talk to various components	NO

Make Instructions

Before making the Codec Test Application, please refer to Getting Started Guide for details on installing the MCSDK Video and its required tools.

When it is desired to change one (or more) of the codec algorithms or add a new codec algorithm, the following steps are needed in order to change and rebuild the Codec Test Application.

• Make a copy of dsp directory and modify desired files in their home directories and save.
• Configure environment: the build directory is dsp\mkrel. Run the batch file "\dsp\mkrel\setupenvMsys.bat bypass" to configure the environment. The batch file calls "\dsp\mkrel\setupenvMsys.sh" which will check if all the required components and tools are available at the specified locations.
• Run dsp/mkrel/makefile to build DSP code: make sv04.
• The build procedure will produce directory: \dsp\mkrel\sv04 with the following files: sv04.out, sv04.map, readme_sv04.txt

Useful Tip 1. If a source debugger requires all source files to be combined into a single directory, "FLAT=YES" may be added in the make command line, which will create the directory mkrel\sv04\flat containing all source and header files used for the build. 2. After making the first build, if there is no source file change in \components directory, "RTSCBYPASS=YES" may be added in the make command line, which will bypass compiling the components. If there is no source file change in dsp\ggcfg\build\hdg\sv04\bios directory, "BIOSCFGPKGBYPASS=YES" may be added in the make command line, which will bypass compiling the BIOS configuration package.

Test Instructions

After sv04.out file is built, the following steps can be used in order to run sv04 on TI C66x multi-core DSPs.

TFTP server installation on PC

Download and install TFTP server v3.35 from TFTP Download Link.

Codec Test Instructions

1. Prepare EVM

• Connect TI C6678 EVM to the PC via switch.
• Set the boot pins on the EVM to one of the boot modes (e.g., ROM Ethernet Boot) as specified at Boot Mode Dip Switch Settings

Note: General instructions of setting up your TMDXEVM6678L Evaluation Module (EVM) can be found at the link TMDXEVM6678L EVM Hardware Setup. Browsing this link to become familiar with the Hardware Setup of TMDXEVM6678L EVM is strongly recommended.

2. Take DSP out of Reset

This can be done by unplugging the EVM power and plugging it in again.

3. Prepare CCS

Useful Tip General instructions of connecting the JTAG to the EVM can be found at http://processors.wiki.ti.com/index.php/BIOS_MCSDK_2.0_Getting_Started_Guide#Use_JTAG_to_Load_the_Application. Browsing this link to become familiar with the procedure is strongly recommended.

• Open CCS
• Launch the target configuration
• On Core 0: load the GEL file "<CCS_INSTALL_LOCATION>\ccsv5\ccs_base_5.0.3.00028\emulation\boards\evmc6678l\gel\evmc6678l.gel" into CCS by selecting GEL Files under the Tools menu, and then select Load GEL… as shown below

• Connect Core 0 (via selecting Connect Target under the Target menu)
• Connect other cores when needed for multi-core codecs

4. Prepare PC

• Make sure the MTU (Maximum Transmission Unit) size for the network interface is 1500 so that TFTP server is not sending fragmented packets. This is because DSP does not support re-assembly of fragmented IP packets.

• Start TFTP Server on the PC. Click on the settings button and increase the timeout duration in the Settings window. A timeout of 20 seconds should be sufficient.

• Enable ARP entry on the PC so that it can send packets to EVM. This is done by opening the command prompt and entering the command: arp –s <dsp_ip_addr> <dsp_mac_addr>. Note that dsp_ip_addr and dsp_mac_addr are the same as the localIpAddress and localMacAddress, respectively, both of which can be found in the configuration file dsp\siu\vct\testVecs\tftp.cfg. In this tftp.cfg, serverIpAddress and serverMacAddress specify the IP and MAC address of the PC, which can be found be running the command ipconfig /all. Since both the DSP and the PC needs to be in the same subnet, specify DSP's IP address (i.e., localIpAddress) correspondingly after getting the PC's IP address (i.e., serverIpAddress). The DSP's MAC address (i.e., localMacAddress) can be any valid MAC address, such as the physical MAC address of the DSP or the one included in the provided tftp.cfg, as long as the same MAC address is not used anywhere else in the same subnet.

• Prepare input clip(s) for testing. Copy the input clip(s) to the directory which is shown in Current Directory of the TFTP server or set the Current Directory of the TFTP server to the directory including the input clip(s). In this release of MCSDK Video, input clips are provided for all codecs at dsp\siu\vct\testVecs\<codec>\input directory, except for AVCIU encode, JPEG2K encode, and JPEG2K transcode. To create input clips for these three tests (AVCIU encode, JPEG2K encode, and JPEG2K transcode), please follow steps specified at Creating 1920x1080 10bit 4:2:2 YUV clips.

After the clips are ready, please update dsp\siu\vct\testVecs\<codec>\config\multiclip.cfg with the following information: input file name (the first row), output file name(the first row), number of frames to be decoded or encoded (the third row). Multiple groups of [input file name, output file name, number of frames] can be specified in this multiclip.cfg for automated testing of multiple clips. Any number of clips can be specified.

5. Run the Codec Test Application

• Do system reset for Core 0 (via selecting Reset -> System Reset under the Target menu)
• For multi-core codecs, do CPU reset for the other cores (via selecting Reset -> CPU Reset under the Target menu)
• On Core 0, run the GEL file from the Scripts menu by selecting EVMTCI6608 Init Functions -> Global_Default_Setup. Wait until the following message is displayed: “C66xx_0: GEL Output: Global Default Setup... Done.”
• Load the DSP image to Core 0 by selecting Load Program... under the Target menu. Find the image sv04.out in the mkrel/sv04/flat directory.
• For multi-core codecs, load the same image to the other cores.
• Make sure the path in the function siuVctRunTask() (siuVctRun2.c) is correct for the config files (testVecs.cfg and tftp.cfg under siu/vct/testVecs directory). The path can be relative to the location of the DSP image mkrel/sv04/flat/sv04.out, and is by default “../../../siu/vct/testVecs/testVecs.cfg” and “../../../siu/vct/testVecs/tftp.cfg”. If the path is incorrect, the config files must be placed at that location, or the path can be changed which would require the image to be rebuilt.
• Make sure TFTP transfer is running while the codec is processing the data.
• Make sure that the configuration file “siu/vct/testVecs/tftp.cfg” is correct. The configuration file contains the PC’s IP address and MAC address and the DSP’s IP address and MAC address.
• Make sure that the configuration file “siu/vct/testVecs/testVecs.cfg” points to the configuration files for the testing to be conducted. The codecs which are integrated include: AVC Intra and Ultra encode, JPEG2K encode, JPEG2K decode, and JPEG2K transcode, H264BPMP Decoder, H264HP Decoder, MPEG2 Decoder, MPEG4 Decoder, JPEG Decoder, H264HP encoder, MPEG2 Encoder, MPEG4 Encoder, JPEG Encoder. In MCSDK Video 2.2, HEVC encoder and HEVC decoder are added.
• Make sure codec configuration files (siu\vct\testVecs\<codec>\config\multiclip.cfg & codecParams.cfg) are matching with the input clip and the testing to be conducted. As mentioned earlier, multiclip.cfg includes input file name (the first row), output file name(the first row), number of frames to be decoded or encoded (the third row). In this release, codec static and dynamic parameters are exposed along with codec name and core team configuration in siu\vct\testVecs\<codec>\config\codecParams.cfg for individual test cases. codecParams.cfg packaged is matching with the input clips provided in the release. For AVC Intra and Ultra encode, JPEG2K encode, and JPEG2K transcode (encode and then decode), codecParams.cfg packaged is for 1920x1080 10-bit 4:2:2 YUV input. Please make sure the codec parameters are correctly supplied when updating input clips and testing requirements.
• Changes to the configuration files (testVecs.cfg, tftp.cfg, and multiclip.cfg & codecParams.cfg for individual codecs) do not require anything to be rebuilt.
• On Core 0: run the codec test by selecting Target -> Run. For multi-core codecs, then run the other cores. The application will obtain the input file via TFTP. Once the file transfer has been started, the user’s codec will process the data and output frames will be sent to the file specified on the TFTP server.

Integrating new Codec into the build

In MCSDK Video 2.1, all the C6678 codec available at C6678 Codecs have been integrated. In MCSDK Video 2.2, HEVC encoder and HEVC decoder are further integrated. Please follow the following steps to add a new codec.

Codec Algorithm

New Codec can be plug and play in the existing sv04 infrastructure if it is XDM 0.9 or XDM 2.0 compliant encoder or XDM 1.3 compliant decoder. For the product release of MCSDK Video, it further supports XDM 1.0 compliant video encoder and XDM 1.0 compliant video decoder, as well as XDM 1.0 compliant image encoder and XDM 1.0 compliant image decoder. If the codec is not compliant with any of the above XDM versions, XDM wrapper needs to be created using the existing three XDM wrapper as references.

Codec algorithm source code can be compiled either via make or CCS to make a library. Once the Codec library with public API files is ready, need to add its environment to "\dsp\mkrel\setupenvMsys.sh" following the example below.

# H264BP encoder

VIDEO_H264_ENC_VERSION="C66x_h264venc_01_24_00_01_ELF"

VIDEO_H264_ENC_RUNPATH="$VIDEOBASE/$VIDEO_H264_ENC_VERSION"

make_shortname "VIDEO_H264_ENC_RUNPATH"

VIDEO_H264_ENC_SRCPATH="$VIDEO_H264_ENC_RUNPATH"

check_exist "VIDEO_H264_ENC_SRCPATH" "/packages/ti/sdo/codecs/h264venc/ih264venc.h"

COPY_TOOLS_LIST="$COPY_TOOLS_LIST VIDEO_H264_ENC"

...

export VIDEO_H264_ENC_DOS="`echo $VIDEO_H264_ENC_RUNPATH | $make_dos_sed_cmd`/packages"

Then, add the new codec path to "FXDCINC" in "\dsp\mkrel\c64x\makedefs.mk", following any of the video codecs as the example from the code below:

FXDCINC = $(GGROOT)/mkrel/rtsc;$(TOOLSXDCDOS)/packages;$(BIOS_MCSDK_PDK6678_DOS);$(NWAL_C66X_DOS);$(VIDEO_MPEG4_ENC_DOS);$(VIDEO_MPEG4_DEC_DOS);$(VIDEO_MPEG2_DEC_DOS);$(VIDEO_H264_ENC_DOS);$(VIDEO_H264_DEC_DOS);$(VIDEO_H264HP_DEC_DOS);$(VIDEO_AVCIU_ENC_DOS);$(VIDEO_J2K_ENC_DOS);$(VIDEO_J2K_DEC_DOS);$(DSP_INSTALL_DIR_DOS)

Codec Client Glue Code

In order to glue a new codec to the framework, the following files must be written and placed as indicated:

• Decoder: vct<codec>DecClient.c and vct<codec>DecClient.h files in siu\vct\codec\decoder\codec folder
• Encoder: vct<codec>EncClient.c and vct<codec>EncClient.h files in siu\vct\codec\encoder\codec folder

JPEG2K decoder and H.264BP encoder client code can be referred to as example. <codec>encAPI and <codec>decAPI for the new codec must be defined according to the corresponding data structures defined in siuVctCodecAPI.h.

Update siuVctSupportedCodecs.c

Edit the file siuVctSupportedCodecs.c. Include the Client Glue code header file vct<codec>EncClient.h or vct<codec>DecClient.h and update the data structure encoderAPI_t or decoderAPI_t with the new API definition.

Edit makefile

siu\c64x\make\makefile must be edited to include the new header files and also compile the client glue code. Search for “avciu” in this makefile and make similar changes

Update Linker Command File

ggcfg\build\hdg\sv04\ggvf0.becmd must be updated to include the new Codec library

Create Config Files

Create multiClip.cfg and codecParams.cfg for the codec (under siu/vct/testVecs/<codec>/config), and update the codecName in the codecParams.cfg with the string name that is used in the Glue Code. To test the newly added codec, modify siu/vct/testVecs/testVecs.cfg to update the path.

Cache

The following describes the Cache configuration in sv04 build and the APIs available to perform cache operations such as wirteback and invalidate.

Cacheability of DDR

Sections of DDR can be made cacheable or non-cacheable, prefetchable or non-prefetchable using the MAR registers and the granularity is 16MB. For the entire DDR of 512 MB on Shannon (0x80000000 – 0x9FFFFFFF), One 16MB DDR section (0x80000000 – 0x80FFFFFF) is configured as non-cacheable and non-prefetchable while all the other DDR sections are cacheable and prefetchable. The only non-cacheable and non-prefetchable DDR section is used for multi-core synchronization buffers for multi-core codecs. The MAR definition can be found from “TMS320C66x DSP CorePac User Guide” (Literature Number: SPRUGW0A). The DDR Cache setting can be changed by editing the file ggcfg\build\hdg\sv04\ggmemdef.bec in the structure vigdkMemoryContext_t.

Cacheability of L1P, L1D and L2

Cache configuration for L1P, L1D and L2 is done in the file ggcfg\build\hdg\sv04\ gghwcfg.h. Recommended configuration is to use HAL_L1P_CACHE_32K, HAL_L1D_CACHE_32K and HAL_L2_CACHE_64K.

Cache Control APIs

The following functions (defined in dsp\siu\osal\bios6\siuOsalBios6.c) are available to perform various cache operations. They are wrappers calling BIOS 6 Cache APIs.

• void siu_osal_wbinv_cache_all(void);
• void siu_osal_wbinv_cache(void *base, tulong length, bool wait);
• void siu_osal_inv_cache(void *base, tulong length, bool wait);
• void siu_osal_wb_cache(void *base, tulong length, bool wait);

Multi Core Video Interface APIs

In order to facilitate development of multi-core/multi-chip codecs, MCSDK Video provides multi-core video interface (ividmc) APIs. ividmc APIs allow application framework to set and implement the interface APIs. In video codec creation phase, the actual implementation of the APIs is passed through codec static parameters as a set of function pointers. Then, video codec is internally using only these APIs to do inter-core/inter-chip sync-up and/or communication.

MCSDK Video provides two versions of multi-core video interface (ividmc) APIs: ividmc and ividmc3. ividmc supports multi-core within a single chip, while ividmc3 supports multi-core and multi-chip. Their corresponding API files 'ividmc.h’ and 'ividmc3.h’ are located in dsp\siu\ividmc folder. The following describes some details of ividmc3 APIs.

Overall structure of ividmc3 APIs is as follows:

typedef struct IVIDMC3_s {
XDAS_Void *(*keyCreate) (XDAS_UInt8 *name, XDAS_Int32 user_id, IVIDMC3_KEY_SPACE_e key_space, 
                     XDAS_Int32 num_users, XDAS_Int32 *user_ids, IVIDMC3_KeyCfg_t *cfg);
XDAS_Int32 (*barrWait)(XDAS_Int32 user_id, XDAS_Void *barrHandle);
XDAS_Int32 *(*shmMap) (XDAS_Int32 user_id, XDAS_Void *shmemHandle);
XDAS_Int32 (*shmSync)(XDAS_Int32 user_id, XDAS_Void *shmemHandle, XDAS_Int32 *shmem_base,
                  XDAS_Int32 shmem_size, IVIDMC3_SYNC_ATTRIBS shmem_sync_attribs);
XDAS_Int32 (*shmSyncWait)(XDAS_Int32 user_id, XDAS_Void *shmemHandle, XDAS_Int32 *shmem_transid);
XDAS_Int32 (*lockAcquire)  (XDAS_Int32 user_id, XDAS_Void *lockHandle);
XDAS_Int32 (*lockRelease)  (XDAS_Int32 user_id, XDAS_Void *lockHandle);
XDAS_Int32 (*lockCheck)    (XDAS_Int32 user_id, XDAS_Void *lockHandle);
XDAS_Int32 (*mailBoxOpen)(XDAS_Void *mailBoxHandle);
XDAS_Int32 (*mailBoxWrite) (XDAS_Void *mailBoxHandle, XDAS_UInt8 *buf, XDAS_UInt32 size, XDAS_UInt32 trans_id);
XDAS_Int32 (*mailBoxRead) (XDAS_Void *mailBoxHandle, XDAS_UInt8 *buf, XDAS_UInt32 *size, XDAS_UInt32 *trans_id);
XDAS_Int32 (*mailBoxQuery) (XDAS_Void *mailBoxHandle);
XDAS_Int32 num_users;
IVIDMC3_TASK_e task_ID;
XDAS_Int32 user_id;
} IVIDMC3_t

Brief description of each item is as below:

• XDAS_Int32 user_id:
  core ID, including core ID inside the chip and also the chip ID
  
• IVIDMC3_TASK_e task_ID:
  core task identification, can be global master, chip local master, or slave
• num_users:
  number of cores in the team, i.e., the number of cores running together for processing the same stream
• XDAS_Void *(*keyCreate) (XDAS_UInt8 *name, XDAS_Int32 user_id, IVIDMC3_KEY_SPACE_e key_space, XDAS_Int32 num_users, XDAS_Int32 *user_ids, IVIDMC3_KeyCfg_t *cfg):
  key assignment for Barrier, Shmem, Locks and Mailboxes; Barrier, Shmem and Locks resources are also initialized.
• XDAS_Int32 (*barrWait)(XDAS_Int32 user_id, XDAS_Void *barrHandle):
  function call which will block until all other users have also called it
• XDAS_Int32 *(*shmMap) (XDAS_Int32 user_id, XDAS_Void *shmemHandle)
  dynamically allocates shared memory region identified with handle "shmemHandle"; if region with same "shmemHandle" is already allocated, its base address is provided
• XDAS_Int32 (*shmSync)(XDAS_Int32 user_id, XDAS_Void *shmemHandle, XDAS_Int32 *shmem_base, XDAS_Int32 shmem_size, IVIDMC3_SYNC_ATTRIBS shmem_sync_attribs):
  synchronizes shared memory region
• XDAS_Int32 (*shmSyncWait)(XDAS_Int32 user_id, XDAS_Void *shmemHandle, XDAS_Int32 *shmem_transid):
  waits for synchronization operation to finish
• XDAS_Int32 (*lockAcquire) (XDAS_Int32 user_id, XDAS_Void *lockHandle):
  critical region lock acquire in multi-core environment; if spinlock has been succesfully acquired, this function returns 1; if same spinlock has been already acquired by another participant, this function returns 0
• XDAS_Int32 (*lockRelease) (XDAS_Int32 user_id, XDAS_Void *lockHandle):
  critical region lock release in multi-core environment
• XDAS_Int32 (*lockCheck) (XDAS_Int32 user_id, XDAS_Void *lockHandle):
  critical region lock check to see if any of the users currently have acquired the lock
• XDAS_Int32 (*mailBoxOpen)(XDAS_Void *mailBoxHandle): 
  opens a mailBox; this is a blocking call, it waits untill the mailBox creater is ready
• XDAS_Int32 (*mailBoxWrite) (XDAS_Void *mailBoxHandle, XDAS_UInt8 *buf, XDAS_UInt32 size, XDAS_UInt32 trans_id):
  writes into a mailBox to deliver a message to remote; it is a non blocking call 
• XDAS_Int32 (*mailBoxRead) (XDAS_Void *mailBoxHandle, XDAS_UInt8 *buf, XDAS_UInt32 *size, XDAS_UInt32 *trans_id):
  reads from a mailBox; it is a non blocking call
• XDAS_Int32 (*mailBoxQuery) (XDAS_Void *mailBoxHandle):
  polls mailBoxes for any available messages; it is a non blocking call

Useful Resources and Links

Product Download and Updates

For product download and updates, please visit the links listed in the table below.

	Product Download Link
MCSDK Video (2.1 GA) Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/mcsdk_video/latest/index_FDS.html
MCSDK Video (2.2 Alpha) Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/mcsdk_video/02_02_00_23/index_FDS.html http://software-dl.ti.com/sdoemb/sdoemb_public_sw/mcsdk_video/02_02_00_28/index_FDS.html
BIOS MCSDK Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/bios_mcsdk/02_01_02_05/index_FDS.html
Desktop Linux SDK Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/desktop_linux_sdk/01_00_00_07/index_FDS.html
C6678 Codec Download	http://software-dl.ti.com/dsps/dsps_public_sw/codecs/C6678/index.html

MCSDK Video Instructions

Please visit the links below to install MCSDK Video, run the video demos, and get the details on how the MCSDK Video demos are developed.

	Links for Linux	Links for Windows
Getting Started Guide	MCSDK Video Getting Started for Linux Desktop Linux SDK Getting Started	MCSDK Video Getting Started for Windows
Demo Guide	Run PCIe based Demos on Advantech DSPC-8681E & DSPC-8682E	Run TFTP based Demos on TMDXEVM6678LXE
Development Guide	Develop PCIe based Demos in MCSDK Video	Develop TFTP based Demos in MCSDK Video

Technical Support

For technical discussions and issues, please visit the links listed in the table below.

	Forum/Wiki Link
C66x Multicore forum	http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639.aspx
Multimedia Software Codecs forum	http://e2e.ti.com/support/embedded/multimedia_software_codecs/default.aspx
TI-RTOS forum	http://e2e.ti.com/support/embedded/f/355.aspx
Code Composer Studio forum	http://e2e.ti.com/support/development_tools/code_composer_studio/f/81/t/3131.aspx
TI C/C++ Compiler forum	http://e2e.ti.com/support/development_tools/compiler/f/343/t/34317.aspx
Embedded Processors wiki	http://processors.wiki.ti.com

Note: When asking for help in the forum you should tag your posts in the Subject with “MCSDK VIDEO”, the part number (e.g. “C6678”) and additionally the component (e.g. “NWAL”).