MCSDK VIDEO 2.x VA Algorithm Integration Guide

MCSDK Video

Version 2.x

Video Analytics Algorithm Integration Guide

Last updated: 07/02/2014

---

Introduction

This wiki page describes the changes needed to integrate a video analytics algorithm into MCSDK Video 2.2 framework. The changes include both the host application and the DSP build.

Host Application Changes

MCSDK Video Analytics Use Case Dataflow The dataflow of OOB Video analytics demo is given below. <syntaxhighlight lang='javascript'>

  /* Dataflow */

  INPUT THREAD ---------------------------------> Tx Thread --> HOST2DSP MailBox ---+
     |                                                                              |
     +-------------------+                                                         DSP
                         |                                                          |
  OUTPUT THREAD <-- FrameOverlay <--- Reorder <-- Rx Thread <-- DSP2HOST MailBox ---+

</syntaxhighlight> Note: Frame Overlay thread takes input frame from Input thread and overlays algorithm metaData (received from Reorder queue) on it. The modified frame could be written on Disk or could be displayed on screen.

     User needs to update FrameOverlayProcess() function for any new algorithm.

Adding algorithm into the list of supported Algorithms
User needs to add their algorithm in ~/mcsdk_video_x_x_x_x/host/src/algs/va/vaAlgs.h file. Add information in below format. Refer example of DMVAL algorithm for the same.

<syntaxhighlight lang='javascript'> /* Keep alg_key and alg_name as same. Give appropriate name to uniquely identify your algorithm.

  Use same name in demo script at run time to choose right algorithm */

typedef struct {

 const char  *alg_key;
 const char  *alg_name;
 int32_t      input_mem_region;    /* Input region. Use DSP_MEM_REGION_DDR  */
 int32_t      output_mem_region;   /* Output region. Use DSP_MEM_REGION_DDR  */
 void       (*dynCfg);             /* Dynamic Config if any. Not used. Ignore it. */

} algInfo_t; </syntaxhighlight>

Host I/O Memory Management

For video analytics demo, total 8MB is required for 16-Chan Video Analytics @QVGA resolution. Host input memory requirement is proportional to the input frame resolution and no. of channels to be supported. Input frame resolution is determined from command line input parameter (see below Demo Script Section). Host o/p buffer size varies for different algorithms. Host O/p memory is proportional to the algorithm metaData (result) size and no.of channels. Host I/O buffer size could be different for different channel (in multi-channel use case) but worst case buffer size (among all channels) is used for Host I/O buffer memory allocation. See below code snippets to add/update I/O buffer size for new algorithm type.

<syntaxhighlight lang='javascript'> /* Host I/O buffer size computation. See ~/host/src/demoCfgInit.c file. */ /* For VADMVAL algorithm, input is YUV420 format and O/p is the size of O/p metaData struct. */

   if (strcmp(comp->cfg.algorithm,"VADMVAL")   == 0)
   {
     chan->cfg.rawFrameSize = (chan->cfg.width * chan->cfg.height * 3) >> 1;
     chan->cfg.opBuffSize   = sizeof(va_dmval_result);
   }

/* If opBuffSize is set to -1, rawFrameSize (I/p frame size) is used for O/p buffer computation.*/

</syntaxhighlight>

Adding Demo Scripts
To run any demo, user is required to provide algorithm configuration parameters, I/O file names and other configurations(E.g. DSP Binaries, Core Mapping etc.). In MCSDK video demos, all such information is captured in different demo scripts. Demos scripts are divided into Video, VA and common scripts. Each demo example comes with scripts that are used to install and run demo.

• All installation scripts reside in ~/mcsdk_video_x_x_x_x/demos/ path. The purpose of these scripts is to install and uninstall demo config and scripts required to run demos(Read below).
• All other demo scripts are placed in ~/mcsdk_video_x_x_x_x/host/scripts/ folder. All the configuration files are kept in this path.

Please go through below guidelines to add demo scripts for new demo/algorithm.

1. Create a copy of ~demos/va/pcie_dmval_demos/ in ~/demos/va folder. Update install.sh and uninstall.sh scripts. Append details of new algorithm in install.sh script. These are required to install and uninstall demo scripts. Depending upon demo/algorithm type appropriate scripts are installed from ~/host/scripts/ in this path.
2. Create a copy of ~/host/scripts/va/pcie_dmval_demos/ in ~/host/scripts/va folder. User can modify DMVAL scripts as mentioned below as per use case requirement:
   • multiclip*.cfg: Contains input (YUV) and output (algorithm result) file path.
   • vaParams*.cfg: Contains algorithm parameters. Host app copies parameters into buffer and send it to DSP. DSP application parses it and extracts create parameter.
   • demoSave2File*.sh/demoDisplay*.sh : Contains scripts to run demo. Demo run command. Description is given below:

<syntaxhighlight lang='javascript'>

Algorithm Config File format (vaParam*.cfg)

Modify vaParams*.cfg when algorithm, static/dynamic parameters need to be updated.
Use the following format, and Comment shows the parameter name as in the codec public header files.
   <ParameterName> = <ParameterValue> # Comment

</syntaxhighlight>

<syntaxhighlight lang='javascript'> Single Channel MCVA Script Format

 $MCSDK_VIDEO_HOSTBIN_DIR/demo_c667x --dsp-image=$MCSDK_VIDEO_DSPIMAGE_DIR/sv05.out -v --channel="VA DMVAL" --chanid=0 -a VADMVAL
 -f $CLIP_CONFIG0 -c $CODEC_PARAMS0 -t 0 --width=320 --height=240

1. Path variables are set once user run setDemoEnv.sh. Cross check binary path if you have modified it. 2. --channel : Channel Name, Choose appropriate name. 3. --chanid  : Channel ID assigned to current channel configuration. For Multi-Channel use case, specify channel ID in ascending order

              starting from 0. E.g. 0,1,2... etc.

4. -a  : Algorithm name. Use the same name which was added in supported algorithm list. 5. -f  : Clip config file paths. File that contain I/o file path names. E.g. multiClip.cfg 6. -c  : Channel configuration file path. E.g. vaParams0.cfg 7. -t  : Core on which current channel is scheduled. Format used for this, applicable only if algorithm needs to be scheduled on

              multiple cores (Depends on algorithm,, not supported by framework)
    a.  3   : Total 1 core. Channel would be scheduled on core no. 3.
    b.  0:3   : Total 4 cores. StartChannelID:0, EndChannelID:3.
    c.  0 2 4 : Total 3 cores. Core no. used : 0, 2 and 4.

8. --width  : Width of the input YUV. 9. --height  : Height of the input YUV. 10 -F  : Enable frameOverlay. Applicable only for VA demos. Non zero value would enable overlay of DMVAL algorithm metedata on

              Input frame. ~host/src/dataSnk/FrameOverlay thread has functions to decode DMVAL result and
              then this thread modifies input frame. User can modify/add similar functionality for any new algorithm.

<-- Below options are applicable only for demoDisplay scripts --> 11 -D  : Input frame format rgb24/rgb32/yuv420p8. 12 -x  : Display window width. 13 -y  : Display window Height. 14 --display-rate : Frame rate at which frames are displayed.

VA Script Format for multi-channel use case.

     $MCSDK_VIDEO_HOSTBIN_DIR/demo_c667x --dsp-image=$MCSDK_VIDEO_DSPIMAGE_DIR/sv05.out -v
      --channel="VA DMVAL" --chanid=0 -a VADMVAL -f $CLIP_CONFIG0 -c $CODEC_PARAMS0 -t 0 --width=320 --height=240
      --channel="VA DMVAL" --chanid=1 -a VADMVAL -f $CLIP_CONFIG1 -c $CODEC_PARAMS1 -t 1 --width=320 --height=240

</syntaxhighlight>

User can refer Demo Reconfigurationfor more details.

DSP Application Changes

DSP build sv05 has been introduced for Video Analytics (VA):

DSP Binary Name	Video Use Case	Main Application Directory	Compilation Commands (need to run in ~/dsp/mkrel directory
SV05	Video Analytics	~/dsp/siu/va	make sv05

Important DSP Directories <syntaxhighlight lang='javascript'> ~/dsp/mkrel/  : Scripts and Makefiles to build DSP binary. Specify MCSDK Installation directory in setupenvMsys.sh file and

                       source to export all the environment variables.

~/dsp/ggcfg/build/hdg : Linker Command files. SV05 build is for VA application. Add algorithm library path in ggvf0.becmd file.

                       Refer changes w.r.t DMVAL.

~/dsp/siu/va  : MCSDK Video VA DSP application code. ~/dsp/siu/vct  : MCSDK Video Codec DSP application code. ~/dsp/c64x/make/  : Lower level make file to build Codec and VA application code.

                       User need to update makefile in case of new file addition.

~/dsp/siu/common  : Common files which are referred across various DSP applications. Codec, VA etc. </syntaxhighlight> How to integrate algorithm into the DSP Application
DSP application is single threaded application. Different instances of this thread runs on different core. Each core has its own mail box that works in busy waiting loop for any new messages from host application.

For VA use case, main application code on DSP side resides in ~dsp/siu/va/siuVaRun.c file. All the Algorithm integration functions are defined in this DSP wrapper code. Main task function (siuVctProcessClip()) of DSP VA application basically covers following functionality .

1. Mailbox message Receive : siuVctReceiveInput() function check mailbox of current core and returns with a message if there are any non-zero messages arrived from host application.
2. Message Process Call  : All the HOST2DSP messages are processed in this call and appropriate functions are called.
   • Mailbox Command: HOST2DSP_MEDIA_CONFIG_REQ -> Algorithm Create Call.
     1. Host application copies config parameters in a DSP buffer and the address is passed with this command. Configuration parameters are parsed on DSP and siuVctCodecParams_t (siuVctCodecParams) structure is populated. User can pass static and dynamic structure pointers to the algorithm create function.
     2. New algorithm create call can be added in switch statement(in siuVctRunMediaConfigReq() function )as it is done for DMVAL.
     3. Algorithm handle/Object is stored in mediaTaskContext_t->objVaPtr[] array. Each different channel/algorithm create call needs to return algorithm handle which can be used at run time. Algorithm handle and channel ID pairing is stored in mediaTaskContext_t struct.
   • Mailbox Command: HOST2DSP_MEDIA_PROCESS_REQ -> Algorithm Process Call.
     1. Host Application copies input frame into DSP DDR region via PCIe, allocates O/p buffers in DSP DDR and pass this mailBox message with I/O address to respective DSP cores.
     2. New algorithm process call can be added in switch statement (in siuVctRunMediaProcessReq() function ) as it is done for DMVAL.
     3. Refer code snippets given below for better understanding.
   • Mailbox Command: HOST2DSP_MEDIA_FLUSH_REQ -> Algorithm Flush Call
     1. New algorithm flush call can be added in switch statement as it done for DMVAL.
   • Mailbox Command: HOST2DSP_MEDIA_STATS_REQ -> DSP Statistics Call.
     1. Reserved. For future implementation
3. Mailbox message Transmit: Along with create and Process call, DSP2HOST message is prepared and once respective call is completed, mail box message is sent to Host. Host app keeps on checking non-zero message count from each core and whenever new message is available, Host application initiates result Read call.

Adding new algorithm type on DSP Side

Any new algorithm has to be added into SIU_VA_ALGType enum structure defined in ~dsp/siu/va/algs/siuVaAPI.h. All the control code execution happens based on this enumType. At init time, Host application sends algorithm type along with configuration parameter and on DSP side, after parsing same is stored in siuVctRunParams->codecName variable. For each channel, algorithm details are stored in taskCxt->vaAlgConfig struct. Based on siuVctRunParams->codecName, user can set alg Type in siuVctRunParams->codecName. Please refer DMAVl example code for more details.

<syntaxhighlight lang='javascript'>

Sample algorithm Process Call. Process call should return size of the result (e.g., algorithm metaData) generated by the algorithm. User is free to modify/optimize whenever possible.

  outputSize = YourAlgo_Process(taskCxt,                                      /* Task Context to Extract Algorthm handle.
                                                                                 Refer to DMVAL Process Call Details */
                              (XDAS_Int8 *)  rxMsgProcess->inBufPtr[0],       /* Input Frame Buffer pointer*/
                              (XDAS_Int32)   rxMsgProcess->inputId,           /* Input frame ID. Algorithm can use it for any book keeping*/
                              (XDAS_Int8 *)  rxMsgProcess->outBufPtr[0],      /* Output buffer pointer.
                                                                                 Algorithm needs to copy result in this buffer. */
                              taskCxt->vaAlgConfig[idx].chId);                /* ChannelID of the the frame to be processed */

Inside process call add below code. Required for task level bookkeeping and control code.

   /* Update OutArgs Status Params*/
   taskCxt->algOutArgs.frameBytesConsumed   = frameSize;                          /* Frame Data that is consumed.
                                                                                     By deafult set it to input frame size. */
   taskCxt->algOutArgs.buffEleFreeBufId[0]  = inputId;                            /* Input Frame ID, passed along with process call */
   taskCxt->algOutArgs.frameCount           = frameProcessCount;          /* Frame processed till now */
   taskCxt->algOutArgs.displayFrame         = frameProcessCount;          /* Frame processed till now */
   siu_osal_wbinv_cache_all();

Once frame is consumed, process call should return size of the O/p buffer. </syntaxhighlight>

DSP Linker Command file Changes
User needs to update Linker Command File. Link command file ~dsp/ggcfg/build/hdg/sv05/ggvf0.becmd must be updated to include the new Algorithm library. Please look for DMVAL related changes and modify the linker command file as needed.

Allocating Memory on DSP application
In the existing DSP application all the memory allocation for algorithm happens in the ~dsp/siu/va/siuVaMemMgr.c file. User can submit memory request in memtab structure. User can call _ALG_allocMemory and _ALG_freeMemory call for all the memory allocation/de-allocation. User can change the EXTERNAL_DATA_MEM_SIZE and EXTERNAL_DATA_MEM_SIZE_SLAVE macro to change memory size as per requirement.

<syntaxhighlight lang='javascript'> /* Submit memory request in below format. Instruction to use are given below. */ IALG_MemRec memTab; /* Include #include <ti/xdais/ialg.h> for its definition */

memTab.size = YOURMEMSIZE. memTab.alignment = ALIGNMENT memTab.attrs = IALG_PERSIST memTab.space = IALG_EXTERNAL

/*Call below function with the above details. Second parameter is no. of Memtab */

_ALG_allocMemory(memTab, 1);

/*After allocation user would get address in below field. */

memTab.base = AllocatedMemoryAddress </syntaxhighlight>

Useful Resources and Links

Product Download and Updates

For product download and updates, please visit the links listed in the table below.

	Product Download Link
MCSDK Video (2.1 GA) Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/mcsdk_video/latest/index_FDS.html
MCSDK Video (2.2 Alpha) Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/mcsdk_video/02_02_00_39/index_FDS.html
BIOS MCSDK Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/bios_mcsdk/02_01_02_05/index_FDS.html
Desktop Linux SDK Download	http://software-dl.ti.com/sdoemb/sdoemb_public_sw/desktop_linux_sdk/01_00_02_00/index_FDS.html
C6678 Codec Download	http://software-dl.ti.com/dsps/dsps_public_sw/codecs/C6678/index.html

MCSDK Video Instructions

Please visit the links below to install MCSDK Video, run the video demos, and get the details on how the MCSDK Video demos are developed.

	Wiki Links
Getting Started Guide for PCIe demos	MCSDK Video Getting Started for PCIe based Demos Desktop Linux SDK Getting Started
Getting Started Guide for TFTP demos	MCSDK Video Getting Started for TFTP based Demos
Development Guide	MCSDK Video Development Guide

Technical Support

For technical discussions and issues, please visit the links listed in the table below.

	Forum/Wiki Link
C66x Multicore forum	http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639.aspx
Multimedia Software Codecs forum	http://e2e.ti.com/support/embedded/multimedia_software_codecs/default.aspx
TI-RTOS forum	http://e2e.ti.com/support/embedded/f/355.aspx
Code Composer Studio forum	http://e2e.ti.com/support/development_tools/code_composer_studio/f/81/t/3131.aspx
TI C/C++ Compiler forum	http://e2e.ti.com/support/development_tools/compiler/f/343/t/34317.aspx
Embedded Processors wiki	http://processors.wiki.ti.com

Note: When asking for help in the forum you should tag your posts in the Subject with “MCSDK VIDEO”, the part number (e.g. “C6678”) and additionally the component (e.g. “NWAL”).