PDK/PDK VPS VPE Driver User Guide

Memory to Memory Drivers

Introduction

Memory to Memory drivers takes the video buffer from the memory, optionally process the buffer, processing done on the buffer depends on the specific memory to memory driver and puts it back to memory. Memory to memory driver follows the FVID2 interface for the applications.

Following are the general feature set for the memory to memory drivers:

• All the memory to memory driver supports multiple handle. This means the driver can be opened multiple times.
• All the memory drivers supports multiple channels request submission per handle. Multiple channels means the video stream coming from multiple streams like frames coming from decoder over network, multiple capture streams each having same/different frame parameters like height, width etc.
• Memory driver supports parameter configuration for the buffer processing per channel of the handle. There can be individual set of parameters for each channel of the handle like height, width, data format etc or else application can have the same parameters for all the channels of the handle.
• Application can submit multiple channels for processing in a single request call.
• Fvid2_processFrames (queue) and Fvid2_getProcessFrames (de-queue) FVID2 calls for all the memory to memory drivers are non blocking. While the control commands like programming of the scalar coefficients are blocking.
• All memory to memory driver calls the application call back function on completion of the request. Application should de-queue the request after the callback.

VPE Memory to Memory Driver

Introduction

This chapter describes the hardware overview, application software interfaces, typical application flow and sample application usage for VPE memory to memory driver.

The features and limitations of current driver implementation are listed in subsequent sections.

Important

The features supported or NOT supported in any release of the driver may vary from one PDK driver release to another. See respective release notes for exact release specific details.

Features Supported

Features	Supported in TDA2SEDx	Supported in TI814x
Instances
VPS_M2M_INST_VPE1 single scale path	YES	YES
Input Formats
YUV422 Interleaved	YES	YES
YUV420 Semi-Planar	YES	YES
YUV422 Semi-Planar	YES	YES
YUV422 Semi-Planar Tiled	NOT TESTED	NOT TESTED
YUV420 Semi-Planar Tiled	NOT TESTED	NOT TESTED
Output Formats
YUV422 Interleaved	YES	YES
YUV420 Semi-Planar	YES	NO/NA
YUV422 Semi-Planar	YES	NO/NA
YUV420 Semi-Planar Tiled	NOT TESTED	NOT TESTED
YUV422 Semi-Planar Tiled	NOT TESTED	NOT TESTED
RGB888	YES	NO/NA
YUV444	YES	NO/NA
DEI Features
DEI in deinterlacing mode	YES	YES
DEI in progressive bypass mode	YES	YES
Line averaging and field averaging mode of DEI operation	YES	YES
SC Features
Optional scaling using SC1	YES	YES
Scaling from 1/8x to 2048 maximum pixels in horizontal direction	YES	YES
Different types of scalar like poly phase and running average	YES	YES
Horizontal and vertical cropping of the image before scaling	YES	YES
Lazy loading of coefficient	YES	YES
User programmable scalar coefficients	NO	NO
Other Features
Frame drop feature to enable load balancing	NOT TESTED	NOT TESTED
Multi-channel (up to VPS_M2M_MAX_CH_PER_INST channels per instance)	YES	YES
Multi-handle (up to VPS_M2M_MAX_CH_PER_HANDLE channels per instance)	YES	YES
Error callbacks	YES	YES
Slice based scaling when DEI is in progressive bypass mode	NOT TESTED	NOT TESTED
Slice based scaling when DEI is in deinterlacing mode	NO	NO
Interlaced bypass mode	NO	NO
Runtime Configurations
Input resolution change when DEI is in progressive bypass mode	YES	YES
Input resolution change when DEI is in deinterlacing mode	YES	YES
Output resolution change	YES	YES
SC crop and config change on SC1	YES	YES
DEI reset	NOT TESTED	NOT TESTED

Hardware Overview

Below figures show the complete VPE Hardware.

VPE Driver Path As shown in below figures, the VPE memory to memory driver takes in YUYV422/YUV420 interlaced/progressive input via the DEI path and provide single scaled output of the deinterlaced/bypassed output

Overview

De-interlacing operation of a field requires two previous fields. This requirement is abstracted by the driver, the drivers holds back required input fields as context fields. The following expand on the behavior of the driver.

• Considering a single channel operation for VPS_M2M_INST_VPE1,

following steps describe the operations performed on Fvid2_ProcessList.inFrameList referred as inFrameList

• First Fvid2_processFrames() assuming F1 was submitted to be de-interlaced, on completion of this API,

Fvid2_getProcessedFrames() could be called to retrieve the output frame and input field. However the input field would be held back by the driver (F1 in this case). i.e. the numFrames of inFrameList is set 0x1 and inFrameList->frames[0x0] = NULL. The output frame would be available, i.e. the numFrames of outFrameList is set to 0x01 and outFrameList->frames[0x0] will point to a valid frame. The applications are expected to check for valid frames, before performing further operations on the frame.

inFrameList.numFrame = 0x01

inFrameList.frames[0x0] = NULL

outFrameList.numFrames = 0x01

outFrameList.frames[0x0] = valid frame

• On second Fvid2_processFrames() assuming F2 was submitted to be de-interlaced, the above step is repeated for field F2.

i.e. F2 is also held back the driver

• On third Fvid2_processFrames() assuming F3 was submitted to be de-interlaced, the above step is repeated for field F3.

i.e. F3 is also held back the driver

• On fourth Fvid2_processFrames() assuming F4 was submitted to be de-interlaced, on completion of this API,

Fvid2_getProcessedFrames() could be called to retrieve the output frame and input field. The first input field and fourth output frame is given back. i.e.

inFrameList.numFrame = 0x01

inFrameList.frames[0x0] = F1

outFrameList.numFrames = 0x01

outFrameList.frames[0x0] = valid frame

• On fifth Fvid2_processFrames() assuming F5 was submitted to be de-interlaced,

the above step is performed with following output

inFrameList.numFrame = 0x01

inFrameList.frames[0x0] = F2

outFrameList.numFrames = 0x01

outFrameList.frames[0x0] = valid frame

• The driver would hold back previous N-1 and N-2 input fields as context buffers. This buffer could be retrieved using Fivd2_stop API.

• Multiple channel - The same procedures described for single channel applies.

The numFrames of inFrameList and outFramesList defines the number of frames/fields that application should look for in the frames array.

inFrameList.numFrame = 0x04

inFrameList.frames[0x0] = CH1F1

inFrameList.frames[0x1] = CH2F1

inFrameList.frames[0x2] = NULL

inFrameList.frames[0x3] = CH3F1

outFrameList.numFrames = 0x04

outFrameList.frames[0x0] = valid frame

outFrameList.frames[0x1] = valid frame

outFrameList.frames[0x2] = valid frame

outFrameList.frames[0x3] = valid frame

Note that, in above example the driver has held back input field of channel 3, while releasing input fields of other channels. This would mean that channel 3 did not have enough context buffers.

Important

Applications should take into account that any input field could be held back by the driver, an NULL check on field should be performed before using it.

Software Application Interfaces

The driver operation can be partitioned into the below phases:

• System Init Phase: Here the driver sub-system is initialized
• Create Phase: Here the driver handle is created or instantiated
• Run Phase: Here driver is used to submit the frames for processing and getting the processed frames from the driver.
• Delete Phase: Here the driver handle or instance is deallocated
• System De-init Phase: Here the driver sub-system is de-initialized

The subsequent sections describe each phase in detail.

Note

Details of the structure, enumerations and #defines mentioned in the section can be found in PDK API Guide

System Init Phase

VPE M2M driver initialization happens as part of overall VPS system init. This API must be the the first API call before making any other FVID2 calls. Below section lists the APIs which are part of the System Init phase.

VPS Init

Int32 Fvid2_init(Ptr args);

args - NULL currently not used.

Int32 Vps_init(const Vps_InitParams *initPrms);

initPrms - Vps_InitParams * VPS Initialization parameters.

    /* Init FVID2 */
    retVal = Fvid2_init(NULL);
    if (FVID2_SOK != retVal)
    {
        System_printf("FVID2 Init failed\n");
    }

    /* Init VPS */
    VpsInitParams_init(&vpsInitPrms);
    retVal = Vps_init(&vpsInitPrms);
    if (FVID2_SOK != retVal)
    {
        System_printf("VPS Init failed\n");
    }

Create Phase

In this phase user application opens or creates a driver instance. Each instance of the driver supports VPS_M2M_MAX_CH_PER_INST (defined in vps_m2m.h) handles creation. Operation commands from the different handles of the same instance will be serialized by the driver and will be served by the singe instance of the hardware. Below sections lists the API interfaces to be used in the create phase. Create phase allows the application to do the configuration either through control commands exposed by driver or through the parameters passed with the driver create API.

FVID2 Create

This API is used to open the driver. This is a blocking call and it returns the handle which is to be used in subsequent call to this driver.

Fvid2_Handle Fvid2_create(UInt32 drvId,
                          UInt32 instanceId,
                          Ptr createArgs,
                          Ptr createStatusArgs,
                          const Fvid2_CbParams *cbParams);

drvId - FVID2_VPS_M2M_DRV to open the driver.

instanceId - VPS_M2M_INST_VPE1 macro to open VPE1 memory driver.

createArgs - Pointer to Vps_M2mCreateParams structure containing valid create params. This parameter should not be NULL.

createStatusArgs - Pointer to Vps_M2mCreateStatus structure containing the return value of create function and other driver information. This parameter should not be NULL.

cbParams - Pointer to Fvid2_CbParams structure containing FVID2 callback parameters. This parameter should not be NULL.

    Fvid2_Handle            fvidHandle;
    Fvid2_CbParams          cbParams;
    Vps_M2mVpeChParams      chPrms;
    Vps_M2mCreateParams     createPrms;
    Vps_M2mCreateStatus     createStatus;

    /* Init create params */
    VpsM2mCreateParams_init(&createPrms);
    createPrms.numCh           = 1;
    createPrms.chInQueueLength = VPS_M2M_DEF_QUEUE_LEN_PER_CH;
    createPrms.isDeiFmdEnable  = FALSE;

    /* Init callback parameters */
    Fvid2CbParams_init(&cbPrms);
    cbPrms.cbFxn     = &App_m2mVpeAppCbFxn;
    cbPrms.errCbFxn  = &App_m2mVpeAppErrCbFxn;
    cbPrms.errList   = &errProcessList;
    cbPrms.appData   = appObj;

    /* Open the driver */
    fvidHandle = Fvid2_create(
                     FVID2_VPS_M2M_DRV,
                     VPS_M2M_INST_VPE1,
                     &createPrms,
                     &createStatus,
                     &cbPrms);
    if (NULL == fvidHandle)
    {
        System_printf("Create failed!!\n");
    }

FVID2 Control - Set VPE Parameters

IOCTL_VPS_M2M_SET_VPE_PARAMS IOCTL can be used to set the VPE hardware specific parameters. This IOCTL should be called after creating VPE M2M driver instance and before queueing or starting the M2M driver. Starting the M2M driver without calling this IOCTL will result in error. This is a blocking call.

Important

This API should not be called when there are any pending request with the driver.

Int32 Fvid2_control(Fvid2_Handle handle,
                    UInt32 cmd,
                    Ptr cmdArgs,
                    Ptr cmdStatusArgs);

handle - Driver handle returned by create function call. This parameter should not be NULL.

cmd - IOCTL_VPS_M2M_SET_VPE_PARAMS ioctl.

cmdArgs - Pointer to Vps_M2mVpeParams structure. This parameter should not be NULL.

cmdStatusArgs - Not used currently. This parameter should be set to NULL.

FVID2 Control - Set Scalar Coefficient

This is used to issue a control command to the driver. IOCTL_VPS_SET_COEFFS ioctl is used to set the scalar coefficients. This is a blocking call.

Important

This API should not be called when there are any pending request with the driver.

Int32 Fvid2_control(Fvid2_Handle handle,
                    UInt32 cmd,
                    Ptr cmdArgs,
                    Ptr cmdStatusArgs);

handle - Driver handle returned by create function call. This parameter should not be NULL.

cmd - IOCTL_VPS_SET_COEFFS ioctl.

cmdArgs - Pointer to Vps_ScCoeffParams structure containing valid scaling coefficient. This parameter should not be NULL. To set the scalar coefficient for VPE1 scalar, scalarId should be set to VPS_M2M_VPE_SCALER_ID_SC0.

cmdStatusArgs - Not used currently. This parameter should be set to NULL.

FVID2 Control - Get DEI Context Information

When DEI is in de-interlacing mode, DEI requires previous fields and motion vectors. Buffers for storing these context information must be allocated by the application and provided to the driver before starting M2M operation. IOCTL_VPS_GET_DEI_CTX_INFO ioctl is used to get the number of internal buffers to be allocated and their sizes. Application should get this information from the driver and allocate these buffers and provide the buffers to the driver before issuing any request. Once these buffers are given to the driver, application should not modify these buffers. This should be done for each and every channel. This is a blocking call.

Important

This API should not be called when there are any pending request with the driver.

Int32 Fvid2_control(Fvid2_Handle handle,
                    UInt32 cmd,
                    Ptr cmdArgs,
                    Ptr cmdStatusArgs);

handle - Driver handle returned by create function call. This parameter should not be NULL.

cmd - IOCTL_VPS_GET_DEI_CTX_INFO ioctl.

cmdArgs - Pointer to Vps_DeiCtxInfo structure where the DEI context information will be filled by driver. This parameter should not be NULL.

cmdStatusArgs - Not used currently. This parameter should be set to NULL.

FVID2 Control - Set DEI Context Buffers

IOCTL_VPS_SET_DEI_CTX_BUF ioctl is used to set the DEI context buffers for a channel before providing any request to the driver. This is a blocking call.

Important

This API should not be called when there are any pending request with the driver.

Int32 Fvid2_control(Fvid2_Handle handle,
                    UInt32 cmd,
                    Ptr cmdArgs,
                    Ptr cmdStatusArgs);

handle - Driver handle returned by create function call. This parameter should not be NULL.

cmd - IOCTL_VPS_SET_DEI_CTX_BUF ioctl.

cmdArgs - Pointer to Vps_DeiCtxBuf structure valid buffer pointers as requested by the driver for a particular DEI mode of operation. This parameter should not be NULL.

cmdStatusArgs - Not used currently. This parameter should be set to NULL.

FVID2 Control - Get DEI Context Buffers

IOCTL_VPS_GET_DEI_CTX_BUF ioctl is used to get the DEI context buffers for a channel from the driver. Once the DEI context buffer is returned to the application, no more request should be provided to the driver. This is a blocking call.

Important

This API should not be called when there are any pending request with the driver.

Int32 Fvid2_control(Fvid2_Handle handle,
                    UInt32 cmd,
                    Ptr cmdArgs,
                    Ptr cmdStatusArgs);

handle - Driver handle returned by create function call. This parameter should not be NULL.

cmd - IOCTL_VPS_GET_DEI_CTX_BUF ioctl.

cmdArgs - Pointer to Vps_DeiCtxBuf structure where the driver returns back the DEI context buffer to the application. This parameter should not be NULL.

cmdStatusArgs - Not used currently. This parameter should be set to NULL.

Run Phase

M2m drivers are non-streaming drivers. This phase is used to submit the requests for processing and getting the processes request back.

Start

NA

Stop

The driver would retained 3 fields, as context fields. Once application has completed all the de-interlacing operation, this command could be used to retrieve the context fields.

• Should only be used when de-interlacing, i.e. should not be used in bypass mode
• Normally when applications are ready to close, this control command is expected to be used.
• Is a blocking call

Int32 Fvid2_stop(Fvid2_Handle handle,
                 Ptr cmdArgs);

handle - Driver handle returned by create function call. This parameter should not be NULL.

cmdArgs - Not used currently. This parameter should be set to NULL.

FVID2 Process Frames

This API is used to submit video buffers to the driver for processing operation. This is a non-blocking call and should be called from task context. Once the buffer is queued the application loses ownership of the buffer and is not suppose to modify or use the buffer.

Int32 Fvid2_processFrames(Fvid2_Handle handle,
                          Fvid2_ProcessList *processList);

handle - Driver handle returned by create function call. This parameter should not be NULL.

processList - Pointer to Fvid2_ProcessList structure containing the pointer to the FVID2 frames/framelist. This parameter should not be NULL.

FVID2 Get Processed Frames

This API is used by the application to get ownership of the processed video buffer from the memory driver. This is a non-blocking call and could be called from task or ISR context.

Int32 Fvid2_getProcessedFrames(Fvid2_Handle handle,
                               Fvid2_ProcessList *processList,
                               UInt32 timeout);

handle - Driver handle returned by create function call. This parameter should not be NULL.

processList - Pointer to Fvid2_ProcessList structure where the driver will copy the processed FVID2 frames/framelist. This parameter should not be NULL.

timeout - Not used currently as only non-blocking queue/dequeue operation is supported. This parameter should be set to FVID2_TIMEOUT_NONE.

Delete Phase

In this phase FVID2 delete API is called to close the driver handle. Hardware resources are freed once all the handles of the particular instance are freed. Handle can be opened again with different configuration.

FVID2 Delete

This API is used to close the memory driver. This is a blocking call and returns after closing the handle.

Int32 Fvid2_delete(Fvid2_Handle handle, Ptr deleteArgs);

handle - Driver handle returned by create function call. This parameter should not be NULL.

deleteArgs - Not used currently. This parameter should be set to NULL.

System De-Init Phase

FVID2 de-Init

Drivers gets de-initializes as a part of BSP sub-system de-Initialization. Here all resources acquired during system initialization are free'ed. Make sure all driver instances and handles are deleted before calling this API. Typically this is done during system shutdown.

Int32 Fvid2_deInit(Ptr args);

args - Not used

Sample Application

VPE DEI/Scale

This example illustrates the VPE SC/DEI operation supportted by VPE memory to memory driver

• VPE Instance: VPS_M2M_INST_VPE1

Example features: 720x240 interlaced YUV420 data is fed into the DEI path. DEI is configured in deinterlacing mode. The deinterlaced input is scaled to 360x240 (YUV422) and output via the output path.

• Load vps_examples_m2mVpeScale_ipu1_0_release.xem4 executable file found at $(rel_folder)\packages\ti\binary\vps_examples_m2mVpeScale\bin\<platform>

to IPU1 Core 0

• Run the application
• The application will halt for the user to load the input frames and to select driver path. Using loadRaw command in script console of CCS, load 10 fields of

720 x 240 YUV420 semiplanar video to location mentioned in the console print. (Ignore "syntax error" if it appears during loading)

loadRaw(< Location >, 0, " < File Path > ", 32, false);

• User can save the outputs to a file using the saveraw command as printed from the console window.

saveRaw(0, < Location >, " < File Path > ", 432000, 32, true);
saveRaw(0, < Location >, " < File Path > ", 1296000, 32, true);

• Application will stop after processing 10 frames