AM35x-VPFE-Software-Application Notes

Content is no longer maintained and is being kept for reference only!

AM35x VPFE Software Application Note

Read This First

Content is no longer maintained and is being kept for reference only!

About this Manual

This document describes all the internal details which driver developer should refer to for writing any sensor/camera/decoder slave driver for Texas Instruments' AM3517 platform running Linux.

The PSP (Platform Support Package) provides a fundamental software platform for development, which includes support for on-board Video Decoder (TVP5146) and known couple of sensors. Most of the time it always happen that customer would like to use various sensors based on their use-case, so this documents serves the starting point for all such customers/developers who wanted to add support for various sensor available.

The document also provides detailed description of Video Decoder driver (based on TVP5146) as implemented in the PSP.

References

http://focus.ti.com/docs/prod/folders/print/am3517.html

VPFE Hardware Block Diagram

The video processing subsystem (VPSS) includes a video processing front-end (VPFE) controller, which is the video input portion of the processor. The VPFE controller receives input video/image data from external capture devices and stores it to external memory. A built-in DMA engine transfers the capture data into the external memory. An internal buffer block provides a high bandwidth path between the VPSS module and the external memory.

AM3517 VPFE/CCDC block diagram

Features Supported

• Supports conventional Bayer pattern and Foveon sensor formats.
• Can be programmed to synchronize to the external horizontal/vertical sync and field ID signals with various timings.
• Supports progressive and interlaced sensors.
• Supports the REC656/CCIR-656 standard.
• Supports YCbCr 422 format, either 8-bit or 16-bit with discrete horizontal and vertical sync signals.
• Has built-in digital clamping and black level compensation.
• 10-bit to 8-bit A-law compression hardware is provided.
• Max pixel clock is 75 MHz.
• Supports up to 16K pixels (image size) in both the horizontal and vertical directions.

CCDC

CCDC is front end to the all external device interface like Video Decoder and sensors. The shared buffer logic is unique block, which acts as the primary source to the CCDC module that is sending data to the SDRAM/DDRAM. In order to efficiently utilize the external SDRAM/DDRAM bandwidth and to absorb the large latency that can be seen in a DDR memory system. The shared buffer logic/memory interfaces with the DMA system via a high bandwidth bus (32-bit wide). The shared buffer logic/memory also interfaces with the CCDC module via a 32-bit wide bus. The shared buffer logic/memory divided into the write buffer and arbitration logic.

Each request going out to the SDR/DDR EMIF is up to a transfer of 256 bytes. CCDC has its own buffer of 1024 bytes to absorb latency.

Interrupt Capabilities

AM3517 CCDC is capable of generating interrupts to main CPU (ARM) for the following events,

• After receiving the number of horizontal lines(VDIN_HD pulse signals) set in the VDINT0 bit in the VDINT register. (CCDC_VD0_INT)
• After receiving the number of horizontal lines(VDIN_HD pulse signals) set in the VDINT1 bit in the VDINT register. (CCDC_VD1_INT)
• On External write-enable signal (VDIN_WEN) via external pin (CCDC_VD2_INT)

CCDC VD0 & VD1 interrupt generation with respect VD polarity

CCDC VD2 interrupt generation with respect to external write enable event

Software Architecture

Linux driver for AM3517 VPFE is based on standard Video4Linux sub-device framework, which provides Master-Slave architecture to seamless multiple sensor/decoder interface to single master driver.

V4L2 Sub-Device Architecture

V4L2 Sub Device Init Sequence

So each Sensor AND/OR Decoder will have slave driver registering themselves to VPFE master driver. In case of AM3517 you can locate the source code under

1 Master Driver - <Linux Home Directory>/drivers/media/video/davinci/vpfe.c 1 TVP5146 Slave Driver - <Linux Home Directory>/drivers/media/video/tvp514x.c

Like wise, when we support any raw sensor (e.g. MT9V113) then the corresponding driver will be available under <Linux Home Directory>/drivers/media/video/mt9v113.c

Interfacing BT656 compatible device to AM3517 CCDC

This section describes the image/data processing in the YUV or BT656 data mode.

CCDC Image Processing - YUV/BT656 Mode

AM3517 supports interfacing to the device (slave-device) which is capable of giving output in BT656 format either 10-bit or 8-bit. If the output is 10-bit BT656 then it is important to interface upper 8-bit output of slave-device to lower 8-bit of AM3517 CCDC input. The CCDC does support both 8 & 10bit format but it becomes important pack it to 8-bit to get YUV422 data for further processing in software.

NOTE (AM3517EVM Errata): On AM3517EVM the pins are connected one-to-one (TVP5146[0-9] => CCDC [0-9]), which leads to loss of color data, since CCDC reads lower 8-bit information and saves it to memory. The workaround implemented for this is, configure CCDC in raw mode, apply A-Law table to convert 10-bit data to 8-bit data and again apply reverse A-Law in application to get original colors. Please note that this workaround does consume significant amount of CPU cycles.

AM3517 PSP SDK supports BT656 interface with on-board TVP5146 Video Decoder, below are the details,

• Data pin interfacing - TVP5146 [0-9] => CCDC[0-9]
• Pixel Clock - 27MHz adhering BT656 standard
• Input Frame type - Interlacing fields adhering BT656 standard
• Support for Control configuration - Brightness, Contrast, Hue and Gain control
• Input interface support - Composite and S-Video Input
• Input Standard - NTSC and PAL standard
• External SYNC signals - not Applicable, since it CCDC works with Embedded sync signals
• Interrupts being used - Only VD1 interrupt
• Number of interrupts per Second -
  • NTSC Standard - @30FPS with 2 fields => 30 * 2 = 60 interrupts/sec
  • PAL Standard - @25FPS with 2 fields => 25 * 2 = 50 interrupts/sec

Since the input stream is interlacing fields we have to use single buffer for both odd and even field. CCDC support seamless dumping of both the fields to the memory by configuring CCDC.SDOFST register.

CCDC De-Interlacing using SDOFST register configuration

Interfacing RAW/RGB Sensor device to AM3517 CCDC

This section describes the image/data processing in the raw/RGB data mode.

Sensor interfacing with AM3517

CCDC interface to sensor in Raw/RGB mode

Image Processing inside AM3517 VPFE

CCDC Image Processing - RAW/RGB Mode

AM3517 supports interfacing to the device (slave-device) in raw mode configurable between 8-bit and 16-bit. In raw mode user can use image processing features like, black clamping, A-Law compression, etc... Please refer to the above block diagram.

• Sensor - MT9T111 (3M pixel Sensor)
• Data pin interfacing - Sensor [0-15] => CCDC[0-15]
• Pixel Clock - TBD
• Input Frame type - Progressive
• Support for Control configuration - TBD
• Input interface support - RAW/RGB mode
• Input Resolution - TBD
• Interrupts being used - Only VD1 interrupt
• External SYNC signals - Both HS and VS is being used.

Software Change

Driver Files -

Bridge/Master Driver -
<kernel home directory>/drivers/media/video/ti-media/vpfe_capture.c

CCDC Driver -
<kernel home directory>/drivers/media/video/ti-media/dm644x_ccdc.c

Sensor Driver -
<kernel home directory>/drivers/media/video/<sensor>.c

The AM3517 V4L2 driver follows Sub-Device Framework, so the sensor driver under discussion here supposed to follow sub-device framework. Please refer to the tvp514x.c file for more details on sub-device driver operations.

The VPFE CCDC driver code does support both raw (bayer) and YUV format, but has not been validated against any sensor device. In raw (bayer) mode is supports following pixel format -

• SBGGR8
• SBGGR16

In YUV mode it does support (8-bit) -

• UYVY
• YUYV

In order to add support for Raw/RGB sensor, please follow below steps -

NOTE: Please note that the below description assumes that you have sub-device (slave) driver ready for your respective sensor configured in Raw/RGB mode.

- Board Hookup configuration -

• Include following header file in your board-xxx.c file

1. include <media/ti-media/vpfe_capture.h>

• Define struct vpfe_subdev_info to register your sub-device to VPFE driver -

E.g -

static struct vpfe_subdev_info vpfe_sub_devs[] = {
    {
        .name           = "mt9t111",
        .grp_id         = 0,
        .num_inputs     = 0,
        .can_route      = 0,
        .ccdc_if_params = {
            .if_type = VPFE_RAW_BAYER,
            .hdpol   = VPFE_PINPOL_POSITIVE,
            .vdpol   = VPFE_PINPOL_POSITIVE,
         },
         .board_info     = {
            I2C_BOARD_INFO("mt9t111", <i2c addr>),
            .platform_data = &mt9t111_pdata,
         },
    },
};

Please note that above parameters are board specific and user should change based on their board interface schema.

There are some gaps in CCDC driver which have to be fixed for raw/RGB mode,

The function "ccdc_set_hw_if_params" doesn't support raw mode configuration, you have to change this function to support it. Rest of the configuration should happen automatically.

NOTE: The above information is created by code walk-in, so might require some more modifications ontop of above changes.

Important
Please note that We are in the process of adding couple of sensors (in RAW/RGB mode) to the AM3517, so I will update this page for further details once we are done with this.