H.264 DM36x Ver 2.0 Codec

---

H.264 DM36x Platinum Codec (ver 2.x) - Overview and features

Version 2.0 of DM36x H.264 encoder has been released which has more feature set and better performance compared to the older ver 1.1 H.264 codec. This new codec can be downloaded from Click here to download

---

Platinum codec - Introduction

• What is H.264 Platinum codec?
  • H.264 encoder
    • New enhanced codec version with 1.5x more performance than earlier version without impacting quality.
    • Achieves 1080P@30fps on DM368
    • More feature rich codecs which includes
      • Smart codec technology whcih includes Region of interest coding and pseudo two pass
      • Low latency API support
      • Enhanced interlaced encoding with ARF support
      • Temporal scalable video coding with H.264 or SVC-T compatible syntax
      • Multiple slice with fixed packet size
      • CVBR RC which allows bitrate to vary depending on scene complexity
      • MegaPixel support – 5MP, 10MP
  • H.264 decoder
    • Achieves 1080P@30fps on DM368 for streams encoded by DM36x encoder
    • More feature rich codec which includes
      • Low latency API support with fixed packet size mode
      • Can decode SVC-T stream produced by DM368 encoder

• Feature list of ver DM36x ver 2.x H.264 encoder at a glance

---

H.264 Codec version history

• Note: Ver 2.x codec for DM36x is part of DVSDK 4.x

---

Performance

• Platinum codec provides 1.5x performance improvement compared to previous version
• High performance comes without compromising encoder quality compared to previous version

	DM365	DM368	Ratio
ARM (MHz)	297	432	1.45
DDR	270	340	1.26
HDVICP	270	340	1.26
Encoding/Decoding* performance	Ver 1.1 - 720p@30fps	Ver 2.0 (Platinum codecs) 1080P@30fps 1080i@60fields 720P@60fps

* Decoder performance on DM365 is for universal mode and for DM368 is for closed loop decoding mode. In universal mode on DM368, the perfromance will linearly scale w.r.t.DM365 performance

---

Smart codec technology

• Region of interest encoding
  • Give more bits to important region(e.g face) compared to other regions of the frame
  • Typical use case - Improving quality of important region during low bitrate scenario
  • Video conferencing – Give more importance to face region compared to background. Feedback can be taken from Face detect h/w
  • Surveillance – Give more importance to door compared to adjoining region

• Pseudo multi pass encoding
  • Pass encoding utilizes encoding metadata information from lower resolution to higher resolution
  • This helps improve quality of higher resolution stream.

• Details - Smart codec technology in DM36x

---

Scalable video coding – Temporal

• Scalable video coding – Temporal
• Ability to drop higher temporal levels without affecting the decodability of lower levels.
• Supports both SVC and H.264 syntax
• SVC-T stream is decodable by any existing standards complaint H.264 decoder.
• Support for 4 temporal layers.
• Option for having buffer management using MMCO and sliding window.
• Support both interlaced and progressive content

• Example use case scenario of SVC-T
  • IPNC: Temporal scalability allows one stream to be sent to multiple end devices/channels having different processing and transmission capability.
  • DVR: Temporal scalability allows easy archiving
  • VC: Effective error resilience tool

• Details - Temporal SVC in DM36x

---

Low latency API and fixed packet size support

• Low latency API support
  • New set of API to provide sub-frame level data exchange between codec and application
  • Data exchange done at slice level instead of frame level
  • Reduces latency of codec from 33ms to <2ms
  • Example use case
    • Low latency API will be helpful in all application which needs very low end to end latency viz - Video conferencing, Gaming consoles/servers etc
• Fixed packet size support
  • Allows to specify slice size in units of bytes in case of multiple slice operation.
  • Slice produced by encoder is guaranteed not to exceed the specified slice size

---

MegaPixel encoder

• Support for higher megapixel upto 4k x 4k resolution
• Any arbitrary resolutions up to 4Kx4K can be supported, frame rate would reduce in proportion as the image resolution increases
• Example use case
  • Video surveillance and conferencing: Higher image resolution allows more details to be captured. Real-life experience. Allows better digital zoom.

H.264 HP Encoder			Estimated Performance
	Frame Width	Frame Height	DM368@432 MHz	DM365@300 MHz
Resolution
5MP	2592	1944	10 fps	8 fps
10MP	3840	2748	5 fps	3 fps

---

CVBR Rate control

• DM36x originally supported constant quality(Fixed QP), constant bitrate(CBR) and variable bitrate (VBR) RC method
  • Both CBR and VBR tries to reach bitrate over period of time with CBR giving more importance to instanteneuos bitrate/buffer control and VBR giving more importance to overall quality.

CVBR is a new rate control which allows bitrate to change in a given time interval based on the complexity of the scene

• CVBR Allows to specify average, min and max bitrate over which the bitrate modulation can happen
• Allows to specify the time duration for which the CVBR stays at high complexity mode over a running sequence
• Reaches the average bitrate over larger period of time
• Example use case
  • This rate control is specially suited for video surveillance where one would intend to encode with better quality when there is increase in scene complexity

---

Gradual Decoder Refresh (GDR)

In H264 encoder periodic Intra frames are inserted in sequence for Error Resiliency and to allow Random access. In this mode all MBs of a frame are intra coded. It is done to terminate error propagation also. The disadvantage of this is higher bit consumption in I-frame compare to P-frame that might lead to higher decoding delay and frame skip. It might also increase breathing artifacts at low bitrates. To address these issues DM36x based encoder supports a feature called Gradual Decoder Refresh (GDR). The details of it are given below:

• The process of intra refreshing is extended over N frames
• Only part of frame is coded with intra MBs.
• Next non-overlapping part (with previous intra coded parts) is intra coded
• This is repeated until entire frame is refreshed

GDR advantages

• Bits consumption is now distributed over a group of N frames
  • Delay at decoding end reduces
  • Frame skips are reduced
• At the end of N frames all MBs are intra coded - Error propagation is terminated
• Breathing artifacts at low bitrates can be reduced

How to use of GDR:

• set enableGDR flag to 1 or 2: It enables GDR feature in encoder. When set to 1 normal quality GDR is enabled and when set 2 high quality GDR with small performance degradation is enabled.
• Set value of GDRduration: It tells the duration in which GDR refresh will happen.
• Set value of GDRinterval: It tells interval between successive GDR refresh.

Below figure illustrates GDRduration and GDRinterval -

---

Adaptive Longterm Frame Insertion

In a typical video conferencing scenarios, whenever decoder reports any error or frame loss to the encoder, encoder generates an intra frame. The drawback of this approach is that, I frames consume a lot of bits and is not desirable in scenarios where network bandwidth is a constraint. To address this issue, DM36x encoder supports feature called adaptive longterm frame insertion. The details are as below

• Ability to mark a frame as longterm frame
• Support to use longterm frame for reference
• Allows to specify the interval for refreshing the longterm frame used for reference.

• Typical use case of the above feature is shown below

How to use AdaptiveLTI:

• set EnableLongTermFrame to 1. This enable AdaptiveLTI feature
• set LongTermRefreshInterval. This sets the interval to refresh longterm frames. It can take any non zero positive value.
• set UseLongTermFrame. This variable is set when longterm frame has to be used for reference
• set SetLongTermFrame. This variable is used to mark current frame as longterm frame.

---

Chain Free P frame encoding

• All P frames refer to previous Intra/IDR frame.

• Useful in DVR applications for data archiving and storage
• Helps in 'storage thinning'. Over a period of time surveillance videos can be thinned to accommodate space for new videos
• Robust to errors. Loss of any P frame does not affect the video quality.
• Enables trick play capability. Fast forwarding to any frame without quality degradation.

How to use Chain Free Encoding:

• Set numTemporalLayers to 255
• Set IntraPeriod or IDRFramePeriod for the intra/IDR frame insertion. If IntraPeriod and IDRFramePeriod is set to 0 then an IDR will be inserted after 128 frames.

---

Comparison between ver 1.1 and 2.0(Platinum) encoder

Feature	Ver 1.1 - Gen 1	Ver 2.0 - Platinum
Resolution	Min – 128 x 96 Max – 2k x 2k	Min – 320 x 128 Max – Current (2k x 2k), Planned (4k x 4k)
Performance	720P@30fps on DM365	1080P@30fps on DM368
New features	Selectively present	Present
EDMA channels	37	46
Support for Ver 1.1 – Gen1	NA	YES

For details, please see the user guide of ver 2.0 encoder

---

Usage: integrate ver 2.0 codec in application/DVSDK

Refer to H.264 ver 2.x migration/integration guide

---

Frequently Asked Questions

Does new Platinum codec support older ver 1.1 encoding modes?

• Yes, the older STD mode of version 1.1 is supported for backward compatibility.

What is the need of supporting older ver 1.1 mode in Platinum 2.0?

• This is supported for backward compatibility. It also uses lesser number of EDMA channels. Some smaller resolution are only supported in ver 1.1 mode

My application needs more channels, which codec mode should I do?

• You should use ver 1.1 mode which uses 37 channels.

Is there something like ver 1.2 codecs?

• This version was 1.1 + smart features. This version is discontinued and replaced with version 2.0. If you are using ver 1.2, please migrate to 2.0 version.

I am using ver 1.1 codecs, should I migrate to 2.0?

• Final decision is left to user. We recommend to migrate to version 2.0 as it has better quality tuning, higher performance and bug fixes over previous version. Also, further support will be on ver 2.0 only.

Will the new features work if I use version 1.1 mode in the new Platinum ver 2.0 codec?

• Few features will work like LowLatency, smart codecs etc. But if your application is not limited by EDMA channel, it is strongly recommended to move to the main ver 2.0 mode.

Will the ver 2.0 codecs run on DM365 or it can run only on DM368?

• All s/w component of DM365 are 100% compatible with DM368 and vice versa. This includes all DVSDK components viz codecs, Framewortk Component, PSP, demos et.al. Hence ver H.264 2.0 codec will run on both Dm365 and DM368.