From Texas Instruments Embedded Processors Wiki

Contents 1 DM365 codecs FAQ 1.1 What is DM365 SoC? 1.2 Which are the codecs supported by the device? 1.3 What are the codecs available? 1.4 Does H.264 How is the codec distribution on VICP and HDVICP? 1.5 Why the word "closed" for some of the decoders? 1.6 What is the performance of each codec? 1.7 Does the performance quoted in "DM365 Codec Availability Schedule" worst case performance? 1.8 Does the codec support multi channel operation? 1.9 Does the codec support multi stream/codec operation? 1.10 What are the codec combos supported? 1.11 Is Mpeg2, VC1, H.264 1080P and Mpeg4 universal codecs available in DVSDK release? 1.12 Can H.264 and Mpeg/Jpeg run concurrently as they use different accelerators? 1.13 What are the inputs and output formats for the codecs? 1.14 What is the interface for the codecs? 1.15 Is rotation and resize part of codec API? 1.16 What is the application scenario considered when specifying the performance? 1.17 What is the latency of the codec? 1.18 What is the codec ARM926 MHz load? 1.19 Are the codecs OS independent? 1.20 Are the codecs CCS compiled libraries available? 1.21 What are the main features of H.264 encoder? 1.22 Can H.264 encoder do all Intra frames encoding? 1.23 There is no H.264 BP mentioned in the codec data sheet or availability chart, is it a subset of HP? If yes, will they be part of one library? 1.24 How does the quality of H.264 encoder look like? 1.25 What are the main features of H.264 decoder? 1.26 Is the H.264 HP decoder universal? 1.27 Does H.264 support multiple sequences to be decoded without deleting and creating the instance again? 1.28 What are the modes in which H.264 Decoder can be invoked? 1.29 Will there be change in performance for H.264 BP and HP profile? 1.30 What is the roadmap of H.264 codec on DM365? 1.31 Can H.264 codec do 1080P@30fps? Is there some new codec to achieve this? 1.32 What type of error concealment is supported in H.264 decoder? 1.33 What is the DDR b/w of H.264 encoder and decoder?= 1.34 What are the main features of Mpeg4 encoder? 1.35 How does the quality of Mpeg4 encoder look like? 1.36 What is the difference between 1.00.00 and 1.10.00 version of Mpeg4 encoder? 1.37 What are the main features of Mpeg4 decoder? 1.38 What type of error concealment is supported in Mpeg4 ASP decoder? 1.39 What is the difference between 1.00.00 and 1.10.00 version of Mpeg4 decoder? 1.40 What are the main features of Jpeg encoder? 1.41 What are the main features of Jpeg decoder? 1.42 What are the main features of Mpeg2 encoder? 1.43 What are the main features of Mpeg2 decoder? 1.44 What are the main features of VC1 decoder? 1.45 Is VC1 encoder also available? 1.46 What are the main features of Mpeg4 ASP decoder? 1.47 Is the Mpeg4 decoder universal? 1.48 What is smart codec? 1.49 What is Region of Interest coding? 1.50 What is pseudo multi – pass encoding? 2 DM36x H.264 encoder FAQ 3 Will DM365 codecs run on DM368?

---

DM365 codecs FAQ

---

---

What is DM365 SoC?

DM365 is a low power imaging, video and audio SOC for applications like Digital camera, IP based security, Digital picture frame, Low end STB, Home and industrial automation. It supercedes DM355 SoC with improved video capabilities

• H.264 encode and decode
• VC1 encode and decode
• Universal Mpeg4 ASP encode and decode

For more details - www.ti.com/dm365prod

---

Which are the codecs supported by the device?

The codecs supported are –

• H.264 BP/MP/HP decoder
• H.264 BP/MP/HP encoder
• VC1 SP/MP/AP decoder
• VC1 SP/MP/AP encoder
• MPEG4 SP/ASP Decoder & Encoder
• MPEG2 MP @ HL Decoder & Encoder
• JPEG SEQ encoder and decoder

---

What are the codecs available?

The following codecs are developed.

• H.264 BP/MP/HP encoder and decoder
• MPEG4 SP/ASP encoder & decoder
• MPEG2 MP @ HL encoder & decoder
• VC1 SP/MP/AP decoder
• JPEG Sequential encoder and decoder

For more details on availability, see: DM365 Codec Availability Schedule

---

Does H.264 How is the codec distribution on VICP and HDVICP?

• VICP/MJCP
  • MPEG4 SP encoder support BP and MP also along with HP?
  • MPEG4 SP closed loop decoder
  • JPEG encoder and decoder
  • Noise filter
  • Other post/pre-processing
• HDVICP
  • H264 encoder and decoder
  • VC1 encoder and decoder
  • MPEG4 universal and ASP codec
  • MPEG2 encoder and decoder

---

Why the word "closed" for some of the decoders?

Closed loop decoders can only decode stream encoded by the encoder of that corresponding device. It cannot decode any stream available under the supported profile and level.

---

What is the performance of each codec?

The below link gives a tentative performance chart

DM365 Codec Availability Schedule

---

Does the performance quoted in "DM365 Codec Availability Schedule" worst case performance?

No, the performance quoted is typical case performance. Depending on the stream characteristics and encoder configuration, performance may vary.

---

Does the codec support multi channel operation?

Yes, all the DM365 codecs support multichannel operation. One can create codec instance for each channel and achieve multichannel operation.

---

Does the codec support multi stream/codec operation?

Yes, all the DM365 codecs can co-exist with each other. One can create codec instance for any codec combination and they will get scheduled in a frame interleaved manner to achieve real-time operation.

---

What are the codec combos supported?

Any codec combo can be achieved using frame level interleaving. The performance of the codec combo has to be scaled down linearly depending on the individual performance and the combo needed

---

Is Mpeg2, VC1, H.264 1080P and Mpeg4 universal codecs available in DVSDK release?

No, the below stage 2 and 3 codecs are not part of DVSDK 2.10.

• H.264 HP 1080P encoder and decoder
• Mpeg2 MP @ HL encoder and decoder
• Mpeg4 SP encode and Mpeg4 ASP decoder
• VC1 decoder

These codec will be formally integrated in DVSDK 3.1 release further down in timeline. As part of codec release, we will be providing a readme which will guide how to integrate a codec in DVSDK 2.10. Customer can use this readme to integrate the codec in DVSDK and get started.

---

Can H.264 and Mpeg/Jpeg run concurrently as they use different accelerators?

The version 1.00.00 of H.264 and Mpeg cannot run concurrently. H.264 encoder uses VICP buffer for intermediate storage of data, hence Mpeg4 cannot run concurrently with H.264. But this does not limit Mpeg4 and H.264 multi-channel use case. This can be indeed achieved by running them sequentially in a frame interleaved manner.

Version 2.00.00 H.264 encoder will have option to not to use the VICP/MJCP buffers. With those codecs, one can run VICP codecs and H.264 concurrently. But since both of them use EDMA channels, one should be sure that the sum of EDMA channels used by them should not increase the required limit.

---

What are the inputs and output formats for the codecs?

The input and the output format of the codecs are –

	H.264, Mpeg2, HDVICP Mpeg4 encode	H.264, Mpeg2, VC1 and Mpeg4 ASP decode	Mpeg4 encode	Mpeg4 decode	Jpeg encode	Jpeg decode
Input	YUV 420 SP	YUV420 bitstream	YUV422 ILE YUV 420 SP	YUV420 Bitstream	YUV422 ILE YUV422 P YUV420P YUV 420 SP	YUV422 bitstream YUV 420 bitstream YUV444 bitstream Grey level bitstream
Output	YUV420 Bitstream	YUV 420 SP	YUV420 Bitstream	YUV422 ILE YUV 420 SP	YUV422 bitstream YUV 420 bitstream	YUV422 ILE YUV 420 SP

Note: 420 SP: 420 Semi Planer, this format consists of Y as one plane and CbCr interleaved as other.

---

What is the interface for the codecs?

The codecs implement the following XDM-defined interfaces:

• IVIDENC1
• IVIDDEC2
• IIMGENC1
• IIMGDEC1

---

Is rotation and resize part of codec API?

It is there for few codecs and for limited resolution.

	Rotation	Resize	comments
Jpeg Decode	Yes	Yes
Jpeg Encode	Yes	No
Mpeg4 Decode	Yes	No	Rotation support upto D1 only
Mpeg4 Encode	Yes	No	Rotation support upto D1 only
H.264, VC1, Mpeg2, Mpeg4 ASP universal Decode	No	No
H.264, Mpeg2, Mpeg4 HDVICP Encode	No	No

---

What is the application scenario considered when specifying the performance?

While specifying the performance, basic application scenario is considered.

For decoder, it consists of stream i/p, decode and display. For encoder, it consists of capture, encode and stream o/p.

---

What is the latency of the codec?

All the codecs operate at frame level and hence the latency is one frame encode or decoder time. For a 720P@30fps on 300MHz device, the latency will be approx 33 ms.

In later version of H.264 codec (ver 2.00.00 and beyond), there is a provision of slice level call back API which enables application to give/take data from the codec at sub frame level. For the same 720P@30fps on 300 MHz device, the latency will decrease to 5ms compared to 33 ms.

But end to end system latency will be a sum of capture + encode + transmit buffering <-Network-> receive buffering + decode +display. So there are many other factors other than codecs which comes into play here.

---

What is the codec ARM926 MHz load?

The ARM926 MHz is approximately 25 MHz in DVSDK demo.

For an 8ch CIF in DVR ref design, ARM926 load due to codec is around 100MHz for H.264 encoder.

---

Are the codecs OS independent?

Yes, the codecs are OS independent. The codecs use Framework Components (FC) to satisfy all OS specific needs like cache, physical<->virtual memory translation, etc. FC supports both Linux and WinCE; to use the codecs on other OSes, users will need to port FC. FC releases include full source code.

---

Are the codecs CCS compiled libraries available?

Currently CCS compiled libraries are not available. TI is working on getting the CCS build available. Please check with your TI contact for latest update

---

What are the main features of H.264 encoder?

• H.264 limited High profile encoder upto level 5.0
• Able to encode upto 2Kx2K resolution
• Supports YUV 4:2:0P data as an input, output in byte stream format
• Supports both progressive and interlaced encode(no PicAFF or MBAFF support)
• Supports frame level re-entrancy
• Supports sub-window encoding, frame width >= encoded image width, no support for offset
• Supports CBR and VBR flavours of rate control
• Supports changing the size of video packets at create time
• Forced Intra Frame support
• Insertion of Picture timing information
• Supports the following tools, depending on quality factor chosen
  • 1 MV - 4MV
  • Quarter pel MV
  • 4x4 and 16x16 intra modes
  • CABAC/CAVLC
  • 8x8 transform
  • Scaling matrix
  • Loop filter
• Following BP/MP/HP features are NOT supported
  • ER tools – ASO/FMO/RS
  • B frames
  • PICAFF/MBAFF encode
  • Weighted prediction
  • More than 1 reference frame
  • MMCO and reference index reordering

---

Can H.264 encoder do all Intra frames encoding?

Yes, H.264 encoder can do all intra frames encoding. One has to set IntraFrameInterval with correct value.

---

There is no H.264 BP mentioned in the codec data sheet or availability chart, is it a subset of HP? If yes, will they be part of one library?

Yes, H.264 BP is a subset of HP encoder. There will be only one library for all BP, MP and HP profiles. The required profile can be chosen by giving the right configuration to the encoder. In case of decoder, any profile stream can be given and the decoder will decode it.

---

How does the quality of H.264 encoder look like?

Please get in touch with your TI contact for details of DM36x video quality.

---

What are the main features of H.264 decoder?

• H.264 universal Baseline/Main/High profile decoder upto level 4.2
• Supports Universal decode
• Able to decode upto 1080P or 1080i bitstream
• Support output of YUV 4:2:0 data only
• Supports frame level re-entrancy
• Supports sub-window decoding, Display width >= Actual image width, no support for offset
• Output order conformance using frame using bumping process supported
• Sliding window and Adaptive MMCO
• Following BP/HP feature set supported
  • B frames
  • Weighted prediction
  • CABAC/CAVLC
  • 8x8 transform (for lower resolution)
  • Scaling matrix
  • Interlaced support, PiCAFF and MBAFF
• Following BP feature NOT supported
  • ER tools – ASO/FMO/RS

---

Is the H.264 HP decoder universal?

The H.264 HP decoder supports all features of HP and MP. The only feature not available is FMO, ASO and RS. Apart from that it is true BP/MP/HP universal decoder. Version 2.00.00 will have the above features. Apart from that it will also be capable of performing closed loop decoding of 1080P@30fps on DM368. It will be available in May 2010.

---

Does H.264 support multiple sequences to be decoded without deleting and creating the instance again?

Decoder can decode multiple sequences as long as there is no change in the resolution. In case there is resolution change but the new resolution is lower than the earlier resolution, one call XDM_RESET control command and then run the new stream. The application level DPB/display buffer management and to be accordingly handled.

---

What are the modes in which H.264 Decoder can be invoked?

As per IVIDDEC2 specification, decoder supports two modes namely, decode header only mode and the normal mode. When decode header only mode is set, decoder parses only the headers(SPS and PPS) and return, while in normal mode it parses all type of NALs, including headers.

---

Will there be change in performance for H.264 BP and HP profile?

H.264 decoder: Yes, there will be change in performance when decoding H.264 BP vs decoding H.264 HP stream. This is because HP can have tool chains like B frames and weighted prediction which take more time.

H.264 encoder: In the current S/W setup, there will be no change in performance between H.264 BP and HP. This is because HP tool chains like B frame, weighted prediction are not supported. The limited H.264 HP encoder consists of CABAC, Intra8x8, Inter8x8 transform, scaling matrix which does not add to ARM or DMA load.

---

What is the roadmap of H.264 codec on DM365?

• Version 1.0 (superseded)
  • Stage 1 codec
  • 720P
  • GA - available
• Version 1.1 (available)
  • Build over version 1.00.00
  • 1080 support, interlaced, MBAFF support for decoder
  • GA - Dec 2009
• Version 2.0 (in progress)
  • All features of version 1.1
  • 1080P@30fps encoder and closed loop decoder on DM368
  • Smart codecs – Region of interest based encoding, pseudo two pass, low latency codec
  • GA – May 2010

---

Can H.264 codec do 1080P@30fps? Is there some new codec to achieve this?

H.264 encoder can do 1080P@30fps on DM368. This performance will be in ver 2.00.00 codec which will be available in May 2010. A new encoding mode/preset is added which will support this performance. The existing standard and high quality mode will also be available in the same library.

---

What type of error concealment is supported in H.264 decoder?

H.264 Decoder performs predictor based spatial and temporal error concealment on erroneous frames. Concealment is applied to only that part of picture which could not be decoded due to error or insufficient data. If a picture is concealed, the status is reflected in extended error code.

---

What is the DDR b/w of H.264 encoder and decoder?=

For H.264 encoder (STD quality mode), the DDR b/w is 230Mbytes/sec for 720P@30fps. For H.264 HP progressive decoder, the DDR b/w is 350Mbytes/sec for 720P@30fps.

---

What are the main features of Mpeg4 encoder?

• Mpeg-4 simple profile encoder upto level 5
• Able to encode upto 1080P resolution
• Supports YUV 4:2:2 interleaved and YUV 420 SP data as an input
• Performance of SXVGA@30fps with 4:2:2 ILE input
• Supports rotation of 90, 180, and 270 degree till D1 resolution
• Supports frame level re-entrancy
• Supports sub-window encoding, frame width >= encoded image width, no support for offset
• Supports CBR, VBR and CVBR flavours of rate control
• Support changing the size of video packets at create time
• Forced Intra Frame support
• Supports Short Video Header
• Compliant with the Video Buffering Verifier as per MPEG4 standard.
• Insertion of Picture timing information
• Supports the following SP tools
  • 1 MV
  • Half pel MV
  • Resync marker
  • AC/DC prediction for fixed QP
  • UMV
• NOT supported SP features
  • 4 MV
  • DP/RVLC/HEC

---

How does the quality of Mpeg4 encoder look like?

The overall quality of Mpeg4 encoder will be same as DM355 solution. There is new support of Low bitrate Rate control which gives better rate control at low bitrates.

---

What is the difference between 1.00.00 and 1.10.00 version of Mpeg4 encoder?

Version 1.10.00 is the latest version of VICP based Mpeg4 encoder. It has the below new features

• Support resolution upto 1920x1920.
• Supports new CVBR rate control
• Bug fixes over previous 1.00.00

---

What are the main features of Mpeg4 decoder?

• Mpeg4 simple profile closed loop decoder upto level 5, should be able to decode streams encoded by DM365 Mpeg-4 encoder
• Able to decode upto 1080P bitstream
• Support output of YUV 4:2:2 and YUV 420SP data only
• Supports rotation of 90, 180, and 270 degree
• Supports streams with Short Video Header
• Supports frame level re-entrancy
• Supports sub-window decoding, Display width >= Actual image width, no support for offset
• Supports the following SP tools
  • 1 MV
  • Half pel MV
  • Resync marker
  • AC/DC prediction
  • UMV
• NOT supported SP features
  • 4 MV
  • DP/RVLC/HEC

---

What type of error concealment is supported in Mpeg4 ASP decoder?

Mpeg4 ASP decoder performs predictor based spatial and temporal error concealment on erroneous frames. Concealment is applied to only that part of picture which could not be decoded due to error or insufficient data. If a picture is concealed, the status is reflected in extended error code.

---

What is the difference between 1.00.00 and 1.10.00 version of Mpeg4 decoder?

Version 1.10.00 is the latest version of VICP based Mpeg4 decoder. It has the below new features

• Support resolution upto 1920x1920.
• Bug fixes over previous 1.00.00

---

What are the main features of Jpeg encoder?

• The JPEG baseline sequential encoder algorithm shall be supported
• Tested for 10 MP image size
• Support YUV 4:2:0 planar, 4:2:2 planar, 4:2:0 SP and 4:2:2 interleaved input and be able to produce either yuv420 or yuv422 output bitstream
• Best performance of 66 MP/sec with 422 ILE input and 4:2:0 output (normal mode)
• Supports rotation of 90, 180, and 270 degree
• Supports frame level re-entrancy
• Supports sub-window encoding, frame width >= encoded image width, no support for offset
• Fixed Quant table with adjustable quality factor from 1 – 97
• Include a standard JPEG header
• Not supported
  • MJPEG as an interface, application can alternatively call JPEG and do the bitstream stitching and rate control

---

What are the main features of Jpeg decoder?

• The JPEG baseline sequential decoder algorithm shall be supported.
• Image size upto 64MP
• support YUV 4:2:0, 4:2:2, 4:2:2 V and 4:4:4 and Gray encoded i/p streams and output of YUV 4:2:2 interleaved and 4:2:0 SP data only
• Best performance of 61 MP/sec with 420 input and 4:2:2 ILE output (normal mode, 1:10 compression ratio)
• Supports resizing by various factors from 1/8 to 7/8
• Supports rotation of 90, 180, and 270 degree
• Supports frame level re-entrancy
• Supports sub-window decoding, Display width >= Actual image width, no support for offset
• Below features are not supported
  • JFIF header parsing whcih is skiped
  • MJPEG as an interface. Application can alternatively call JPEG decoder for each frame for MJPEG
  • Huffman tables and quantization tables for U and V components must be the same

---

What are the main features of Mpeg2 encoder?

• MPEG2 Simple/Main Profile @ High Level, upto 1080p/i
• I and P Frame Encoding.
• Image width and height as multiple of 16.
• Motion Estimation: full pixel, half pixel.
• Supports configuration of target bit rate
• Same source and target frame rate
• Adaptive Intra/Inter MB decision
• Interlace encoding (All the pictures will be field)
• Performance planned is 30fps 720P
• Not supported Features
  • Interlace encoding: Adaptive frame or field at MB level
  • B picture support
  • Adaptive frame or field DCT

---

What are the main features of Mpeg2 decoder?

• Supported Features (Runs on HDVICP)
• MPEG2 Simple/Main Profile @ High Level, upto 1080p/i resolution
• I, P and B pictures
• Restricted Slice structure
• Progressive and Interlace pictures
• Frame and Field DCT
• Motion Compensation
• Multiple Sequence Headers
• Compatible to MPEG1 ISO/IEC 11172-2 constrained parameter bitstreams
• Not supported Features
  • Frame repetition due to 3:2 pull down indication

---

What are the main features of VC1 decoder?

• Supports simple, main and advance profile, resolution support upto 1080p
• Baseline intra frame compression
• Variable-sized transform
• Overlapped transform
• 4 motion vectors per macroblock
• Luma Quarter-pixel motion compensation
• Cb, Cr Quarter-pixel motion compensation
• Start codes
• Extended motion vectors
• Loop filter
• Dynamic resolution change
• Adaptive macroblock quantization
• Bidirectional (B) frames
• Intensity compensation
• Range adjustment
• Sequence level metadata
• Entry point layer
• support of RCV versions V1 and V2.
• Not supported Features
  • Dynamic Resolution change
  • Interlace Support
  • Display metadata

---

Is VC1 encoder also available?

No, VC1 encoder is not available on DM365.

---

What are the main features of Mpeg4 ASP decoder?

• Supported Features (Runs on HDVICP)
• Universal ASP upto 1080P(I) resolution
• IVOP and PVOP
• DC prediction
• AC prediction
• 4 MV, unrestricted MV
• GOB Resynchronization
• Data Partitioning
• Reversible VLC
• Short Video Header
• BVOP
• Method1 Quantization
• Method2 Quantization
• Interlace support
• Quarter-pel Motion Compensation
• Multiple VOL Headers
• H.263 Baseline Profile 0 with levels 10 to 45
• Performance planned is 20 fps 720P ASP universal decoder measured over progressive content
• Not supported Features
  • Global Motion Compensation
  • Annex I, J, K and T of H.263 and level 50, 60 and 70

---

Is the Mpeg4 decoder universal?

There are two Mpeg4 decoders available - Version 1.00.00 which runs on MJCP is a closed loop Mpeg4 decoder. It has performance of SXVGA@30fps on 300 MHz device.

Mpeg4 Decoder version 2.00.00 runs on HDVICP and it is a universal Mpeg4 SP/ASP decoder. It has performance of 720P@24fps on 300MHz device.

---

What is smart codec?

Smart codec is name given to new H.264 codec available in version 2.00.00. It has features like region of interest encoding and pseudo two pass encoding which can help achieve superior perceptual quality compared to traditional encoding techniques.

---

What is Region of Interest coding?

There exist many applications where particular spatial and temporal regions of the video signal are more interest to the user than other areas. Examples are (i) in video conferencing applications, the viewer pays more attention to the face regions when compared to other regions, (ii) in security applications, areas of potential activity (e.g. doors, windows) are more important. These more important regions or the regions where the viewer pays more attention to are called regions of interest (ROI). In such scenarios it is important that the ROI areas are reproduced as reliable as possible since they contribute significantly towards the overall quality and perception of the video. When encoding a video signal the video encoder prioritizes the ROI areas and encodes them at higher fidelity when compared to non-ROI areas. This is achieved by assigning higher number of bits to the ROI areas when compared to non-ROI areas.

---

What is pseudo multi – pass encoding?

In pseudo-multi-pass encoding, initially, a low resolution of the video frame is encoded and statistics from this low resolution encoding are used when encoding the higher resolution of the video frame. The small resolution encoding can generate the following statistics which can be passed to high resolution encoding:

• Number of Intra MBs in the frame
• Number of Inter MBs in the frame
• Number of bits consumed by the frame
• Number of bits consumed by each MB
• Average QP value used for the frame
• Motion Vectors for each MB
• Macroblock coding mode for each MB

---

DM36x H.264 encoder FAQ

A DM36x H.264 encoder specific FAQ is available.

---

Will DM365 codecs run on DM368?

All s/w component of DM365 are 100% compatible with DM368 and vice versa. This includes all DVSDK components viz codecs, Framework Components, PSP, demos et.al.

---

Leave a Comment

Comments

Comments on DM365 Codecs FAQ

---

Pbansal said ...

Sujit nist said ...