Please note as of Wednesday, August 15th, 2018 this wiki has been set to read only. If you are a TI Employee and require Edit ability please contact x0211426 from the company directory.

XDM 1.x Semantics

From Texas Instruments Wiki
Jump to: navigation, search


XDM 1.x Semantics: General Information

The information provided in this section is applicable to any XDM 1.x interface. For illustration purposes the IAUDENC1 interface is sometimes used as an example. However, the examples provided are generic and apply to the other XDM 1.x interfaces.


Introduction

The XDAIS Algorithm Standard was developed to ease algorithm system integration. It originally defined two interfaces, the IALG interface which deals with the creation of algorithm instance and the IDMA3 interface which handles the DMA resources.

XDM is an extension to the IALG interface. It defines several interfaces between client applications and multimedia algorithms. In this topic we will discuss the semantics for the interfaces defined for audio, speech, imaging and video encode/decode algorithms: IAUDENC1, IAUDDEC1, ISPHENC1, ISPHDEC1, IIMGENC1, IIMGDEC1, IVIDENC1 and IVIDDEC2.

An XDM compliant algorithm still implements the methods required by the IALG interface but also provides the new methods control() and process(). The IALG algControl() method is not used; rather XDM defines a new control() method. For more information about IALG and XDM please read XDAIS Documentation, the XDAIS-DM (Digital Media) User Guide and the XDM FAQ. The XDM header files are provided with the XDAIS product. They are located in the ti.xdais.dm package (in the xdais_#_##/packages/ti/xdais/dm directory). The XDM 1.x header files have names that end with "1", with the exception of ividdec2.h. The XDM html documentation is available online, as well as in the XDAIS product (see xdais_#_##/docs/html/index.html).

The purpose of XDM is to enable plug and play capabilities for multimedia codecs. This is achieved by providing uniform APIs accross algorithm classes and defining common configuration parameters for each class.

Applications using XDM 1.x codecs have the following common flow:

  1. Creating Codec Instance and Setting Static Parameters
  2. Setting Dynamic Params
  3. Getting Buffer Requirements, Allocate Buffers
  4. Processing Loop

Invoking an XDM 1.x codec

An application can invoke an XDM compliant codec in two ways: directly using the XDM and IALG APIs and indirectly using the Codec Engine (CE) framework APIs (also called VISA APIs).

If the application invokes the XDM codec directly, the application is responsible for providing the codec with the requested resources. This can be performed by using the Framework Components APIs.

Using CE provides several advantages:

  • Common interface (VISA API) for local (same processor as the app) and remote( different processor than the app) algorithms. For example, on a heterogeneous platform, the same CE application can be used to invoke audio codecs that run on the GPP and on the DSP.
  • Cache Management for remote algorithms. When executing a remote algorithm, CE manages the cache on the remote processor.
  • Seamless resource management using Framework Components. Apps just see create() and delete() calls and all the resource negotiation (e.g. for memory, DMA, HW accellerators, etc) is hidden.
  • Easy to enable XDM spec sanity checking.

Codec Engine can be downloaded here. The CE API headers files are available in codec_engine_#_##/packages/ti/sdo/ce/*.

The following table presents both APIs for an XDM 1.x audio encoder. APIs are similar for the other classes. For illustration purposes the IAUDENC1 interface is used as an example.

Codec Engine invocation (VISA): Direct invocation (XDM):
codec_engine_#_##/packages/ti/sdo/ce/audio1/audenc1.h xdais_#_##/packages/ti/xdais/dm/iaudenc1.h
AUDENC1_Params encParams; IAUDENC1_Params encParams;
AUDENC1_DynamicParams dynParams; IAUDENC1_DynamicParams dynParams;
AUDENC1_Handle hEncode; IAUDENC1_Handle hEncode;
AUDENC1_Status encStatus; IAUDENC1_Status encStatus;
AUDENC1_InArgs inArgs; IAUDENC1_InArgs inArgs;
AUDENC1_OutArgs outArgs; IAUDENC1_OutArgs outArgs;
XDM1_BufDesc inBufDesc; XDM1_BufDesc inBufDesc;
XDM1_BufDesc outBufDesc; XDM1_BufDesc outBufDesc;
AUDENC1_Handle AUDENC1_create(Engine_Handle e, String name,
AUDENC1_Params *params);
ALG_Handle ALG_create(IALG_Fxns *fxns, IALG_Handle parent, IALG_Params *params);
Int32 AUDENC1_control(AUDENC1_Handle handle, AUDENC1_Cmd id,
AUDENC1_DynamicParams *params, AUDENC1_Status *status);
XDAS_Int32 (*control)(IAUDENC1_Handle handle, IAUDENC1_Cmd id,
IAUDENC1_DynamicParams *params, IAUDENC1_Status *status);
Int32 AUDENC1_process(AUDENC1_Handle handle, XDM1_BufDesc *inBufs,
XDM1_BufDesc *outBufs, AUDENC1_InArgs *inArgs, AUDENC1_OutArgs *outArgs);
XDAS_Int32 (*process)(IAUDENC1_Handle handle, XDM1_BufDesc *inBufs,
XDM1_BufDesc *outBufs, IAUDENC1_InArgs *inArgs,IAUDENC1_OutArgs *outArgs);

A few remarks about the APIs:

  • The control() and process() APIs are the new additions to XDM. For these methods, the XDM API is similar to the CE VISA API
  • ALG_create() is actually not an IALG API. It is part of ALG, a "wrapper" API that invokes the IALG APIs (and, depending on the codec, optionally its IDMA3 and IRES APIs), making them easier to use. Source code is provided in framework_components_#_##/packages/ti/sdo/fc/utils/api or codec_engine_#_##/packages/ti/sdo/ce/alg . For more information about the ALG API please read Section 3 in XDAIS API Reference User's Guide (Note that part of this document is out of date. The ALG interface has been updated since this document was published. See header files for current interface.)
  • These APIs will be discussed in more detail in the next sections.

Read the Resources, FAQ section for more information about where to find examples of CE and non-CE application using XDM codecs.

Flow of an Application using an XDM 1.x codec.

Creating Codec Instance and Setting Static Parameters

The internal state of a codec is defined by its configuration parameters. An XDM codec has two types of parameters, static and dynamic. The static parameters are used to initialize the state of the codec at creation time. They are also called creation parameters. The dynamic parameters can be used to update the internal state of the codec before processing each frame.

A codec instance is created and initialized with the same call:

Codec Engine invocation (VISA): Direct invocation (XDM):
AUDENC1_Handle AUDENC1_create(Engine_Handle e, String name,
AUDENC1_Params *params);
ALG_Handle ALG_create(IALG_Fxns *fxns, IALG_Handle p, IALG_Params *params);


In a CE application, an engine must be opened using the Engine_open() CE API before a codec instance can be created.The engine handle will be passed as an argument to AUDENC1_create() . Every CE application has a configuration file (extension .cfg) that defines the configuration of the engine used by the application. Depending on the configuration of the engine, the create() call will create a local or remote instance of the codec.

The String name argument is the name of the codec. For heterogenous platforms the codec name is defined when the server is created. (In a CE environment, server is the name given to the executable that runs on a remote processor. A server includes codec libraries,DSP/BIOS, Codec Engine, Framework Components code linked together. For more information about Server Packages see File:TIDC 2008 XDC for Multimedia Codecs.pdf.)

When a server is created, information about the codecs included in the server is stored in the server package in servers/<server_name>/package/info/server_name.info.js This information is used to configure the engine when the Engine.createFromServer() API is used. See Configuring_Codec_Engine_in_Arm_apps_with_createFromServer

/*
 *  ======== Engine Configuration ========
 */
var Engine = xdc.useModule('ti.sdo.ce.Engine');

var myEngine = Engine.createFromServer(
    "aacenc",
    "./aacenc_unitserver_evmdm6467.x64P",
    "ti.sdo.servers.aacenc_unitserver_evmdm6467");

For a homogenous platform there is no server and the codec name is defined in the application config file with the Engine.create() API

/*
 *  ======== Engine Configuration ========
 */
var Engine = xdc.useModule('ti.sdo.ce.Engine');
var myEngine = Engine.create("decode", [
    {name: "aacenc", mod:AACENC, local: true, groupId: 1},
]);

If the engine handle or the codec name are not correct the creation call will fail.

For a direct invocation, the create API accesses the codec throught the IALG function table. The pointer to the algorithm function table IALG_Fxns *fxns is passed as the first argument of the API. The second argument, IALG_Handle parent, is a pointer to another instance of the same codec. This feature allows to share read-only tables among several instanced of a codec to save memory. However, most of the time this argument is set to NULL because there is no parent instance. The last argument is a pointer to the algorithm static parameter structure IAUDENC1_Params.

Example of direct invocation:

IALG_Handle handle = (IALG_Handle)ALG_create((IALG_Fxns *)&AACENC_TII_IAACENC
                        ,(IALG_Handle) NULL, (IALG_Params *)&params)


Some of the static params are used to determine the memory requirements for the algorithm. The memory is allocated during the create call.

Passing the params in the create call is optional. If the parameters are not supplied, internal default parameters defined in the codec are used. The internal default parameters are documented in the codec User's Guide that is located in the codec package doc folder. For example, for the DM6446 h264 encoder the User's Guide is located in dm6446_dvsdk_combos_2_02/packages/ti/sdo/codecs/h264enc/docs.

It is not possible to initialize only a subset of the static parameters. The codec will update it's internal state with all the values passed in the parameter structure during the create call.

The valid parameter values are documented for each codec in the User's Guide. If one of the parameters passed is not valid, the creation of the codec instance will fail and the call will return AUDENC1_Handle = NULL


Setting Dynamic Parameters

The dynamic parameters are defined in the IAUDENC1_DynamicParams structure. These parameters are used to modify the run-time state of the codec before processing a frame.

The dynamic parameters are updated using the control API.

Codec Engine invocation (VISA): Direct invocation (XDM):
Int32 AUDENC1_control(AUDENC1_Handle handle, AUDENC1_Cmd id,
AUDENC1_DynamicParams *params, AUDENC1_Status *status);
XDAS_Int32 (*control)(IAUDENC1_Handle handle, IAUDENC1_Cmd id,
IAUDENC1_DynamicParams *params, IAUDENC1_Status *status);

The control API is new with XDM. It is used to send commands to the codec using a command id argument AUDENC1_Cmd id. The list of available commands is defined in the XDM_CmdId enumeration in xdm.h. Not all commands are implemented by all the codecs. Read codec User's Guide for supported commands. If a command is not supported the call should return the value AUDENC1_EUNSUPPORTED

A non-NULL pointer to the AUDENC1_Status structure must be passed in the API.

In most controll calls, AUDENC1_Status.data buffer will not be used. However, before performing a controll call, the pointer to the buffer must be initialized as follows (no matter what the command id is)

  AUDENC1_Status.data.size = sizeof(AUDENC1_Status);
  AUDENC1_Status.data.buf = NULL;

The AUDENC1_Status.data is a general purpose buffer that can be used as input or output depending on the command. This buffer is allocated and owned by the application. There is no support in the interface for the encoder to negotiate the size of the buffer. The codec should not hold a pointer to this buffer.

When a control() call completes successfully, the call returns the value AUDENC1_EOK . When a control() call fails, the call will return AUDENC1_EFAIL and the error ID will be provided in AUDENC1_Status.extendedError

The dynamic parameters are initialized using the XDM_SETPARAMS command id. This command id must be supported by all the codecs which need to update the dynamic parameters. It is not possible to update only a subset of the dynamic parameters. All the values in the dynamic parameter structure are taken into account. The codec User's Guide defines a valid range for each parameter. If the values used to initialize the dynamic parameters are not in the valid range the call will fail. It is not mandatory to update the dynamic parameters. The codec has internal defaults that will be used when the dynamic parameters are not set. The defaults are documented in the codec User's Guide.

The application can update the dynamic parameters of the codec at any time after the codec instance has been created.

Application Buffer Requirements

Before entering the processing loop, the application must allocate the input and output buffers needed by the codec to process the data. The size and number of these buffers is dependant on the codec static configuration. Since this dependancy is most clearly shown in the video encoder, the IVIDENC1 class will be used as an example in this section.

For example, for a video encoder, the size of the input buffers will depend on VIDENC1_Params.maxHeight, VIDENC1_Params.maxWidth, VIDENC1_Params.inputChromaFormat . These buffers must not be confused with the codec memory requirements which are documented in the codec user's guide and are allocated at creation time (scratch, constant ...).

The application will request this information from the codec by using a controll call with the XDM_GETBUFINFO command id. As a response to this command, the codec will update the VIDENC1_Status.bufInfo structure with the number of input and output buffers and with the buffer sizes required. For most codecs, this information depends only on the static params and will not change during the life of the codec instance. The video decoders make an exception to this rule and will be discussed in a specific section later.

The following example shows how the VIDENC1_Status.bufInfo structure is populated for a video encoder in response to a XDM_GETBUFINFO command . The information is dependant on the static parameters used to configure the video encoder.


Void appMain() {
...

/*Static Parameters for Encoder Configuration*/
VIDENC1_Params.maxHeight              = 480
VIDENC1_Params.maxWidth               = 720
VIDENC1_Params.inputChromaFormat      = XDM_YUV_420P

...

/* Get buffer information from video encoder */
status = VIDENC1_control(hEncode, XDM_GETBUFINFO, dynParams, &encStatus);

printf("VIDENC1_Status.bufInfo.minNumInBufs     = %ld /n", encStatus.bufInfo.minNumInBufs);
printf("VIDENC1_Status.bufInfo.minInBufSize[0]  = %ld /n", encStatus.bufInfo.minInBufSize[0]);
printf("VIDENC1_Status.bufInfo.minNumOutBufs    = %ld /n", encStatus.bufInfo.minNumOutBufs);
printf("VIDENC1_Status.bufInfo.minOutBufSize[0] = %ld /n", encStatus.bufInfo.minOutBufSize[0]);

...
}

Output:


VIDENC1_Status.bufInfo.minNumInBufs     = 1
VIDENC1_Status.bufInfo.minInBufSize[0]  = 518400
VIDENC1_Status.bufInfo.minNumOutBufs    = 1
VIDENC1_Status.bufInfo.minOutBufSize[0] = 345600

Processing Loop

The process() API is called repeatedly in a loop to process the data.

Codec Engine invocation (VISA): Direct invocation (XDM):
Int32 AUDENC1_process(AUDENC1_Handle handle, XDM1_BufDesc *inBufs,
XDM1_BufDesc *outBufs, AUDENC1_InArgs *inArgs, AUDENC1_OutArgs *outArgs);
XDAS_Int32 (*process)(IAUDENC1_Handle handle, XDM1_BufDesc *inBufs,
XDM1_BufDesc *outBufs, IAUDENC1_InArgs *inArgs,IAUDENC1_OutArgs *outArgs);

In addition to the codec instance handle, AUDENC1_Handle handle, the process API has two types of arguments.

The first type are the input/output buffer descriptors XDM1_BufDesc *inBufs, XDM1_BufDesc *outBufs. The buffer descriptors provide the number of input and output buffers, XDM1_BufDesc.numBufs, requested by the codec in response to the XDM_GETBUFINFO controll call. The buffer descriptors also provide the addresses, XDM1_BufDesc.descs.buf, and sizes XDM1_BufDesc.descs.bufSize of the buffers requested by the codec. In a Linux environment, the address is a virtual address. In a DSP/BIOS environment, the address is a physical address. In addition to this information, the buffer descriptors also provide for each buffer an access mask, XDM1_BufDesc.descs.accessMask, which will be filled by the codec to describe how the codec processor has accessed the buffer. This information is useful for cache management.

The second type of arguments are the input/output run time arguments passed in AUDENC1_InArgs *inArgs, AUDENC1_OutArgs *outArgs These arguments include information that is accessed or generated when each frame is processed. It is advantageous to make this information part of the process() call so that there is no need for an additional call to set or read it. A valid question is to ask why the input arguments are not part of the dynamic parameters structure, since this structure contains information that can be updated before each process() call. If the input arguments had been part of the dynamic structure, a control() call would have been mandatory before each process() call to set the input arguments. This would have been an issue because on a heterogenous platform such as OMAP3530 this call would be a remote call which has a large overhead. In the same way, many of the output arguments could have been part of the status structure, however an additional control() call would have been required to read their values ( see more details next section). More details about how the run time arguments are used are provided in the next class specific sections. The input arguments do not have internal default values. If a codec uses them, they must be initialized by the application.


If the process call completes successfully,the call returns the value AUDENC1_EOK. If an error occurs during processing, the process call will return AUDENC1_EFAIL and the error ID will be provided in AUDENC1_Status.extendedError. This is a 32-bit field, organized as follows: Bits 31-16 are reserved, bits 8-15 are XDM generic and are the same for all the classes. Bits 0-7 are codec and implementation specific and are described in the codec User's Guide.

After processing a frame it is often useful to get more information about the state of the codec by requesting the codec to update the AUDENC1_Status structure. This is done using a controll call with the XDM_GETSTATUS command id. Do not forget that before the control call, the pointer to the general purpose AUDENC1_Status.data buffer must be initialized as described in a previous section.


XDM 1.x Commands

XDM 1.x supports the following seven commands:

  • XDM_GETSTATUS: Will update the AUDENC1_Status structure with the status of the last process call.
  • XDM_SETPARAMS: Will set the dynamic parameters AUDENC1_DynamicParams and may perform class specific tasks. See class specific information.
  • XDM_GETBUFINFO: Will update the AUDENC1_Status.bufInfo structure.
  • XDM_SETDEFAULT: Will set static and dynamic parameters to default values.
  • XDM_FLUSH: Command implemented only by Video Encoders and Decoders which support B Frames.
  • XDM_RESET: Will set IALG memTab values to default values and prepares the alg so that the next process() call can be processed as first frame. Some algs may call algInit() to help reset their internal state. May also have class specific behavior.
  • XDM_GETVERSION: Will provide version string of the codec - the format of the string is undefined. This is an optional cmd and many codecs do not support it.

XDM 1.x Semantics: Class Specific Information

The following sections provide class specific information. A set of sample static and dynamic parameters is provided for each class. These parameters are used in the DMAI application and are working with the codecs from Texas Instruments included in DVSDK 2.0. It is recommended that an XDM 1.x codec should support these values in order to ensure that the codec will work seemlessly in the DVSDK demos and Gstreamer. The details of the setup and processing loop are described for each class.


IAUDENC1

Sample Static and Dynamic Parameters

/* create params */
AUDENC1_Params.sampleRate                = 48000;
AUDENC1_Params.bitRate                   = 288000;
AUDENC1_Params.channelMode               = IAUDIO_2_0;
AUDENC1_Params.dataEndianness	         = XDM_LE_16;
AUDENC1_Params.encMode                   = IAUDIO_CBR;
AUDENC1_Params.inputFormat               = IAUDIO_INTERLEAVED;
AUDENC1_Params.inputBitsPerSample        = 16;
AUDENC1_Params.maxBitRate                = 192000;
AUDENC1_Params.dualMonoMode	         = IAUDIO_DUALMONO_LR;
AUDENC1_Params.crcFlag	                 = XDAS_FALSE;
AUDENC1_Params.ancFlag	                 = XDAS_FALSE;
AUDENC1_Params.lfeFlag	                 = XDAS_FALSE;
 
/* dynamic params*/
AUDENC1_DynamicParams.bitRate            = 288000;
AUDENC1_DynamicParams.sampleRate         = 48000;
AUDENC1_DynamicParams.channelMode        = IAUDIO_2_0;
AUDENC1_DynamicParams.lfeFlag            = XDAS_FALSE;
AUDENC1_DynamicParams.dualMonoMode       = IAUDIO_DUALMONO_LR;
AUDENC1_DynamicParams.inputBitsPerSample = 16;

Processing Loop

An XDM 1.x audio encoder uses always one input and one output buffer. This means that XDM1_BufDesc.numBufs are always initialized as follows:

    inBufDesc.numBufs           = 1;
    outBufDesc.numBufs          = 1;

In order to process a frame an XDM 1.x audio encoder must know the number of input samples for each channel. This information is passed in the input argument AUDENC1_InArgs.numInSamples and is computed as follows. The number of input bytes passed to the audio encoder (this is the value passed in inBufDesc.descs[0].bufSize ) is divided by the number of bytes in a sample (defined as AUDENC1_DynamicParams.inputBitsPerSample >>3) to obtain the number of input samples. The number of channels is determined by AUDENC1_DynamicParams.channelMode. For example the WMA encoder from Texas Instruments supports AUDENC1_DynamicParams.channelMode = IAUDIO_1_0, which corresponds to one channel and AUDENC1_DynamicParams.channelMode = IAUDIO_2_0, which corresponds to two channels. Following pseudo-code shows how to compute AUDENC1_InArgs.numInSamples for the TI WMA encoder.

    inputBytes = inBufDesc.descs[0].bufSize ;
    inputBytesPerSample = (AUDENC1_DynamicParams.inputBitsPerSample >>3);
    numChannels = AUDENC1_DynamicParams.channelMode == IAUDIO_1_0 ? 1 : 2;
    numInSamples         = (inputBytes /inputBytesPerSample)/numChannels

The second input argument is AUDENC1_InArgs.ancData. If the ancilliary buffer data is not used it must be initialized as follows:

    AUDENC1_InArgs.ancData.bufSize      = 0L;
    AUDENC1_InArgs.ancData.buf          = (XDAS_Int8 *) NULL;


After completion of the process() call the, the audio encoder provides useful information in the AUDENC1_OutArgs structure. AUDENC1_OutArgs.bytesGenerated represent the number of bytes generated during the process call. and AUDENC1_OutArgs.numInSamples provide the number of samples consummed during the call.


XDM Commands

This section presents IAUDENC1 Class specific XDM Commands

  • XDM_SETPARAMS: For AAC audio encoders, in addition to setting the dynamicParams, this command will re-initialize the Huffman tables if the sample rate or the number of channels has changed.
  • XDM_RESET: For AAC audio encoders, in addition to setting the dynamicParams, this command will re-initialize the Huffman tables.

IAUDDEC1

Sample Static and Dynamic Parameters

AUDDEC1_Params.outputPCMWidth              = 16
AUDDEC1_Params.pcmFormat                   = IAUDIO_INTERLEAVED
AUDDEC1_Params.dataEndianness              = XDM_BYTE

AUDDEC1_DynamicParams.downSampleSbrFlag    = XDAS_FALSE

Processing Loop

An XDM 1.x audio decoder may use the following input arguments to decode a frame:

typedef struct IAUDDEC1_InArgs {
    XDAS_Int32 size;
    XDAS_Int32 numBytes;
    XDAS_Int32 desiredChannelMode;
    XDAS_Int32 lfeFlag;
} IAUDDEC1_InArgs;

The application must provide in AUDDEC1_InArgs.numBytes the number of valid bytes provided in the input buffer. In addition to that, the channel related information must be provided in AUDENC1_InArgs.desiredChannelMode. This field is populated with one of the enumerators defined in IAUDIO_ChannelMode. The audio decoder User's Guide provides more information on the supported channel modes. Some audio decoders such as AAC and AC3 support a Low Frequency Enhancement (LFE) channel. The AUDDEC1_InArgs.lfeFlag tells the decoder whether the LFE channel data is desired in the output.

After completion of the process() call, the audio decoder provides useful information in the AUDDEC1_OutArgs output arguments. AUDDEC1_OutArgs.sampleRate informs the application about the sample rate of the stream. This information is necessary to configure an audio peripheral driver. The number of bytes generated by the process call can be computed using AUDDEC1_OutArgs.numSamples, AUDDEC1_OutArgs.channelMode, AUDDEC1_Status.outputBitsPerSample . An XDM_GETSTATUS controll call is necessary to determine AUDDEC1_Status.outputBitsPerSample .

For example, the number of bytes generated by the process call is computed as follows in DMAI (Adec1.c). Notice that the number of bytes per sample is hardcoded to 2. All the audio decoders available today on the DaVinci and OMAP platforms have output samples of 2 bytes (16 bits).


    channels = numChans[outArgs.channelMode];
    Buffer_setNumBytesUsed(hOutBuf, outArgs.numSamples * 2 * channels);

ISPHENC1

Reference Guides

Sample Static and Dynamic Parameters

SPHENC1_Params.frameSize                     = 160;
SPHENC1_Params.compandingLaw                 = 1;
SPHENC1_Params.packingType                   = 0;
SPHENC1_Params.vadSelection	             = 0;
SPHENC1_Params.codecSelection                = 0;
SPHENC1_Params.bitRate                       = 0;
SPHENC1_Params.reserved                      = 0;
SPHENC1_Params.tablesPtr                     = NULL;
SPHENC1_DynamicParams.frameSize              = 0;
SPHENC1_DynamicParams.bitRate 		     = 0;
SPHENC1_DynamicParams.mode	             = 0;
SPHENC1_DynamicParams.vadFlag                = ISPEECH1_VADFLAG_DEFAULT;
SPHENC1_DynamicParams.noiseSuppressionMode   = 0;
SPHENC1_DynamicParams.ttyTddMode             = 0;
SPHENC1_DynamicParams.dtmfMode               = 0;
SPHENC1_DynamicParams.dataTransmit           = 0;
SPHENC1_DynamicParams.reserved               = 0;

Processing Loop

The ISPHENC1 interface was defined to support only speech codecs which require one input and one output buffer. Therefore single buffer descriptors, XDM1_SingleBufDesc are used in the process() API:

Codec Engine invocation: Direct invocation:
Int32 SPHENC1_process(SPHENC1_Handle handle, XDM1_SingleBufDesc *inBuf, XDM1_SingleBufDesc *outBuf, SPHENC1_InArgs *inArgs, SPHENC1_OutArgs *outArgs);XDAS_Int32 (*process)(ISPHENC1_Handle handle, XDM1_SingleBufDesc *inBuf, XDM1_SingleBufDesc *outBuf, ISPHENC1_InArgs *inArgs,ISPHENC1_OutArgs *outArgs);

An ISPHENC1-compliant speech encoder may use the following input arguments:

typedef struct ISPHENC1_InArgs {
    XDAS_Int16 size;
    XDAS_Int16 nullTrafficChannel;
    XDM1_SingleBufDesc data;
} ISPHENC1_InArgs;

ISPHENC1_InArgs.nullTrafficChannel is used to enable the null traffic channel frames for CDMA speech encoders. Can be ignored if not used. The ISPHENC1_InArgs.data must be initialized as follows even when not used:

ISPHENC1_InArgs.data.buf = (XDAS_Int8 *) NULL;
ISPHENC1_InArgs.data.bufSize = 0L;

During the process() call, the decoder will update the ISPHENC1_OutArgs.frameType field to indicate which type of frame was encoded. For normal frames, the application will assume that the number of bytes generated by the process() call is the size of the output buffer passed in XDM1_SingleBufDesc.bufSize. This means that the input buffer was completely filled with encoded data.

ISPHDEC1

Sample Static and Dynamic Parameters

SPHDEC1_Params.compandingLaw            = 1
SPHDEC1_Params.packingType              = 0
SPHDEC1_Params.codecSelection           = 0
SPHDEC1_Params.bitRate                  = 0
SPHDEC1_Params.reserved                 = 0
SPHDEC1_Params.tablesPtr                = NULL
SPHDEC1_DynamicParams.postFilter	= 0

Processing Loop

Similar to ISPHENC1, the ISPHDEC1 interface was defined to support only speech codecs which require one input and one output buffer. Therefore single buffer descriptors, XDM1_SingleBufDesc are used in the process() API.

An XDM 1.x speech decoder may use the following input arguments:

typedef struct ISPHDEC1_InArgs {
    XDAS_Int16 size;
    XDAS_Int16 frameType;
    XDM1_SingleBufDesc data;
} ISPHDEC1_InArgs;

If a decoder supports several types of frames, the application must initialize ISPHDEC1_InArgs.frameType to inform the decoder what type of frame must be decoded. The ISPHDEC1_InArgs.data must be initialized as follows even when not used:

    ISPHDEC1_InArgs.data.buf = (XDAS_Int8 *) NULL;
    ISPHDEC1_InArgs.data.bufSize = 0L;

The ISPHDEC1_OutArgs.dataSize argument is useful only for wideband CDMA speech decoders which receive streams with raw data along with the compressed speech. The ISPHDEC1_OutArgs.dataSize argument is used to let the application know how much raw data is in the decoded buffer.


IIMGENC1

Sample Static and Dynamic Parameters

IIMGENC1_Params.maxHeight                   = 0
IIMGENC1_Params.maxWidth                    = 0
IIMGENC1_Params.maxScans                    = XDM_DEFAULT
IIMGENC1_Params.dataEndianness              = XDM_BYTE
IIMGENC1_Params.forceChromaFormat           = XDM_YUV_422P
IIMGENC1_DynamicParams.numAU                = XDM_DEFAULT
IIMGENC1_DynamicParams.inputChromaFormat    = XDM_YUV_422ILE
IIMGENC1_DynamicParams.inputHeight          = 0
IIMGENC1_DynamicParams.inputWidth           = 0
IIMGENC1_DynamicParams.captureWidth         = 0
IIMGENC1_DynamicParams.generateHeader       = XDM_ENCODE_AU
IIMGENC1_DynamicParams.qValue               = 75


Processing

During the process() call, the encoder will update IIMGENC1_OutArgs.bytesGenerated to notify the application about the size of the encoded frame.



IIMGDEC1

Sample Static and Dynamic Parameters

IIMGDEC1_Params.maxHeight                   = 576
IIMGDEC1_Params.maxWidth                    = 720
IIMGDEC1_Params.maxScans                    = 15
IIMGDEC1_Params.dataEndianness              = XDM_BYTE
IIMGDEC1_Params.forceChromaFormat           = XDM_YUV_422ILE
IIMGDEC1_DynamicParams.numAU                = XDM_DEFAULT
IIMGDEC1_DynamicParams.decodeHeader         = XDM_DECODE_AU
IIMGDEC1_DynamicParams.displayWidth         = 0


Processing

During the process() call the decoder updates the IIMGDEC1_OutArgs.bytesConsumed to inform the application about the number of bytes from the input buffer that were used to generate the current decoded frame.

The application can also find out what the dimensions of the output image are by requesting this information from the codec through an XDM_GETSTATUS controll call. IIMGDEC1_Status.outputHeight, IIMGDEC1_Status.outputWidth provide this information.

Motion JPEG (MJPEG) Processing

When using the JPEG decoder in a loop to decode several frames, for example in a Motion JPEG (MJPEG) type of application, it is necessary to perform an XDM_SETPARAMS control() call before each process() call. The XDM_SETPARAMS should call internally a reset() function that will reset the necessary parameters for the next process() call.

/* MJPEG processing loop */
Loop{
 
    control(XDM_SETPARAMS);
 
    process();
 
}

IVIDENC1

Sample Static and Dynamic Parameters

VIDENC1_Params.encodingPreset            = XDM_DEFAULT
VIDENC1_Params.rateControlPreset         = IVIDEO_LOW_DELAY
VIDENC1_Params.maxHeight                 = 720
VIDENC1_Params.maxWidth                  = 1280
VIDENC1_Params.maxFrameRate              = 30000
VIDENC1_Params.maxBitRate                = 6000000
VIDENC1_Params.dataEndianness            = XDM_BYTE
VIDENC1_Params.maxInterFrameInterval     = 0
VIDENC1_Params.inputChromaFormat         = XDM_YUV_420P
VIDENC1_Params.inputContentType          = IVIDEO_PROGRESSIVE
VIDENC1_Params.reconChromaFormat         = XDM_CHROMA_NA
VIDENC1_DynamicParams.inputHeight	 = 720
VIDENC1_DynamicParams.inputWidth	 = 1280
VIDENC1_DynamicParams.refFrameRate       = 30000
VIDENC1_DynamicParams.targetFrameRate	 = 30000
VIDENC1_DynamicParams.targetBitRate	 = 6000000
VIDENC1_DynamicParams.intraFrameInterval = 30
VIDENC1_DynamicParams.generateHeader	 = XDM_DECODE_AU
VIDENC1_DynamicParams.captureWidth	 = 0
VIDENC1_DynamicParams.forceFrame	 = IVIDEO_NA_FRAME
VIDENC1_DynamicParams.interFrameInterval = 1
VIDENC1_DynamicParams.mbDataFlag	 = 0

Processing Loop

The application must also provide information about the input and output buffers. For the input buffers, the VIDENC1 class uses a video input buffer descriptor buffer IVIDEO1_BufDescIn that includes more information than a simple buffer descriptor XDM_BufDesc used for the output buffers.

During the process() call, the encoder will update the output structure IVIDENC1_OutArgs with information useful to the application:

  • IVIDENC1_OutArgs.bytesGenerated notifies the application about the size of the encoded frame.
  • IVIDENC1_OutArgs.reconBufs provides information related to the reconstructed buffer.
  • IVIDENC1_OutArgs.encodedFrameType let's the application know what type of frame was encoded.
  • IVIDENC1_OutArgs.encodedBuf and IVIDENC1_OutArgs.outputID are useful for video encoders which support B frames.

B-Frame Support

  • Parameter Configuration

In order to use B-frames, the VIDENC1_Params.maxInterFrameInterval and VIDENC1_DynamicParams.interFrameInterval must be configured properly to notify the encoder how many B-frames should be encoded.

  • Encoded Stream Structure

When encoding a stream with B-frames, the group of pictures (GOP) structure will include I, P and B-frames.The first P frame is encoded using the previous I frame as a reference. The following P frames are encoded using the previous P frame as reference. The frames between subsequent I or P frames are encoded as B frames. For example the GOP can look like this: IBBPBBPBBPBBP.... Encoding a stream with N B-frames means that there are N B-frames between subsequent I or P frames. In the example above, the stream is encoded with N=2 B-frames.

It is important to understant that in order to encode a B-frame, the encoded content of the I or P reference frames is necessary. This means that in order to encode a stream with N B-frames, the encoder must buffer N input frames before it has access to the input frame that will be encoded as P (or I) frame and used as reference for the B-frames. As a consequence, the encoding order for the GOP provided above is the following: IPBBPBBPBB .... This is the order of the frames in the encoded stream. When the stream is decoded, the frames are reordered in the display order: IBBPBBPBBP ...


  • Videnc1 01.jpg


  • Buffer Management

The encoder will use IVIDENC1_InArgs.inputID in order to keep track of the input frames. Zero is considered an invalid inputID. The first N process() calls are used to buffer the input frames and there is no encoding performed. When the encoder unlocks a buffer it will provide the buffer ID in IVIDENC1_OutArgs.outputID.

  • Videnc1 04.jpg
  • Encoding Completion: Flushing the Residual Frames

Assume a stream is encoded with N B-frames, i e there are N B-frames between subsequent I and P frames. The application must notify the encoder when processing completes using a XDM_FLUSH controll call. After receiving this command, the encoder will require N process() calls before it completes processing. During each of these calls the encoder will encode and release one of the frames it has buffered. The ID of the released buffer is communicated to the application using the IVIDENC1_OutArgs.outputID. The frames encoded during these last process() calls are either P or I frames since there is no more reference frame to encode B frames.

  • Videnc1 05.jpg

Interlaced Support

If the encoder supports interlaced input, the correct input type must be configured with VIDENC1_Params.inputContentType. Before the process() call the field order must be defined in IVIDENC1_InArgs.topFieldFirstFlag.


XDM Commands

  • XDM_FLUSH: This command must be supported only by codecs which support B-frames.

IVIDDEC2

Sample Static and Dynamic Parameters

VIDDEC2_Params.maxHeight                 = 576
VIDDEC2_Params.maxWidth                  = 720
VIDDEC2_Params.maxFrameRate              = 30000
VIDDEC2_Params.maxBitRate                = 6000000
VIDDEC2_Params.dataEndianness            = XDM_BYTE
VIDDEC2_Params.forceChromaFormat         = XDM_YUV_420P
VIDDEC2_DynamicParams.decodeHeader	 = XDM_DECODE_AU
VIDDEC2_DynamicParams.displayWidth	 = 0
VIDDEC2_DynamicParams.frameSkipMode      = IVIDEO_NO_SKIP
VIDDEC2_DynamicParams.frameOrder	 = IVIDDEC2_DISPLAY_ORDER
VIDDEC2_DynamicParams.newFrameFlag	 = 0
VIDDEC2_DynamicParams.mbDataFlag	 = 0

Application Buffer Requirements

In the IVIDDEC2 class the codec notifies the application about the application buffer requirements in two steps.

  • The first step is similar to the other classes:the application creates the decoder instance by invoking the VIDDEC2_create() API. With this call the application passes the static creation parameters VIDDEC2_Params to the decoder. After that, the application queries the decoder using a XDM_GETBUFINFO controll() call for the application buffer requirements. The decoder will respond with requirements based on the static creation parameters. Indeed, at this point, the decoder has no direct knowledge about the stream it will decode and can not communicate to the application the buffer requirements required by the specific stream.
  • The second step occurs after the decoder has decoded the first frame. The application queries a second time the decoder using a XDM_GETBUFINFO controll() call for the application buffer requirements. At this point the decoder will respond to this command with the buffer requirements for the specific stream it is decoding.
  • Viddec2 07.jpg

Processing and Buffer Management

Video decoders may use multiple reference frames to enhance the quality of the decoded video. A video decoder will usually lock these reference frames as long as they are needed and the application should not overwrite the locked frames. In order to support this behavior, the IVIDDEC2 interface provides IVIDDEC2_InArgs.inputID to identify an output buffer descriptor provided to the decoder and IVIDDEC2_OutArgs.freeBufID[] to characterize the output buffer descriptors released by the decoder. Notice that the freeBufID[] is an array which allows the decoder to release more than one buffer during the same process() call.


Do not be mislead by the name "inputID" and think that theses IDs refer to input buffers. For the decoder, the IDs ALWAYS refer to output buffer descriptors.

The IDs of the buffers ready for display are provided upon completion of the process() call in the IVIDDEC2_OutArgs.outputID[] array. The corresponding display frame buffer descriptors are provided in IVIDDEC2_OutArgs.displayBufs[].

Zero is not a valid inputID. The first zero value in the IVIDDEC2_OutArgs.outputID[] and IVIDDEC2_OutArgs.outputID[] indicates the end of valid IDs in the arrays.


  • Viddec2 03.jpg

Processing Completion: Flushing the Display Buffers

When there are no more input frames, the decoder will be notified using the XDM_FLUSH command. After receiving this command, the decoder will release a display buffer per process() call. At this point there is no more input data, however a dummy input buffer must still be used as input buffer for the process() API so that the Codec Engine does not generated an error.

  • Viddec2 06.jpg

Resources, FAQ

Where can we find examples of Codec Engine Applications using XDM codecs?

  • The Davinci Multimedia Application Interface (DMAI) is a CE application that provides for each class configuration parameters which work with all the codecs available from Texas Instruments in that class. DMAI also provides sample applications which help understand the XDM 1.x semantics.
  • The DVSDK demos show how a real life application uses an XDM codec.
  • Codec Engine examples provided with the DVSK include both, simple XDM copy codecs and applications using these codecs. Examples are located in codec_engine_#_##/examples/ti/sdo/ce/examples

Where can we find examples of applications using XDM codecs without Codec Engine?

  • In DVSDK 2.x, non-CE application are included with the codec packages. For example, for DM6446 h264 encoder, a DSP non-CE CCS project is provided in dm6446_dvsdk_combos_2_02/packages/ti/sdo/codecs/h264enc/app/Client/Build .