Please note as of Wednesday, August 15th, 2018 this wiki has been set to read only. If you are a TI Employee and require Edit ability please contact x0211426 from the company directory.

C6000 Dynamic Loader

From Texas Instruments Wiki
Jump to: navigation, search


This article is currently under construction! More details to come!

The C6000 compiler supports dynamic loading, provided you build with EABI. If you are not already familiar with the limitations of EABI support in the C6000 compiler, please see EABI Support in C6000 Compiler.

Dynamic Loader Architecture

Dynamic Loader Architecture.jpg

Dynamic Loader Reference Implementation

In conjunction with the C6000 Compiler Tools support for Lightweight and Linux Dynamic Linking Models, Texas Instruments Incorporated provides the source code for a reference implementation of a dynamic loader. The source code is provided under a TI BSD open source license. You can download the latest version of the source for the dynamic loader reference implementation from the project page on gforge.ti.com. For a detailed description of the dynamic linking models that are supported in the C6000 Compiler, please see the Dynamic Linking article.

DL Reference Implementation.jpg

Dynamic Loader API

The Dynamic Loader Application Programming Interface (DLOAD API) defines the boundary between the dynamic loader core and the dynamic loader wrapper. The Dynamic Loader Client / Wrapper provides host operating system services to the dynamic loader: file I/O, host memory management, target memory management, etc. The wrapper is host-specific, but independent of what object file format or target processor is supported in the dynamic loader core.

The Dynamic Loader Core provides object file format-specific and target-specific services to the dynamic loader: read and comprehend object file content, perform target-specific processing of dynamic relocations, etc.

The DLOAD API defines exactly which functions and services are to be provided by the wrapper, and which functions and services are to be provided by the dynamic loader core. In general, the naming convention for the DLOAD API is as follows:

  • Client / Wrapper Provided Functions
DLIF_* prefix
  • Core Provided Functions
DLOAD_* prefix

The following sections will go into detail about the functions and/or data structures that are to be managed by each part of the dynamic loader.

Dynamic Loader Client /Wrapper

File I/O

The client side of the dynamic loader must provide basic file I/O capabilities so that the core loader has random access into any object file that it is asked to load.

The client side of the dynamic loader must provide a definition of the LOADER_FILE_DESC in dload_filedefs.h. This allows the core loader to be independent of how the client accesses raw data in an object file.

/*---------------------------------------------------------------------------*/
/* DLIF_fseek()                                                              */
/*                                                                           */
/*    Seek to a position in a file (accessed via 'stream') based on the      */
/*    values for offset and origin.                                          */
/*                                                                           */
/*---------------------------------------------------------------------------*/
int      DLIF_fseek(LOADER_FILE_DESC *stream, int32_t offset, int origin);
/*---------------------------------------------------------------------------*/
/* DLIF_ftell()                                                              */
/*                                                                           */
/*    Return the current file position in the file identified in the         */
/*    LOADER_FILE_DESC pointed to by 'stream'.                               */
/*                                                                           */
/*---------------------------------------------------------------------------*/
int32_t  DLIF_ftell(LOADER_FILE_DESC *stream);
/*---------------------------------------------------------------------------*/
/* DLIF_fread()                                                              */
/*                                                                           */
/*    Read 'size' * 'nmemb' bytes of data from the file identified in the    */
/*    LOADER_FILE_DESC object pointed to by 'stream', and write that data    */
/*    into the memory accessed via 'ptr'.                                    */
/*                                                                           */
/*---------------------------------------------------------------------------*/
size_t   DLIF_fread(void *ptr, size_t size, size_t nmemb,
                    LOADER_FILE_DESC *stream);
/*---------------------------------------------------------------------------*/
/* DLIF_fclose()                                                             */
/*                                                                           */
/*    Close a file that was opened on behalf of the core loader. Ownership   */
/*    of the file pointer in question belongs to the core loader, but the    */
/*    client has exclusive access to the file system.                        */
/*                                                                           */
/*---------------------------------------------------------------------------*/
int      DLIF_fclose(LOADER_FILE_DESC *fd);

Host Memory Management

Allocate and free host memory as needed for the dynamic loader's internal data structures. If the dynamic loader resides on the target architecture, then this memory is allocated from a target memory heap that must be managed separately from memory that is allocated for a dynamically loaded object file.

/*---------------------------------------------------------------------------*/
/* DLIF_malloc()                                                             */
/*                                                                           */
/*    Allocate 'size' bytes of memory space that is usable as scratch space  */
/*    (appropriate for the loader's internal data structures) by the dynamic */
/*    loader.                                                                */
/*                                                                           */
/*---------------------------------------------------------------------------*/
void*    DLIF_malloc(size_t size);
/*---------------------------------------------------------------------------*/
/* DLIF_free()                                                               */
/*                                                                           */
/*    Free memory space that was previously allocated by DLIF_malloc().      */
/*                                                                           */
/*---------------------------------------------------------------------------*/
void     DLIF_free(void* ptr);

Target Memory Allocator Interface

The client side of the dynamic loader must create and maintain an infrastructure to manage target memory. The client must keep track of what target memory is associated with each object segment, allocating target memory for newly loaded objects and release target memory that is associated with objects that are being unloaded from the target architecture.

The two client-supplied functions, DLIF_allocate() and DLIF_release(), are used by the core loader to interface into the client side's target memory allocator infrastructure.

/*---------------------------------------------------------------------------*/
/* DLOAD_SEGMENT_FLAGS - segment characteristics.                            */
/*---------------------------------------------------------------------------*/
typedef uint32_t DLOAD_SEGMENT_FLAGS;
static const int DLOAD_SF_executable = 0x1;  /* Memory must be executable    */
static const int DLOAD_SF_relocatable = 0x2; /* Segment must be relocatable  */
/*---------------------------------------------------------------------------*/
/* DLOAD_MEMORY_SEGMENT - Define structure to represent placement and size   */
/*      details of a segment to be loaded.                                   */
/*---------------------------------------------------------------------------*/
struct DLOAD_MEMORY_SEGMENT
{
   uint32_t       target_page;          /* requested/returned memory page    */
   TARGET_ADDRESS target_address;       /* requested/returned address        */
   uint32_t       objsz_in_bytes;       /* size of init'd part of segment    */
   uint32_t       memsz_in_bytes;       /* size of memory block for segment  */
};
/*---------------------------------------------------------------------------*/
/* DLOAD_MEMORY_REQUEST - Define structure to represent a target memory      */
/*      request made by the core loader on behalf of a segment that the      */
/*      loader needs to relocate and write into target memory.               */
/*---------------------------------------------------------------------------*/
struct DLOAD_MEMORY_REQUEST
{
   LOADER_FILE_DESC            *fp;           /* file being loaded           */
   struct DLOAD_MEMORY_SEGMENT *segment;      /* obj for req/ret alloc       */
   void                        *host_address; /* ret hst ptr from DLIF_copy()*/
   BOOL                         is_loaded;    /* returned as true if segment */
                                              /* is already in target memory */
   uint32_t                     offset;       /* file offset of segment's    */
                                              /* raw data                    */
   uint32_t                     flip_endian;  /* endianness of trg opp host  */
   DLOAD_SEGMENT_FLAGS          flags;        /* allocation request flags    */
   uint32_t                     align;        /* align of trg memory block   */
};
/*---------------------------------------------------------------------------*/
/* DLIF_allocate()                                                           */
/*                                                                           */
/*    Given a DLOAD_MEMORY_REQUEST created by the core loader, allocate      */
/*    target memory to fulfill the request using the target memory           */
/*    management infrastrucutre on the client side of the dynamic loader.    */
/*    The contents of the DLOAD_MEMORY_REQUEST will be updated per the       */
/*    details of a successful allocation.  The allocated page and address    */
/*    can be found in the DLOAD_MEMORY_SEGMENT attached to the request.      */
/*    The boolean return value reflects whether the allocation was           */
/*    successful or not.                                                     */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLIF_allocate(struct DLOAD_MEMORY_REQUEST *req);
/*---------------------------------------------------------------------------*/
/* DLIF_release()                                                            */
/*                                                                           */
/*    Given a DLOAD_MEMORY_SEGMENT description, free the target memory       */
/*    associated with the segment using the target memory management         */
/*    infrastructure on the client side of the dynamic loader.               */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLIF_release(struct DLOAD_MEMORY_SEGMENT* ptr);

Target Memory Read/Write Access

The client side's target memory allocator infrastructure communicates with the core loader through the DLOAD_MEMORY_REQUEST and DLOAD_MEMORY_SEGMENT data structures defined above. To complete the loading of an object segment, the segment may need to be relocated before it is actually written to target memory in the space that was allocated for it by DLIF_allocate().

The client side of the dynamic loader provides two functions to help complete the process of loading an object segment, DLIF_copy() and DLIF_write().

These functions help to make the core loader truly independent of whether it is running on the host or target architecture and how the client provides for reading/writing from/to target memory.

/*---------------------------------------------------------------------------*/
/* DLIF_copy()                                                               */
/*                                                                           */
/*    Copy segment data from the object file described in the 'fp' and       */
/*    'offset' of the DLOAD_MEMORY_REQUEST into host accessible memory so    */
/*    that it can relocated or otherwise manipulated by the core loader.     */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLIF_copy(struct DLOAD_MEMORY_REQUEST* req);
/*---------------------------------------------------------------------------*/
/* DLIF_write()                                                              */
/*                                                                           */
/*    Once the segment data described in the DLOAD_MEMORY_REQUEST is ready   */
/*    (relocated, if needed), write the segment contents to the target       */
/*    memory identified in the DLOAD_MEMORY_SEGMENT attached to the request. */
/*                                                                           */
/*    After the segment contents have been written to target memory, the     */
/*    core loader should discard the DLOAD_MEMORY_REQUEST object, but retain */
/*    the DLOAD_MEMORY_SEGMENT object so that the target memory associated   */
/*    with the segment can be releases when the segment is unloaded.         */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLIF_write(struct DLOAD_MEMORY_REQUEST* req);
/*---------------------------------------------------------------------------*/
/* DLIF_read()                                                               */
/*                                                                           */
/*    Given a host accessible buffer, read content of indicated target       */
/*    memory address into the buffer.                                        */
/*---------------------------------------------------------------------------*/
BOOL     DLIF_read(void *ptr, size_t size, size_t nmemb, TARGET_ADDRESS src);

Executing Code in Target Memory

/*---------------------------------------------------------------------------*/
/* DLIF_execute()                                                            */
/*                                                                           */
/*    Start execution on the target architecture from given 'exec_addr'.     */
/*    If the dynamic loader is running on the target architecture, this can  */
/*    be effected as a simple function call.                                 */
/*                                                                           */
/*---------------------------------------------------------------------------*/
int32_t  DLIF_execute(TARGET_ADDRESS exec_addr);

Loading / Unloading Dependent Files

The dynamic loader core loader must coordinate loading and unloading dependent object files with the client side of the dynamic loader. This allows the client to keep its bookkeeping information up to date with what is currently loaded on the target architecture.

For instance, the client may need to interact with a file system or registry. The client may also need to update debug information in synch with the loading and unloading of shared objects (this is actually supported in the dynamic loader reference implementation, see the section about DLL Debug for more details.

/*---------------------------------------------------------------------------*/
/* DLIF_load_dependent()                                                     */
/*                                                                           */
/*    Ask client to find and open a dependent file identified by the         */
/*    'so_name' parameter, then, if necessary, initiate a DLOAD_load()       */
/*    call to actually load the shared object onto the target.  A            */
/*    successful load will return a file handle ID that the client can       */
/*    associate with the newly loaded file.                                  */
/*                                                                           */
/*---------------------------------------------------------------------------*/
int      DLIF_load_dependent(const char* so_name);
/*---------------------------------------------------------------------------*/
/* DLIF_unload_dependent()                                                   */
/*                                                                           */
/*    Ask client to unload a dependent file identified by the 'file_handle'  */
/*    parameter.  Initiate a call to DLOAD_unload() to actually free up      */
/*    the target memory that was occupied by the object file.                */
/*                                                                           */
/*---------------------------------------------------------------------------*/
void     DLIF_unload_dependent(uint32_t handle);

Error / Warning Registration and Reporting

The dynamic loader client / wrapper is responsible for communicating error and warning situations to the user of the dynamic loader. If a problem is encountered in the loader core (DLOAD) while loading an object file, then the problem is registered with the WRAPPER through one of the following functions:

  • Warning
/*---------------------------------------------------------------------------*/
/* DLIF_warning()                                                            */
/*                                                                           */
/*    Log a warning message with the client's error/warning handling         */
/*    infrastructure.                                                        */
/*                                                                           */
/*---------------------------------------------------------------------------*/
void     DLIF_warning(LOADER_WARNING_TYPE wtype, const char *fmt, ...);
  • Error
/*---------------------------------------------------------------------------*/
/* DLIF_error()                                                              */
/*                                                                           */
/*    Log an error message with the client's error/warning handling          */
/*    infrastructure.                                                        */
/*                                                                           */
/*---------------------------------------------------------------------------*/
void     DLIF_error(LOADER_ERROR_TYPE etype, const char *fmt, ...);

Data Segment Base Table (DSBT) Support

The C6000 Compiler Tools support the Linux Dynamic Linking Model and use the Data Segment Base Table (DSBT) mechanism as a means of managing multiple data page pointers during the execution of an application in a multi-threaded environment. To complete the support of the Linux Dynamic Linking Model, the dynamic loader takes on the responsibility of populating the DSBT for each dynamic object that uses a DSBT when that dynamic object is loaded. For more details on exactly how the DSBT mechanism works, please see the Data Segment Base Table (DSBT) section below.

The dynamic loader client /wrapper (WRAPPER) is responsible for assigning a DSBT index for each dynamic shared object (DSO) that is included in an application. The DSBT index assigned for each DSO must be unique. In a multi-threaded environment, the WRAPPER must interact with the host operating system kernel to determine which DSBT index ID has been assigned to which DSO. The WRAPPER will process requests for new DSBT indices as objects are submitted to be loaded onto the target. Once all of the DSBT index requests have been received, the WRAPPER will make the actual DSBT index assignment for each DSO that it received a request for. At that point, the WRAPPER can then fill in the content of each DSO's DSBT based on the static base address associated with each DSO in the DSBT.

These are the functions that are supplied by the dynamic loader client / wrapper in support of the DSBT mechanism:

  • Accept a Request for a DSBT Index
/*---------------------------------------------------------------------------*/
/* DLIF_register_dsbt_index_request()                                        */
/*                                                                           */
/*    Register a request for a DSBT index with the client. A module can      */
/*    make a specific DSBT index request or it can allow the client to       */
/*    assign a DSBT index on its behalf (requested_dsbt_index == -1). The    */
/*    client implementation of this function must check that a specific DSBT */
/*    index request does not conflict with a previous specific DSBT index    */
/*    request.                                                               */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLIF_register_dsbt_index_request(const char *requestor_name,
                                          int32_t     requestor_file_handle,
                                          int32_t     requested_dsbt_index);
/*---------------------------------------------------------------------------*/
/* DLIF_assign_dsbt_indices()                                                */
/*                                                                           */
/*    Bind each module that registered a request for a DSBT index to a       */
/*    specific slot in the DSBT. Specific requests for DSBT indices will be  */
/*    honored first. Any general requests that remain will be assigned to    */
/*    the first available slot in the DSBT.                                  */
/*                                                                           */
/*---------------------------------------------------------------------------*/
void     DLIF_assign_dsbt_indices(void);
/*---------------------------------------------------------------------------*/
/* DLIF_get_dsbt_index()                                                     */
/*                                                                           */
/*    Given a module that uses the DSBT model, return the identity of the    */
/*    DSBT slot that was assigned to it by the client. This function can     */
/*    only be called after the client has assigned DSBT indices to all       */
/*    loaded object modules that use the DSBT model. The implementation of   */
/*    this function will check that a proper DSBT index has been assigned to */
/*    the specified module and an invalid index (-1) if there is a problem.  */
/*                                                                           */
/*---------------------------------------------------------------------------*/
int32_t  DLIF_get_dsbt_index(int32_t file_handle);
/*---------------------------------------------------------------------------*/
/* DLIF_update_all_dsbts()                                                   */
/*                                                                           */
/*    Populate the client's model of the master DSBT with the static base    */
/*    for each assigned slot in the DSBT, then write a copy of the master    */
/*    DSBT to each module's DSBT location. The implementation of this        */
/*    function must check the size of each module's DSBT to make sure that   */
/*    it is large enough to hold a copy of the master DSBT. The function     */
/*    will return FALSE if there is a problem.                               */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLIF_update_all_dsbts(void);

Dynamic Loader Core

The dynamic loader core (DLOAD)provides all object file format-specific and target-specific functions and services to the dynamic loader. The following sections will detail the functions and/or data strucutures that DLOAD is responsible for. The sections have been grouped according to functionality where appropriate to provide mroe context about their use and purpose.

Object File Loading and Unloading

  • Loading an Object File
/*---------------------------------------------------------------------------*/
/* DLOAD_load()                                                              */
/*                                                                           */
/*    Dynamically load the specified file and return a file handle for the   */
/*    loaded file.  If the load fails, this function will return a value     */
/*    zero (0).                                                              */
/*                                                                           */
/*    The core loader must have read access to the file pointed by fp.       */
/*                                                                           */
/*---------------------------------------------------------------------------*/
int      DLOAD_load(LOADER_FILE_DESC* fp, int argc, char** argv);
  • Unloading an Object File
/*---------------------------------------------------------------------------*/
/* DLOAD_unload()                                                            */
/*                                                                           */
/*    Given a file handle ID, unload all object segments associated with     */
/*    the identified file and any of its dependents that are not still in    */
/*    use.                                                                   */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLOAD_unload(uint32_t pseudopid);

Symbol Management and Lookup

  • Loading Symbols from an Object File
/*---------------------------------------------------------------------------*/
/* DLOAD_load_symbols()                                                      */
/*                                                                           */
/*    Load externally visible symbols from the specified file so that they   */
/*    can be linked against when another object file is subsequntly loaded.  */
/*    External symbols will be made available for global symbol linkage.     */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLOAD_load_symbols(LOADER_FILE_DESC* fp);
DLOAD_load_symbols() is used by the wrapper to get information into the loader core about the global symbols that are defined in an object file that has already been loaded into target memory.
  • Symbol Queries
/*---------------------------------------------------------------------------*/
/* DLOAD_get_entry_names()                                                   */
/*                                                                           */
/*    Given a file handle, build a list of entry point names that are        */
/*    available in the specified file.  This can be used when querying       */
/*    the list of global functions available in a shared library.            */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLOAD_get_entry_names(uint32_t file_handle, int32_t* entry_pt_cnt,
                               char*** entry_pt_names);
/*---------------------------------------------------------------------------*/
/* DLOAD_query_symbol()                                                      */
/*                                                                           */
/*    Query the value of a symbol that is defined by an object file that     */
/*    has previously been loaded.  Boolean return value will be false if     */
/*    the symbol is not found.                                               */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLOAD_query_symbol(uint32_t file_handle, const char *sym_name,
                            TARGET_ADDRESS *sym_val);
/*---------------------------------------------------------------------------*/
/* DLOAD_get_entry_point()                                                   */
/*                                                                           */
/*    Given a file handle, return the entry point target address associated  */
/*    with that object file.  The entry point address value is written to    */
/*    *sym_val.  The return value of the function indicates whether the      */
/*    file with the specified handle was found or not.                       */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLOAD_get_entry_point(uint32_t file_handle, TARGET_ADDRESS *sym_val);

Data Segment Base Table (DSBT) Support

The C6000 Compiler Tools support the Linux Dynamic Linking Model and use the Data Segment Base Table (DSBT) mechanism as a means of managing multiple data page pointers during the execution of an application in a multi-threaded environment. To complete the support of the Linux Dynamic Linking Model, the dynamic loader takes on the responsibility of populating the DSBT for each dynamic object that uses a DSBT when that dynamic object is loaded. For more details on exactly how the DSBT mechanism works, please see the Data Segment Base Table (DSBT) section below.

These are the functions that are supplied by the dynamic loader core in support of the DSBT mechanism:

/*---------------------------------------------------------------------------*/
/* DLOAD_get_dsbt_size()                                                     */
/*                                                                           */
/*    Query the size of the DSBT associated with a specified file. The       */
/*    client will check the size of a module's DSBT before it writes a copy  */
/*    of the master DSBT to the module's DSBT. If the module's DSBT is not   */
/*    big enough, an error will be emitted and the load will fail.           */
/*                                                                           */
/*---------------------------------------------------------------------------*/
uint32_t  DLOAD_get_dsbt_size(int32_t file_handle);
/*---------------------------------------------------------------------------*/
/* DLOAD_get_dsbt_base()                                                     */
/*                                                                           */
/*    Find DSBT address for specified file. The client will query for this   */
/*    address after allocation and symbol relocation has been completed.     */
/*    The client will write a copy of the master DSBT to the returned DSBT   */
/*    address if the module's DSBT size is big enough.                       */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLOAD_get_dsbt_base(int32_t file_handle, TARGET_ADDRESS *dsbt_base);
/*---------------------------------------------------------------------------*/
/* DLOAD_get_static_base()                                                   */
/*                                                                           */
/*    Find static base for a specified file. The client will query for this  */
/*    address after allocation and symbol relocation has been completed.     */
/*    The client will use the returned static base value to fill the slot    */
/*    in the master DSBT that is associated with this module.                */
/*                                                                           */
/*---------------------------------------------------------------------------*/
BOOL     DLOAD_get_static_base(int32_t file_handle, TARGET_ADDRESS *static_base);

Miscellaneous Services

  • API and Loader Core Version Identification
Each open source release of dynamic loader API and loader core will have a unique version identification associated with it. A given version identification has three components:
  • Major Version Number - indicates a change to the API that is incompatible with previous versions of the API and loader core.
  • Minor Version Number - indicates added functionality to the API or in the loader core that is not incompatible with previous versions of the API and loader core.
  • Patch Version Number - indicates that one or more performance improvements, source code refactorings, or defect repairs have been installed in the loader core source code.
The version of the API and loader core that is included in a given implementation of the dynamic loader can be accessed via the DLOAD_version() function/macro.
/*---------------------------------------------------------------------------*/
/* DLOAD_version()                                                           */
/*                                                                           */
/*    Return a string constant representation for the version ID of the      */
/*    dynamic loader's core loader source code.                              */
/*                                                                           */
/*---------------------------------------------------------------------------*/
#include "version.h"
#define DLOAD_version() VERSION
  • Initialization
/*---------------------------------------------------------------------------*/
/* DLOAD_initialize()                                                        */
/*                                                                           */
/*    Construct and initialize data structures internal to the dynamic       */
/*    loader core.                                                           */
/*                                                                           */
/*---------------------------------------------------------------------------*/
void     DLOAD_initialize(void);
The dynamic loader core requires the CLIENT to call DLOAD_initialize() before trying to load the first object file. This function sets up data structures that are internal to the dynamic loader core, like a global symbol table and a list of loaded dynamic modules.

DLL Debug Support

Still to come!

Data Segment Base Table (DSBT) Support

Still to come!

DSO Initialization and Termination Support

Still to come!