Troubleshooting DSPLink configuration issues

From Texas Instruments Wiki
Jump to: navigation, search

END OF LIFE

DSP Link is still available for download, but no further releases or updates are planned. Please see IPC Software Options for details and alternatives.

Contents

Overview

DSPLink is a configurable software, provided with full source code and an associated build and configuration system. This page gives gives a trouble-shooting guide for resolving common configuration issues seen with DSPLink, by identifying the causes for various common scenarios, and providing possible solutions.

Trouble-shooting guide

IMPORTANT: Wherever the DSPLink configuration file $(DSPLINK)/config/all/CFG_<PLATFORM>.c is mentioned, if DSPLink is used within Codec Engine, the method of configuration is different, and modifying the default configuration file within DSPLink will not have any impact.

Problem: I changed the memory map split between GPP and DSP, now my application does not work anymore

Symptom: Changing memory map caused the application to stop working.
Possible cause 1: The complete set of steps for modifying the memory map must be done. Please look at Changing DSPLink memory map wiki page for a description of all the steps that must be performed to ensure that the memory map change is complete.
Possible cause 2: If you are using DSPLink within Codec Engine, the steps to be taken for modifying memory map are different. For this information, refer to this page
Possible cause 3: Have you done a rebuild post the changes?
Possible cause 4: Is your application linking in the libraries from the correct path?

Problem: Modifying DSPLink configuration file is not making any difference in memory map

Symptom: Using instructions to modify DSPLink memory map is still causing Engine_open failure in Codec Engine, and/or printing:

Error: DSP-side memory map does not match configuration.
Compare DSP-side TCF/MAP file with /dsplink/config/all/CFG_<PLATFORM>.c

Even though the specified /dsplink/config/all/CFG_<PLATFORM>.c configuration file has been modified to match DSP-side TCF file, still the problem is being seen.
Possible cause: Engine_open failure seems to indicate that DSPLink is being used inside Codec Engine. In such cases, modifying the default configuration file within DSPLink as mentioned above will not make any difference. This file gets overwritten by Codec Engine, which provides a dynamic configuration to DSPLink. The actual changes to ensure correct memory map need to be made as mentioned at this page

Problem: PROC_setup failure with DSP_ECONFIG error

Symptom: PROC_setup failed with DSP_ECONFIG error.
Possible cause: This indicates that some configuration information passed within the DSPLink configuration file $(DSPLINK)/config/all/CFG_<PLATFORM>.c is incorrect. In Linux as well as PrOS a print is shown which tells the exact mismatch. On Linux it is a kernel print and you can run dmesg to see the failure, accordingly make the changes and rebuild. Since the most common changes are memory entry settings and pool size mismatch, verify those first. If you have made more advanced changes the prints tell the exact failure.

Problem: Configuration is correct but PROC_setup failed

Symptom: All my configurations are correct but PROC_setup still failed.
Possible cause: The memory being used for memory entries is not configured. In some Operating Systems, for example QNX, a configuration file might need to be edited to determine which the memory split between GPP and DSP.

Problem: PROC_attach failed with DSP_EALREADYCONNECTED error

Symptom: PROC_attach failed with DSP_EALREADYCONNECTED error
Possible cause: Another thread of the same process has already attached to the processor. Check the application code.

Problem: PROC_attach failed with DSP_EFAIL error

Symptom: PROC_attach failed with DSP_EFAIL error
Possible cause: DSP_EFAIL is a very common error. The application writer should enable SET_FAILURE_REASON to see failure. If you see the following symptoms:

  • Assertion in PROC_attach
Assertion failed ((isrObj!= NULL) && (ISR_InstalledIsrs [isrObj->dspId][isrObj->irq] == isrObj)). File : dsplink/gpp/src/../../gpp/src/osal/Linux/2.6.18/isr.c Line : 504
  • On doing cat /proc/interrupts" to look at the assigned interrupts, you see
26:          0        INTC  DspBridge  mailbox <br>
28:          0        INTC  DspBridge  iommu fault <br>

This can happen when CONFIG_MPU_BRIDGE is defined. This causes DspBridge interrupts to be assigned. In "make xconfig" for Linux kernel, do not select "DSPBridge support" and re-build the kernel. Rebuild DSPLink samples with the re-built kernel and try.


Problem: PROC_load failed with DSP_ERANGE error

Symptom: PROC_load failed with DSP_ERANGE error.
Possible cause: This error indicates that the DSP memory map does not match the memory map specified within DSPLink configuration file $(DSPLINK)/config/all/CFG_<PLATFORM>.c. DSPLink checks the memory map configuration before loading the DSP executable. Check the DSP-side TCF file or MAP file to see if any DSP-side code/data sections are falling outside of the range specified within DSPLink configuration.

Problem: DSP image size > default 1MB

Symptom: The size of my DSP executable is more than the 1MB default supported by DSPLink. Due to this, I am getting a DSP_ERANGE error in PROC_load. How do I fix this?
Possible cause: If the size of the DSP executable is more than the default 1MB in DSPLink configuration, the DSPLink configuration must be modified to have a different GPP-DSP memory split.

Problem: PROC_load failed with DSP_EFILE error

Symptom: PROC_load failed with DSP_EFILE error
Possible cause: The DSP executable does not exist at the path provided to PROC_load. Recheck the path.

Problem: PROC_load failed with DSP_EMEMORY error

Symptom: PROC_load failed with DSP_EMEMORY error
Possible cause: The DSP executable size is too big and heap memory is not available. Recheck the system heap size configuration.

Problem: PROC_load failed with DSP_ESIZE error

Symptom: PROC_load failed with DSP_ESIZE error
Possible cause: Configured DSP-side .args section size is insufficient. Set required size for .args section in DSP. For e.g. bios.MEM.ARGSSIZE = 0x0032;

Problem: PROC_start failed with DSP_EVERSION error

Symptom: PROC_start failed with DSP_EVERSION error
Possible cause: The version of DSPLink used for building GPP-side and DSP-side is different. This is possible if GPP and DSP-sides are being built on different machines, or pre-built libraries are being shared directly. The version mismatch error also prints out information of the GPP and DSP-side versions found.

Problem: PROC_start failed with DSP_ERANGE error

Symptom: PROC_start failed with DSP_ERANGE error
Possible cause 1: The address specified for RESETVECTOR in LINKCFG_dspObjects in $(DSPLINK)/config/all/CFG_<PLATFORM>.c configuration file does not match the actual start address of RESETCTRL memory entry in the same file. Make sure they match.
Possible cause 2: There is a mismatch in DSP and GPP-side memory configuration. Check the DSP-side TCF/MAP file and GPP-side configuration file to make sure the memory regions match.
Possible cause 3: For DSPLink v1.40.05_p4 / v1.50 onwards, check if the .data:DSPLINK_shmBaseAddress section has been placed into an address that is outside of the range configured within the GPP-side configuration file. This can be easily verified through the DSP-side MAP file. If this issue is seen, then check your application's linker command file for the SECTIONS directive that is placing this section into the incorrect address.

Problem: PROC_start failed with configuration mismatch/failure

Symptom: PROC_start timed out and/or printed configuration mismatch failure information of the type:

DSP-side configuration mismatch/failure
0              -> success
Positive value -> DSP-side failure code.
(Uint32) -1    -> DSP-side component was not initialized.
    DRV configuration status    [0xffffffff]
    IPS configuration status    [0xffffffff]
    POOL configuration status   [0xffffffff]
    MPCS configuration status   [0xffffffff]
    MPLIST configuration status [0xffffffff]
    MQT configuration status    [0xffffffff]
    DATA configuration status   [0xffffffff]
    RingIO configuration status [0xffffffff]

Explanation: This failure indicates that DSPLink failed while attempting to start the DSP running. DSPLink internally performs handshake within the drivers on the GPP and DSP. It verifies that each component on the DSP has been correctly initialized, as required by the GPP.
The above print indicates the status of initialization of each DSPLink component on the DSP-side.
For example, the above prints indicate that none of the DSPLink components were initialized. 0xffffffff means that the value is -1. This indicates that the specific DSP-side component with this value was not initialized. When this appears for all components, it indicates that none of the components were initialized. This implies that there was some generic issue in running the DSP.
Possible cause 1: DSP has not been loaded correctly, and hence is not executing correctly to reach the point where the driver completes the handshake. In case of a custom board, check if the board setup/configuration has been done correctly.
Possible cause 2: There is a configuration mismatch in memory regions between the GPP and DSP, resulting in the GPP looking for handshake at one location, and the DSP looking at another location. Check the DSP-side and GPP-side configuration files.
Possible cause 3: User has not reserved the required amount of memory from Linux kernel space. On Linux, check the bootcmd for mem parameter as specified at Changing DSPLink memory map wiki page
Possible cause 4: There is some other issue on the DSP-side resulting in the execution not reaching the driver handshake. This could also be an application integration issue, and can only be identified by debugging the DSP-side by connecting Code Composer Studio (CCS).

Problem: PROC_start failed with configuration mismatch/failure showing modules with 0xffffffff but print for one of the modules is missing

Symptom: PROC_start timed out and/or printed configuration mismatch failure information, for example:

DSP-side configuration mismatch/failure
0              -> success
Positive value -> DSP-side failure code.
(Uint32) -1    -> DSP-side component was not initialized.
    DRV configuration status    [0xffffffff]
    IPS configuration status    [0xffffffff]
    POOL configuration status   [0xffffffff]
    MPCS configuration status   [0xffffffff]
    MPLIST configuration status [0xffffffff]
    MQT configuration status    [0xffffffff]
    RINGIO configuration status [0xffffffff]

Explanation: On GPP-side, DSPLink checks for all modules that are enabled in the build. Prints will appear for configuration status of all such modules. However, if there is a mismatch in selection of modules, such that GPP-side does not have the module enabled, while DSP-side has it enabled, then this can cause such a failure. This failure can happen because the shared memory configuration between DSP and GPP would not match, and hence the module initializations would fail. Another reason for the failure may be if any of POOL, CHNL (DATA) or MSGQ are enabled on DSP but disabled on GPP. In such case, the DSP-side would hang, polling for GPP-side POOL/DATA/MSGQ configuration, and hence cause this failure.
Possible cause: In this particular failure, in the above prints seen from DSPLink, DATA module print is missing. This indicates that GPP-side does not have DATA module enabled in the build. Actual expectation for modules is: DRV, IPS, POOL, MPCS, MPLIST, MQT, DATA, RINGIO. In the above, DATA is missing, indicating that DSP-side was built with CHNL enabled, and GPP-side was built with CHNL disabled.


Problem: PROC_start failed with configuration mismatch/failure showing some modules with 0 and others with 0xffffffff

Symptom: PROC_start timed out and/or printed configuration mismatch failure information, for example:

DSP-side configuration mismatch/failure
0              -> success
Positive value -> DSP-side failure code.
(Uint32) -1    -> DSP-side component was not initialized.
    DRV configuration status    [0x0]
    IPS configuration status    [0x0]
    POOL configuration status   [0x0]
    MPCS configuration status   [0x0]
    MPLIST configuration status [0x0]
    MQT configuration status    [0x0]
    DATA configuration status   [0xffffffff]
    RingIO configuration status [0x0]

Explanation: The above print indicates that all modules except DATA were successfully initialized with status 0 (DSP-side SYS_OK code). However, DATA module was not initialized at all (value 0xffffffff).
Possible cause: There is a mismatch in components selected for GPP and DSP-side build. If GPP and DSP side code is being built by different teams or at different locations, this can happen. From DSPLink v1.4x.xx onwards, this can be easily seen through component initialization information printed out on failure. For example in the above example, it is seen that DATA component (i.e. CHNL component) is the only one showing 0xFFFFFFFF status. This means that CHNL component was not included in DSP-side build, but GPP-side had included it. Check that the components selected for both GPP and DSP-side builds are identical.

Problem: PROC_start failed with configuration mismatch/failure showing some modules with 0, some with 1, 2 etc. and others with 0xffffffff

Symptom: PROC_start timed out and/or printed configuration mismatch failure information, for example:

DSP-side configuration mismatch/failure
0              -> success
Positive value -> DSP-side failure code.
(Uint32) -1    -> DSP-side component was not initialized.
    DRV configuration status    [0x0]
    IPS configuration status    [0x1]
    POOL configuration status   [0x0]
    MPCS configuration status   [0xffffffff]
    MPLIST configuration status [0xffffffff]
    MQT configuration status    [0xffffffff]

Explanation: The above print indicates that modules MPCS, MPLIST and MQT were not initialized (0xffffffff is -1 indicating that the specific DSP-side component was not initialized.). DRV and POOL show status of 0, i.e. SYS_OK.
However, more importantly, IPS module shows value of 0x1. This indicates error value SYS_EALLOC. This means that there was a memory allocation failure while attempting to initialize IPS component.
Possible cause: DSP-side heap size is not sufficient, resulting in memory allocation failure in IPS component.

Problem: Scaling out some modules resulted in PROC_start hang or failure with configuration mismatch/failure showing some modules with 0 and others with 0xffffffff

Symptom: An attempt to scale out some modules in DSPLink resulted in PROC_start hang or timeout and/or configuration mismatch failure information prints, for example:

DSP-side configuration mismatch/failure
0              -> success
Positive value -> DSP-side failure code.
(Uint32) -1    -> DSP-side component was not initialized.
    DRV configuration status    [0x0]
    IPS configuration status    [0x0]
    POOL configuration status   [0x0]
    MPCS configuration status   [0x0]
    MPLIST configuration status [0x0]
    MQT configuration status    [0x0]
    DATA configuration status   [0xffffffff]
    RingIO configuration status [0xffffffff]

In case of hang, the same could be seen by debugging and printing the contents of DSPLINK_shmBaseAddress on DSP-side.
Explanation: The above indicates that all modules except DATA and RingIO were successfully initialized with status 0 (DSP-side SYS_OK code). However, DATA and RingIO modules were not initialized at all (value 0xffffffff).
Possible cause: There is a mismatch in components selected for GPP and DSP-side build. If GPP and DSP side code is being built by different teams or at different locations, this can happen. From DSPLink v1.4x.xx onwards, this can be easily seen through component initialization information printed out on failure. For example in the above example, it is seen that DATA component (i.e. CHNL component) and RingIO component are the ones showing 0xFFFFFFFF status. This means that CHNL and RingIO components were not included in DSP-side build, but GPP-side had included it. Check that the components selected for both GPP and DSP-side builds are identical.
Possible cause: If the correct components are selected on HLOS-side when running dsplinkcfg script, make sure that HLOS-side DSPLink library and applications are rebuilt after the change. Do a clean build.
Possible cause: If a problem is still seen, run make with VERBOSE=1 to see if the RINGIO_COMPONENT and CHNL_COMPONENT defines are appearing in the HLOS make (they should not appear).
Possible cause: If a problem is still seen, make sure that the newly generated dsplinkk.ko is copied to the file system of the target and re-inserted (i.e. existing module removed by doing rmmod and new module reinserted by doing insmod) before running the applications.
Possible cause: If a problem is still seen, try running one of the DSPLink applications instead of your own application. For example, mpcsxfer application can be run with scalability configuration --comps=pons while running the dsplinkcfg script, since it does not use MSGQ, RingIO or DATA, and only uses PROC (p), POOL (o), NOTIFY (n) and MPCS (s). If you don't see any problem with this, it means that basic scalability is working fine in DSPLink and the specific problem is with your application build, which needs to be debugged further.
NOTE:When scalability is reduced such that only PROC, POOL, NOTIFY, MPCS are used, HLOS-side PROC_start will not wait till DSP-side calls DSPLINK_init. This is a different behavior from what is seen when MSGQ, RingIO etc. are enabled. This is because internally, POOL and data drivers (CHNL) are already initialized by the time the application calls DSPLINK_init. The POOLs and DATA drivers internally get plugged into BIOS. Hence, in the POOL initialization itself, DSPLink_init is processed internally, which completes the handshake and initialization from POOL perspective. NOTIFY is alwasy included and gets initialized automatically. Since MSGQ is not included (which completes initialization later), the handshake completes early. If MSGQ had also been included in the build, the HLOS handshake would block till MSGQ was also completely initialized.

Problem: DSPLink samples run fine, but my application does not

Symptom: DSPLink sample applications run correctly, but when I run my application, PROC_start hangs or fails. How do I fix this?
Possible cause: This must be a configuration issue. Ensure that the DSPLink configuration for both GPP and DSP-side matches.

Problem: Getting error when inserting DSPLink module dsplinkk.ko on Linux

Symptom: I am getting errors when inserting DSPLink module on Linux:

$ insmod dsplinkk.ko
dsplinkk: no version for "struct_module" found: kernel tainted.
dsplinkk: version magic '2.6.10_mvl401 preempt ARMv5 gcc-3.4' should be '2.6.23-davinci1-g1220f74d-dirty mod_unload ARMv5 '
insmod: error inserting 'dsplinkk.ko': -1 Invalid module format

Explanation: The error in inserting the module indicates that the DSPLink kernel module was built with a different kernel version than the version running on the target. In the above example, DSPLink was built with 2.6.10 kernel and the target is running 2.6.23 kernel.
Possible cause 1: DSPLink was built using different kernel version than the one used on the target. This can happen if DSPLink is being built on a different development machine than the one where the target version was present.
Possible cause 2: Some Codec Engine and DVSDK releases include pre-built binaries of DSPLink for a specific kernel version. In case you are using a different kernel version, you will need to rebuild DSPLink. Instructions on building DSPLink are available within DSPLink release documents and at this page.

Problem: Getting error when running sample applications on Linux

Symptom: I am getting an error when running applications on Linux:

$ ./messagegpp message.out 1000
# ========== Sample Application : MESSAGE ==========
Entered MESSAGE_Create ()
dsplink driver open: /dev/dsplink: No such device or address
Leaving MESSAGE_Create ()
Entered MESSAGE_Delete ()
PROC_stop () failed. Status = [0x80008002]
Leaving MESSAGE_Delete ()
================================================

Explanation: This error indicates that there was a failure in opening a handle to the DSPLink device driver.
Possible cause 1: mknod was not called before running the application.

$ mknod /dev/dsplink c 230 0

Possible cause 2: mknod was called but still this error occurs. In that case, it is possible that the DSPLink kernel modules may have been built on a kernel, after which the kernel was re-built, and the new uImage was used on the target board. In that case, the older DSPLink kernel module will not work with this new image. You will need to rebuild dsplinkk.ko with the newly built kernel.

Problem: Scripts are not executed correctly, parts of strings go mysteriously missing

Symptom: Using instructions to configure/build dsplink, some scripts do not execute properly and parts of strings go mysteriously missing in the printout. E.g.

$ source etc/host/scripts/Linux/dsplinkenv.bash
: command not found
: command not found
: command not found
: command not found
===========================================================
The environment for DSP/BIOS LINK development has been set:
DSPLINK = /home/a0792201/dsplink_1_60/dsplink
/etc/host/scripts/Linux01/dsplink_1_60/dsplink
===========================================================
perl config/bin/dsplinkcfg.pl --platform=OMAP3530 --nodsp=1 --dspcfg_0=OMAP3530SHMEM --dspos_0=DSPBIOS5XX --gppos=OMAPLSP --comps=ponslrmc
...

Configuration done successfully!!
Generating CURRENTCFG.MK file...
mkdir: cannot create directory `/config': Permission denied
/config/BUILD/CURRENTCFG.MKe/a0792201/dsplink_1_60/dsplink


Even though the script looks perfect in a text editor, it doesn't execute properly.


Possible cause: In dsplink 1.60, early releases had script files saved in DOS mode (hence containing invisible characters that causes the scripts e.g. dsplinkenv.bash to fail). Run dos2unix on the file to change it to UNIX mode. The scripts would then run correctly.



Problem: I see a crash as soon as I run a DSPLink sample.

Symptom: I see a hang/crash as soon as I run a DSPLink sample. The crash dump shows only Linux functions and no DSPLink functions E.g.

[42949402.010000]  r7:0000000c r6:c383a1b4 r5:c38f0f00 r4:00010000<br>
[42949402.020000] [<c002f794>] (do_page_fault+0x0/0x238) from [<c0027224>] (do_DataAbort+0x38/0x98)<br>
[42949402.030000] [<c00271ec>] (do_DataAbort+0x0/0x98) from [<c0027c80>] (ret_from_exception+0x0/0x10)<br>
[42949402.030000] Exception stack(0xc3a17fb0 to 0xc3a17ff8)



Possible cause: mem argument in bootargs is wrong. Please check memory map to set mem value in Linux bootargs

Possible cause: PROC_start has hung because the DSPLink init code on DSP is missing. Please check if you have a call to DSPLink_init in the DSP code

Possible cause: There is an overlap in the configuration of timers on Linux and DSP/BIOS. This problem has commonly been seen on OMAPL137 and DA830 where the timer 0 is used by DSP/BIOS and Linux. In that case: Update the sample to change the BIOS timer to use a different timer. See a sample configuration to change to timer 1.

Clock = xdc.useModule(‘ti.sysbios.knl.Clock’);
Clock.timerId = 1;