Early Boot and Late Attach

{{#invoke:Message box|ombox}}

Introduction

Heterogeneous multicore devices, like DRA7XX, often configure their individual cores into master/slave relationships. The master (e.g. DRA7XX's A15) often runs an HLOS like Linux, which manages slaves (e.g. DRA7XX's DSPs and IPUs) that run an RTOS like SYS/BIOS. The bootloader (e.g. u-boot) for the typical master/slave system loads/boots the HLOS, and once the HLOS has enabled enough infrastructure (e.g. file systems, device driver backplanes, etc), the HLOS loads/boots the slave devices.

In some use cases, however, it's important that features enabled by the [traditionally] slave devices become available ASAP, without waiting for the HLOS to boot. In this case, the system integrator might prefer to instead configure the cores into a peer-to-peer relationship, rather than the traditional master/slave. In this peer-to-peer configuration, the bootloader instead loads the RTOS-enabled cores and then the HLOS. This way, both cores independently boot, and the RTOS-running cores can enable functionality very early. This configuration is often called "Early Boot".

In this peer-to-peer configuration, once both HLOS and RTOS cores have independently booted and are ready to communicate with each other, they can establish a communication path. This post-boot handshaking is often called "Late Attach".

This feature has been enabled for DRA7XX, running Linux on the A15 and BIOS on the IPUs. This article describes the boot sequence with Early Boot, details to enable it, and potential issues you may encounter.

Boot Sequence

Early Boot starts with the bootloader, in this case, u-boot. It has been extended to support parsing a resource table, allocating the necessary memory and configuring the mmus, loading the IPUs code/data, and booting the IPUs (running BIOS) before it loads and boots the Linux kernel.

In order to obtain memory for loading the IPUs, u-boot leveraged the same page order allocation scheme as used by the CMA allocator from the Linux kernel. This has two benefits:

1. Reduced modification to the Linux kernel. The Linux rproc drivers process the firmware images and "allocate" memory like they traditionally have - the higher layers don't know any difference. Note that the allocator did require a modification to not zero the memory during allocation (since that would erase the code u-boot loaded and the IPUs are actively executing!).
2. Code reuse. Better to leverage the well-tested page order allocation policy (as used by the CMA allocator) than create something from scratch. The GPL license helped facilitate this code reuse.

A potential maintenance issue with this approach is that u-boot must stay in sync with the allocation policy of the Linux CMA allocator. As changes are made in CMA, those must be integrated back into u-boot.

Enabling Early Boot

u-boot

By default, the configuration in the config file "include/configs/dra7xx_evm.h" is done for IPU2. You could enable the corresponding flags (both at kernel and u-boot) to use the functionality for IPU2.

To enable the feature for IPU1, kindly replace the config "CONFIG_LATE_ATTACH_BOOTIPU2" --> "CONFIG_LATE_ATTACH_BOOTIPU1" in the file and change the undef to define for the late attach config.

Also, make sure the name of the IPU binary that you want to load is also changed to the correct name. For the name of the IPU binary, change the config CONFIG_SPL_FAT_LOAD_IPU_PAYLOAD_NAME. Ideally for IPU1, this name should be dra7-ipu1-fw.xem4.

Build the u-boot with the configuration dra7xx_evm:

       <host> make dra7xx_evm (from the u-boot folder)

After building with these changes, copy the MLO and u-boot.img in the boot partition of the SD card.

Enabling Late Attach

Linux kernel

Run the following commands on the linux directory to enable late attach to IPU2.

   <host> export ARCH=arm
   <host> export CROSS_COMPILE=arm-linux-gnueabihf-
   <host> make omap2plus_defconfig
   <host> make menuconfig

In the menuconfig GUI, enable the following config

   Device Drivers  --->
     Remoteproc Drivers  --->
       <M> OMAP remoteproc support
       [*]   OMAP remoteproc support for IPU
       [*]     Attach to already-running IPU2 (no reset/load/start)
       [*]   OMAP remoteproc support for DSP
       [*]   OMAP remoteproc support for IPU1 (DRA7xx only)
       [ ]     Attach to already-running IPU1 (no reset/load/start)
       [*]   OMAP remoteproc support for DSP2 (DRA7xx only)

Note that "IPU" with no appended digit refers to IPU2.

Enabling Late attach on Android (6AJ1.3)

IMPORTANT NOTE: This feature works only with single stage boot loader configuration!!!

Enabling this feature involves building 4 different components:

• u-boot

• Android Kernel

• Sysbios IPU1 binary

• Android userspace binary to test the feature

Tools

Kernel/u-boot Same as mentioned in the Android release notes.

SysBios

BIOS : bios_6_35_03_47

XDC : xdctools_3_25_04_88

CG Tools : TICGTARM_5.0.1

IPC  : 3.21.00.07

Building kernel

Choose the following 2 configs for IPU1 under

Drivers-->Remoteproc drivers-->
	[*] OMAP remoteproc support for IPU1 (DRA7xx only) - (CONFIG_OMAP_REMOTEPROC_IPU1)
	[*] Attach to already-running IPU1 (DRA7xx only) - (CONFIG_OMAP_REMOTEPROC_LATE_ATTACH_IPU1)

Save the kernel config and build the kernel as mentioned in the release notes. This generates the zImage and dtb to be flashed.

Building u-boot

Cherry-pick the following patch and applying it on top of the Release Notes.

http://review.omapzoom.org/34844

    git fetch http://review.omapzoom.org/repo/u-boot refs/changes/44/34844/1 && git cherry-pick FETCH_HEAD

Build the u-boot as mentioned in the Release Notes.

Building IPU1 binary

To build the sample MessageQ_single.xem4

cd to <mydroid>/hardware/ti/ipc.

Make sure that the ipc version supports late attache (e.g. 3.21.00.07)

Fixup product.mak

Following variables need to be set before building the ipu1 binary.

<syntaxhighlight lang='make'>

1. DEPOT is often a directory in your SDK containing your TI tools eg: XDC,BIOS. folder

DEPOT = _your_depot_folder_ PLATFORM = DRA7XX TOOLCHAIN_INSTALL_DIR = $(DEPOT)/_your_arm_code_gen_install_ XDC_INSTALL_DIR = $(DEPOT)/_your_xdctools_install_ BIOS_INSTALL_DIR = $(DEPOT)/_your_BIOStools_install_ ti.targets.arm.elf.M4 = $(DEPOT)/<folder name of TI ARM CGT Tools> </syntaxhighlight>

Run the following build command

make -f ipc-bios.mak all

A binary would be generated here:

<mydroid>/hardware/ti/ipc/packages/ti/ipc/tests/bin/ti_platforms_evmDRA7XX_ipu1/messageq_single.xem4

Since the size of the binary is quiet large ( ~5M) we need to strip the symbols on the xem4. To do this, execute:

<CGT ARM tools folder>/armstrip -p <mydroid>/hardware/ti/ipc/packages/ti/ipc/tests/bin/ti_platforms_evmDRA7XX_ipu1/messageq_single.xem4 -o <patch of destination folder>/dra7-ipu1-fw.xem4.

Building Android userspace binary

Builing remoteproc (IPU1) userspace sample to test late attach: Setup your android build (cd to your "mydroid" folder) and issue the following commands:

  lunch full_jacinto6evm-userdebug <-- If not already done as per release notes

  make messageQApp messageQBench messageQMulti nameServerApp ping_rpmsg lad_dra7xx libmmrpc

  Note:: libmmrpc and lad_dra7xx might be already built since they are used for multimedia and radio use cases.

This should generate the sample binaries under the <mydroid>/out/target/product/jacinto6evm/system/bin and <mydroid>/out/target/product/jacinto6evm/system/lib

Binaries to be flashed into the eMMC

Flash the u-boot and SPL into the QSPI and reboot.

sudo ./fastboot oem spi
sudo ./fastboot flash xloader <path to built MLO>
sudo ./fastboot flash bootloader <path to built u-boot.img>
sudo ./fastboot reboot

Enter fastboot on the device and flash IPU1 binary, zImage and dtb into eMMC and reboot.

sudo ./fastboot flash ipu <mydroid>/hardware/ti/ipc/dra7-ipu1-fw.xem4
sudo ./fastboot flash environment <path to built dra7-evm.dtb>
sudo ./fastboot flash zImage <path to built zImage>
sudo ./fastboot reboot

Binaries to be pushed to file system

adb push <mydroid>/out/target/product/jacinto6evm/system/bin/messageQApp /system/bin
adb push <patch of ipu1 binary>/dra7-ipu1-fw.xem4 /system/vendor/firmware

NOTE

In case lad_dra7xx and libmmrpc are needed, also push those to the relevant folders.

Once this is done. Change the switch setting to QSPI production as mentioned here.

Reboot to device. You should be able to see the single stage directly load the kernel.

Validation

Once in the android command prompt, launch lad_dra7xx if it is not already launched.

To execute the rest sample, execute:

/system/bin/messageQApp 10 2

This should do 10 iterations of the messageQApp (10) with IPU1 (2).

Enabling Early Boot and Late attach on Android (6AL.1.1)

IMPORTANT NOTE: 6AL.1.1 kernel needs some post-release fixes to enable kernel loading of IPU1 - See "IPU1 Race Condition" below

Enabling this feature involves building 4 different components:

• u-boot
• Android Kernel DTB file
• Sysbios IPU1 binary to test the feature
• Android userspace binary to test the feature

Sources

Kernel/u-boot

• Same as mentioned in the Android release notes.

SysBios Tools Versions

The following versions were used for validation:

• IPC 3.36.01.11
• SYS/BIOS 6.41.03.51
• XDCtools 3.30.06.67.core
• CG Tools TICGTARM_5.1.5

Building kernel DTB file

Cherry-pick the following patch and apply it on top of the commit from the Release Notes. This patch adds a new dts file that enables late attach for IPU1 for dra7xx boards with a 10" lcd.

http://review.omapzoom.org/#/c/36616/

    git fetch http://review.omapzoom.org/kernel/omap refs/changes/16/36616/1 && git cherry-pick FETCH_HEAD

Build the dtbs as mentioned in the release notes. This generates the dra7-evm-lcd10-late-attach-ipu1.dtb to be flashed.

Building u-boot

Enable late attach in the dra7xx_evm_android.h config file by defining the CONFIG_LATE_ATTACH flag:

diff --git a/include/configs/dra7xx_evm_android.h b/include/configs/dra7xx_evm_android.h
index 3749671..b2c43e9 100644
--- a/include/configs/dra7xx_evm_android.h
+++ b/include/configs/dra7xx_evm_android.h
@@ -318,7 +318,7 @@
 /* JAMR3 board pin mux */
 #define CONFIG_DRA7XX_JAMR3
 
-#undef CONFIG_LATE_ATTACH
+#define CONFIG_LATE_ATTACH
 
 /* Enable this flag if you want to boot IPU2
  * from SPL */

Build the u-boot as mentioned in the Release Notes.

Building IPU1 binary

To build the sample messageq_single.xem4:

cd <mydroid>/hardware/ti/ipc

Make sure that the ipc version supports late attach (e.g. 3.36.01.11)

Fixup product.mak

Following variables need to be set before building the ipu1 binary.

<syntaxhighlight lang='make'>

1. DEPOT is often a directory in your SDK containing your TI tools eg: XDC,BIOS

DEPOT = _your_depot_folder_ PLATFORM = DRA7XX TOOLCHAIN_INSTALL_DIR = $(DEPOT)/_your_arm_code_gen_install_ XDC_INSTALL_DIR = $(DEPOT)/_your_xdctools_install_ BIOS_INSTALL_DIR = $(DEPOT)/_your_BIOStools_install_ ti.targets.arm.elf.M4 = $(DEPOT)/<folder name of TI ARM CGT Tools> </syntaxhighlight>

Run the following build command

make -f ipc-bios.mak all

A binary will be generated here:

<mydroid>/hardware/ti/ipc/packages/ti/ipc/tests/bin/ti_platforms_evmDRA7XX_ipu1/messageq_single.xem4

Since the size of the binary is quite large ( ~5M) we need to strip the symbols on the xem4. To do this, execute:

<CGT ARM tools folder>/bin/armstrip -p <mydroid>/hardware/ti/ipc/packages/ti/ipc/tests/bin/ti_platforms_evmDRA7XX_ipu1/messageq_single.xem4 -o <path of destination folder>/dra7-ipu1-fw.xem4

Building Android userspace binary

To build the userspace sample to test late attach:

Setup your android build (cd to your "mydroid" folder) and issue the following commands:

  lunch full_jacinto6evm-userdebug <-- If not already done as per release notes

  make libtiipc libtiipcutils libtiipcutils_lad libtitransportrpmsg lad_dra7xx messageQApp

  Note:: lad_dra7xx and the libraries might already be built since they are used for other use cases.

This should generate the sample binaries under the <mydroid>/out/target/product/jacinto6evm/system/bin and <mydroid>/out/target/product/jacinto6evm/system/lib

Binaries to be flashed into the eMMC

Flash the u-boot and MLO into the QSPI and reboot.

sudo ./fastboot oem spi
sudo ./fastboot flash xloader <path to built MLO>
sudo ./fastboot flash bootloader <path to built u-boot.img>
sudo ./fastboot reboot

Enter fastboot on the device and flash IPU1 binary and dtb into eMMC and reboot.

sudo ./fastboot flash ipu1 <mydroid>/hardware/ti/ipc/dra7-ipu1-fw.xem4
sudo ./fastboot flash environment <path to built dra7-evm-lcd10-late-attach-ipu1.dtb>
sudo ./fastboot reboot

Binaries to be pushed to file system

adb push <mydroid>/out/target/product/jacinto6evm/system/bin/messageQApp /system/bin
adb push <patch of ipu1 binary>/dra7-ipu1-fw.xem4 /system/vendor/firmware

NOTE

In case lad_dra7xx and other libraries are needed, also push those to the relevant folders.

Once this is done, you can change the switch setting to QSPI production as mentioned here or stay in dual stage boot mode.

Reboot the device. You should be able to see the single stage directly load the kernel if in single stage mode.

Validation

Once in the android command prompt, launch lad_dra7xx if it is not already launched.

To execute the test sample, execute:

/system/bin/messageQApp 10 2

This should do 10 iterations of the messageQApp (10) with IPU1 (2).

Error Recovery

The 6AL.1.1 release supports error recovery with late attach. You can read more about error recovery here.

IPU1 Race Condition

The kernel version that is part of the 6AL.1.1 release has a race condition that may result in IPU1 failing to load when loaded by the kernel. If you face this issue, please use the post-release fixes from the Application Notes for 6AL.1.1 for "IPU1 Loading fails some time":

   http://omapedia.org/wiki/6AL.1.1_Application_Notes#Post_release_fixes

Then build the kernel as mentioned in the release notes and flash the upated zImage to the target.

sudo ./fastboot flash zImage <path to built zImage>

Reboot the device.

Enabling Early Boot and Late attach on Android (starting in 6AM.1.2)

Enabling this feature involves building 4 different components:

• u-boot
• Android Kernel DTB file
• Sysbios IPU1 binary to test the feature
• Android userspace binary to test the feature

Sources

Kernel/u-boot

• Same as mentioned in the Android release notes.

SysBios Tools Versions

The following versions were used for validation but any compatible ones may be used:

• IPC 3.43.03.05
• SYS/BIOS 6.45.01.29
• XDCtools 3.32.00.06.core
• CG Tools TICGTARM_5.2.7

Building kernel DTB file

The kernel dtb file dra7-evm-lcd-lg-late-attach.dts enables late attach for IPU2 and also has commented sections for enabling late attach for the other cores. Modify this file to enable late attach for the desired core(s) or use it as an example for modifying your dts file.

We'll use IPU1 as an example for now. Modify the file to comment out the IPU2 section and enable IPU1 late attach:

--- a/arch/arm/boot/dts/dra7-evm-lcd-lg-late-attach.dts
+++ b/arch/arm/boot/dts/dra7-evm-lcd-lg-late-attach.dts
@@ -8,6 +8,7 @@

 #include "dra7-evm-lcd-lg.dts"

+/*
 &ipu2 {
        ti,late-attach;
        ti,no-idle-on-init;
@@ -37,10 +38,9 @@
        ti,no-idle-on-init;
        ti,no-reset-on-init;
 };
-
+*/
 /* Uncomment below block to enable late attach for IPU1 */

-/*
 &ipu1 {
        ti,late-attach;
        ti,no-idle-on-init;
@@ -70,7 +70,6 @@
        ti,no-idle-on-init;
        ti,no-reset-on-init;
 };
-*/

Build the dtbs as mentioned in the release notes. This generates the dra7-evm-lcd-lg-late-attach.dtb to be flashed.

Building u-boot

Enable late attach in u-boot for Android eMMC loading by setting the following configs:

CONFIG_LATE_ATTACH=y
CONFIG_LATE_ATTACH_GPT_PART=y

Or from menuconfig:

ARM architecture --->
    Early Boot/Late Attach --->
        [*] Enable late attach
        [*]    Enable loading remotecore binaries from GPT partitions

Build the u-boot as mentioned in the Release Notes.

Building IPU1 binary

Build the IPU binary the same as described in the "Building IPU1 binary" section for 6AL.1.1 here.

A binary will be generated here:

<mydroid>/hardware/ti/ipc/packages/ti/ipc/tests/bin/ti_platforms_evmDRA7XX_ipu1/messageq_single.xem4

Since the size of the binary is quite large ( ~5M) we need to strip the symbols on the xem4. To do this, execute:

<CGT ARM tools folder>/bin/armstrip -p 

<mydroid>/hardware/ti/ipc/packages/ti/ipc/tests/bin/ti_platforms_evmDRA7XX_ipu1/messageq_single.xem4 -o <path of destination folder>/dra7-ipu1-fw.xem4

Building Android userspace binary

Build the Android userspace sample the same as described in the "Building Android userspace binary" section for 6AL.1.1 here

This should generate the sample binaries under the <mydroid>/out/target/product/jacinto6evm/system/bin and <mydroid>/out/target/product/jacinto6evm/system/lib (for 6AM SDKs)

or <mydroid>/out/target/product/jacinto6evm/vendor/bin and <mydroid>/out/target/product/jacinto6evm/vendor/lib (starting with Android O)

Binaries to be flashed into the eMMC

Flash the u-boot and MLO into the QSPI and reboot.

sudo ./fastboot oem spi
sudo ./fastboot flash xloader <path to built MLO>
sudo ./fastboot flash bootloader <path to built u-boot.img>
sudo ./fastboot reboot

Enter fastboot on the device and flash IPU1 binary and dtb into eMMC and reboot.

sudo ./fastboot flash ipu1 <mydroid>/hardware/ti/ipc/dra7-ipu1-fw.xem4
sudo ./fastboot flash environment <path to built dra7-evm-lcd-lg-late-attach.dtb>
sudo ./fastboot reboot

Binaries to be pushed to file system

For 6AM:

adb push <mydroid>/out/target/product/jacinto6evm/system/bin/messageQApp /system/bin
adb push <path of ipu1 binary>/dra7-ipu1-fw.xem4 /system/vendor/firmware

Starting in Android O:

adb push <mydroid>/out/target/product/jacinto6evm/vendor/bin/messageQApp /vendor/bin
adb push <path of ipu1 binary>/dra7-ipu1-fw.xem4 /vendor/firmware

NOTE

In case lad_dra7xx and other libraries are needed, also push those to the relevant folders.

Once this is done, you can change the switch setting to QSPI production as mentioned here or stay in dual stage boot mode.

Reboot the device. You should be able to see the single stage directly load the kernel if in single stage mode.

Validation

Once in the android command prompt, launch lad_dra7xx if it is not already launched.

To execute the test sample, execute:

/system/bin/messageQApp 10 2

This should do 10 iterations of the messageQApp (10) with IPU1 (2).

Using DMA Pool instead of CMA Pool

The 6AM.1.2 release introduces changes that allow selecting a DMA Pool to be used for the remote core instead of a CMA pool. This creates a carveout that can not be touched by the kernel even when it is not in use by remoteproc. This may be desirable in late-attach scenarios where there is a small window between early loading by u-boot and remoteproc allocating the CMA memory.

Enabling DMA Pool:

• In u-boot, in addition to the CONFIG_LATE_ATTACH and CONFIG_LATE_ATTACH_GPT_PART flags, enable the following flag:

CONFIG_LATE_ATTACH_DMA_POOL=y

Or from menuconfig:

ARM architecture --->
    Early Boot/Late Attach --->
        [*] Enable late attach
        [*]    Enable loading remotecore binaries from GPT partitions
        [*]   Enable using a DMA pool instead of a CMA pool

Rebuild the u-boot as mentioned in the release notes.

• In the kernel, use the dts file dra7-evm-lcd-lg-late-attach-no-map.dtb, or follow the example to make the changes for the desired configuration. The differences when compared to doing late attach are to add these additional lines:

+&ipu1_cma_pool {
+       /delete-property/ reusable;
+       no-map;
+};

 &mmu_ipu1{
        ti,late-attach;
+       ti,late-attach-dma-pool;
        ti,no-idle-on-init;
        ti,no-reset-on-init;
 };

This removes the "reusable" property from the IPU1 reserved-mem definition, creating a DMA pool instead of a CMA pool. Additionally, it adds a "late-attach-dma-pool" flag to the MMU entry for IPU1, which is needed for the IOMMU to take special action when using a DMA pool.

To run the IPU1 example from the previous section, modify the dts file to comment the IPU2 section and enable the IPU1 section:

--- a/arch/arm/boot/dts/dra7-evm-lcd-lg-late-attach-no-map.dts
+++ b/arch/arm/boot/dts/dra7-evm-lcd-lg-late-attach-no-map.dts
@@ -8,7 +8,7 @@
 
 #include "dra7-evm-lcd-lg.dts"
 
-
+/*
 &ipu2_cma_pool {
        /delete-property/ reusable;
        no-map;
@@ -44,10 +44,11 @@
        ti,no-idle-on-init;
        ti,no-reset-on-init;
 };
+*/
 
 /* Uncomment below block to enable late attach for IPU1 */
 
-/*
+
 &ipu1_cma_pool {
        /delete-property/ reusable;
        no-map;
@@ -83,7 +84,7 @@
        ti,no-idle-on-init;
        ti,no-reset-on-init;
 };
-*/
+

NOTE

• The allocation scheme for a DMA Pool is different from a CMA Pool. Depending on your image's memory requirements, you might find that the size of the existing pool is no longer sufficient for the allocation and alignment requirements of a DMA pool. If this happens, then either the pool size should be increased or the memory sections in the remote core image should be reassessed to adjust the sizes and/or order of the allocations to fit in the existing pool.
• The DMA Pool method requires vmalloc space whereas the CMA Pool method does not. This is because the CMA pool has a default mapping in the kernel but DMA pool does not. If you face vmalloc errors, the size of the vmalloc space may need to be increased.

Potential Issues

Debugging Crashes

Visibility into the slave execution can be extremely valuable when debugging. This can help detect system aborts, resource allocation failures, timer misconfigurations, blown stacks, and many other issues.

The best way to debug these scenarios is to attach to the crashed slave with CCS, load symbols, and inspect the state of the processor. Interesting things to look at include the program counter, trace logs, memory windows, ROV, etc.

If you can't attach CCS, inspecting logged statements in the trace buffer can still be helpful. The typically configured logger used to fill the trace buffer is a "SysMin" type, which means human-readable strings are rendered directly into the trace buffer (as opposed to a logger like LoggerBuf which is very fast and deterministic, but encodes the logs, which must then be decoded using a tool like CCS). The symbol of the trace buffer can vary based on your device, since different devices use different loggers:

• OMAP5, DRA7XX - ti_trace_SysMin_Module_State_0_outbuf__A
• All others - xdc_runtime_SysMin_Module_State_0_outbuf__A

You can look for this symbol in your slave's memory map, and inspect the memory to see any trace statements emitted during the slave's execution.

A slave's trace buffer can also be displayed from the Linux command line, by way of remoteproc's debugfs filesystem support. A debugfs directory is created for each remoteproc instance. Each remoteproc instance directory contains the following files:

       % ls /debug/remoteproc/remoteproc0
       name   recovery  state    trace0    version

The 'trace0' file contains the contents of the slave's SYS/BIOS trace buffer, and can be displayed with:

       % cat /debug/remoteproc/remoteproc0/trace0

.

The trace buffer contents are displayed starting from the base address of the trace buffer, which means that if trace buffer wraparound has occured then the debugfs file will begin with 'later' trace statements, i.e., the very first trace output will have been overwritten with later trace output. Since trace statements are typically prefixed with a timestamp, the earliest trace statements can be identified by finding the lowest timestamp prefix (and in the case of wraparound, this 'earliest' trace statement will be preceeded by the trace buffers's last or 'latest' trace statement).

Note: in order to have access to the above remoteproc instance debugfs files, you must mount the debugfs filesystem, e.g.,

       % mount -t debugfs none /debug

will mount the debugfs filesystem on the directory '/debug'.

My Tasks on the slave core do not get to run until the host has setup rpmsg

In most IPC examples, there is a line in the SYSBIOS configuration script (.cfg) file that adds IpcMgr_ipcStartup() as a startup function, which gets invoked during BIOS_start(), which is called in main():

<syntaxhighlight code='c'> /*

* Remove this line to _not_ sync with the host processor during startup.
* Remember to manually call IpcMgr_ipcStartup() when ready to "late attach" with the host.
*/

BIOS.addUserStartupFunction('&IpcMgr_ipcStartup'); </syntaxhighlight>

This function spin-waits for the host to setup rpmsg, and needs to be commented out if you want Tasking to be enabled on the slave so that it can do some 'work' before waiting for communication to be established with the host. The application would then call IpcMgr_ipcStartup() in their application whenever they are ready to establish communication with the host. Normally this is best done in a low-priority Task, so that higher-priority Tasks can run while this Task is spin-waiting for the host to setup rpmsg. Only after this function returns can the application use MessageQ to talk to the host.

Another approach is for the application to directly call the Resource_getVdevStatus function from the IPC Resource module to check the communication status, and only after that returns successfully, call IpcMgr_ipcStartup(). In between checks, the task can sleep or yield to prevent blocking other tasks from running. This could be accomplished in a manner like the below:

<syntaxhighlight code='c'> Void wait_for_a15() {

   Char hostStatus = 0;

   do {
       hostStatus = Resource_getVdevStatus(VIRTIO_ID_RPMSG);
       if (hostStatus == 0) {
           Task_sleep(200);
       }
   } while (hostStatus == 0);

} </syntaxhighlight>