TI811X PSP PCI Express Endpoint Boot Driver User Guide

TI811X PCI Express Endpoint Boot Driver User Guide

Linux PSP

Introduction

This document is applicable to TI811X.

TI811X device has a PCI Express hardware module which can either be configured to act as a Root Complex or a PCIe Endpoint. When configured as PCIe Endpoint (EP), the device can be set to boot over PCIe. This document caters to the Endpoint boot operation and describes the Driver needed to configure and operate on TI811X PCI Express Endpoint to perform booting. A sample application (part of release package) is used to carry out boot operation.

NOTE1: Support for TI811X Root Complex is added from 04.01.00.06 release, hence the references/setups described in this document related to TI811X as Root Complex are not applicable for prior TI811X releases.

Scope

This document covers the setup involving two TI811X EVMs connected together over PCIe using a PCIe cable having male connectors on both the ends. Here one device is set to boot in any other peripheral or memory boot mode (e.g., NAND) which will be designated as Root Complex (RC) and other TI811X device configured in PCIe Boot mode (EP).

Conventions

• This document refers TI811X device set up in PCIe boot mode as EP (Endpoint) and the TI811X device acting as PCIe Root Complex as RC.
• The shell commands executed on TI811X RC are preceded with ti811x-evm#.

Known Issues/Restrictions (Read this before you proceed)

This section lists various constraints applicable on TI811X PCIe EP boot operation. Most the the points listed below are covered in respective sections of this document.

• Only 32-bit PCIe boot mode is supported, 64-bit mode not supported currently.
• At the minimum, 3 BARs - BAR0, BAR1 and BAR2 are needed for boot operation and hence only the switch setting mentioned in next section is supported on EP.
• Other possible switch settings which may set up additional BARs may lead to issues in case RC is not able to allocate those BARs.
• In case ramdisk image is used as filesystem on EP, it cannot exceed 16MB. This restriction is imposed by the sample boot application to avoid exceeding the minimum BAR2 size (16MB) on TI811X.
• Note that the boot driver and application are written and tested to work with Linux kernel version 2.6.32 onwards. So make sure that the RC is running Linux kernel as supported.
• Current configuration of boot application skips checking boot flag clear indication from EP U-Boot (indicating DDR setup done) and proceeds to load other images to DDR. This may fail in case EP U-Boot is yet to set up the DDR. Try increasing boot flag check delay in this case. Refer Boot Application section below.
• Similarly, EP U-Boot doesn't check for any boot flag indication from the boot application (to notify about completion of subsequent download) and may proceed to load kernel (or ramdisk or 2nd stage U-Boot where applicable) even before it is completely loaded by boot application. In such cases, you can precede 'sleep 3' (or more) in the boot script or bootcmd. Alternately, you can also tune CONFIG_BOOTDELAY in U-Boot header file for corresponding board to a higher value to delay auto booting.
• Though the boot driver supports static build (building into kernel), some filesystem changes are required if you are using Arago/OE based filesystem with udev device cache. Please refer Updating the Filesystem section for details.

Setup

This document considers following distinct setups:

1. TI811X EVM RC - TI811X EVM BOOT-EP: This setup requires two TI811X boards (EVMs). Ensure that serial port (USB-UART [J6]) on each board is connected to host terminals.
2. TI811X EVM RC - TI8148 EVM BOOT-EP: This setup requires two TI811X boards (EVMs). Ensure that serial port on each board (USB-UART [J6] on TI811X) and UART0 on TI8148 EVM) is connected to host terminals.
3. TI8148 EVM RC - TI811X EVM BOOT-EP: This setup requires two TI811X boards (EVMs). Ensure that serial port on each board (UART0 on TI8148 EVM and USB-UART [J6] on TI811X) is connected to host terminals.

Note: In any of the cases covered here, it is advised to power on the EP device first and then RC.

Conflict with NAND/NOR and TI811X PCIe Boot Mode setting

The GPMC related boot pin CS0BW used for PCIe boot mode configuration is checked by u-boot and kernel to determine the 8-bit/16-bit NAND/NOR type connected on the board.

This will lead to conflict in few cases such as:

1. PCIe 32-bit boot mode with CS0BW=1 for BAR2 will require the DIP switch associated with CS0BW pin to be set to ON but U-Boot will take it as 8-bit NAND even if actual NAND on board was 16-bit. This will lead to NAND detection failures
2. Similar issue would occur when using 8-bit NOR where CS0BW=1 will be taken by NOR driver on Linux or U-Boot as NOR being 16-bit

Setting up the TI811X RC

• Flash the TI811X RC on board NAND with U-Boot. Refer PSP Flashing Tools Guide document for details.
• Set the TI811X EVM switch settings to enable NAND boot mode. Refer U-Boot User Guide for details about building U-Boot for NAND boot mode and switch settings.
• Use Male-Male (x1) PCIe cable to connect RC and with the PCIe slot on EP.
• Alternatively, you can set up RC to boot in any other (non-PCIe) boot mode.

Setting up TI811X EP

• Set the TI811X EVM designated as PCIe EP in PCIe 32-bit Boot mode as follows:

Boot mode select switch SW4[5:1] = 01000. Rest all switches should be '0' (zero) or 'OFF'

• This results into following boot pin configuration for TI811X device

BTMODE[[4:0] = 01000 ==> 32-bit PCIe Boot mode

• Note: Setting up any other combination of switch settings with PCIe Boot mode is not supported/validated.

Keeping the Required Modules Ready

Following section describes various modules involved in EP boot operation and steps for preparing binaries and U-Boot script required to be present on PCIe RC to boot TI811X EP. In addition, it is assumed that a ramdisk filesystem or NFS based filesystem for EP are available.

Note: Though the kernel (uImage) built with default config for TI811X RC will boot on TI811X EP, it will lead to conflicting configuration of the system since the kernel booting on EP will switch the PCIe mode from EP to RC. Due to this, the communication between PCIe RC and EP will no longer happen. For this, the kernel needs to be re-built with PCI support removed for booting on EP.

Also note that the sections below assume you have extracted the TI811X release package to a directory referred as $PKGDIR and build environment is setup as described in TI811X PSP User Guide.

PCIe EP Boot Driver

Note: The default kernel build configuration does not select the driver and needs to be enabled for the kernel as explained in respective sections below.

This driver runs on TI811X RC. It configures the first TI811X EP device detected in the system and configures it to be able to carry boot operation.

Features Supported

• Support for detecting and configuring TI811X/TI814X devices
• Provides character device interface on Linux Kernel to PCIe boot user-space application
• Provide mmap support to enable the boot application to copy image files (U-Boot, kernel etc) to EP memory
• Can be built as loadable module or into kernel

Features NOT Supported

• Operate more than one TI811X EP.
• No interrupt support
• Not validated on any other RC than TI811X/TI814X. This driver will require porting in case you are using Linux kernel prior to 2.6.32 or any other h/w platform than TI811X/TI814X.
• No support for power management (e.g., suspend/resume)

Supported IOCTLs

• TI81XX_PCI_GET_BAR_INFO: Returns the size in bytes of the specified BAR.

int dev_desc;
dev_desc = open("/dev/ti81xx_pcie_ep", O_RDWR);
...
struct ti81xx_bar_info bar;
bar.num = bar_number;
ioctl(dev_desc, TI81XX_PCI_GET_BAR_INFO, &bar);
...

In the above code fragment, the driver returns BAR size in 'size' field of the 'bar' structure object on success.

• TI81XX_PCI_SET_BAR_WINDOW: Application can specify the internal address on EP for specified BAR. For example, the boot application sets BAR1 to OCMC1 start on EP (0x40300000) for TI811X EP using this ioctl.

...
struct ti81xx_bar_info bar;
bar.num = 1;
bar.addr = 0x40300000;
ioctl(dev_desc, TI81XX_PCI_SET_BAR_WINDOW, &bar);
...

• TI81XX_PCI_SET_DWNLD_DONE: Set the bootflag on EP. The driver writes '1' to the location 0x4033B0FC on TI811X EP and waits for the flag to be cleared till maximum of the time (in seconds) as specified by the application. Timeout of '0' means no wait for the flag clearing.

...
ioctl(dev_desc, TI81XX_PCI_SET_DWNLD_DONE, 3);
...

The IOCTL and data structure declarations are in drivers/char/ti81xx_pcie_bootdrv.h file in TI811X kernel source.

Source Files

The driver files are present at following path relative to extracted kernel source directory for TI811X.

• drivers/char/ti81xx_pcie_bootdrv.h
• drivers/char/ti81xx_pcie_bootdrv.c

Steps for Building for TI811X Kernel

• The steps listed below cover building details for TI811X with kernel release (04.01.00.06 onwards) and the EVM default configuration name is ti811x_evm_defconfig.
• Navigate to TI811X kernel source directory and extract the kernel source and create default TI811X EVM configuration.
• Apply the patch by copying it to kernel source directory.

The download link has the patch in gzipped format so it needs to be extracted to get the patch file. You can then use 'patch' utility to apply the patch as:

# gunzip 0001-TI811x-PCIE-Add-TI811X-related-address-offsets-and-c.patch.gz
# patch -p1 < 0001-TI811x-PCIE-Add-TI811X-related-address-offsets-and-c.patch

• Note that the release version suffix used in example below can be any later PSP version.

LINUXHOST$ cd $PKGDIR/src/kernel
LINUXHOST$ tar -xzf linux-04.07.00.01.tar.gz
LINUXHOST$ cd linux-04.07.00.01
LINUXHOST$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm ti811x_evm_defconfig

• Enter the kernel configuration menu

LINUXHOST$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm menuconfig

• This will open the kernel configuration menu
• Use DOWN Arrow key to till "Device Drivers" is highlighted

    General setup  --->
[*] Enable loadable module support  --->
...
Device Drivers  --->

• Press ENTER to go inside this section. Use DOWN Arrow key to select "Character devices"

    Generic Driver Options  --->
...
    Character devices  --->

• Again press ENTER and scroll down to select "TI81XX PCIe Endpoint Boot Driver". Press 'm' to enable to build as a loadable module.

-*- Virtual terminal
...
<M> TI81XX PCIe Endpoint Boot Driver

Note that the above option will be shown only if PCI Bus support is enabled in kernel configuration. This is default in ti811x_evm_defconfig.

• Use RIGHT Arrow key to bring focus on 'Exit' and press ENTER. Repeat this to return to previous menu(s) and eventually back to the shell. Make sure to save the changes you have made when prompted at exit.
• Build the driver

LINUXHOST$ make modules.

• The module will be built as drivers/char/ti81xx_pcie_bootdrv.ko
• The above steps build the driver as loadable kernel module but it can also be built into kernel by pressing 'y' instead of 'm' in kernel menu configuration mentioned above.

U-Boot

For PCIe boot, the ROM on the EP directly jumps to the start of OCMC RAM when indicated by the boot driver (on RC) that the boot image is copied. This means that the raw U-Boot binary (without any preceding header) must be used for booting.

TI811X

For booting TI811X EP, you will need U-Boot binaries for 2 stages:

• For 1st stage, you will need to build the U-Boot binary with following steps:

1. Extract the U-Boot sources and navigate to the base directory of U-Boot source

2. Make sure that the complete build environment as required for U-Boot build is setup on Linux host. For details, refer U-Boot User Guide from the release package or online here

3. Prepare the configuration for build

make ti811x_evm_min_sd

4. Build U-Boot

make u-boot.ti

5. This will create u-boot.bin file, rename this to MLO

• For 2nd stage, you can use the pre-built U-Boot binary u-boot.bin from $PKGDIR/images/u-boot/ti811x directory

TIP: Use following command at Linux shell to strip off 8 bytes -

# dd skip=8 if=MLO of=MLO.no_header bs=1
# mv MLO.no_header MLO  

PCIe EP Boot Application

This application runs on TI811X RC running Linux and accesses the interfaces provided by the EP Boot Driver to download U-Boot, Kernel, etc images to EP and trigger EP boot.

You need not follow the build steps mentioned below if you want to use pre-built saBootApp ELF present in $PKGDIR/images/examples/pcie directory. Note that, this ELF is built to be executed on TI811X RC.

Features Supported

• Download U-Boot, U-Boot bootscript, Kernel and (optionally) filesystem images to EP memory
• Uses 2 stage boot loading in case of TI811X EP -
• Requires only 3 BARs (BAR0, BAR1 and BAR2) to perform complete boot operation. Uses EP boot driver to move internal EP windows to access OCMC and DDR.

Features NOT Supported

• Cannot operate without EP Boot Driver
• Does not support ramdisk filesystem size greater than 8MB
• Kernel image size limit is 8MB and bootscript size is 2MB maximum

Steps for Building for TI811X RC

• Navigate to TI811X examples source directory

LINUXHOST$ cd $PKGDIR/examples/pcie

• Apply the patch by copying it to pcie directory.

The download link has the patch in gzipped format so it needs to be extracted to get the patch file. You can then use 'patch' utility to apply the patch as:

# gunzip 0001-TI811x-PCIE-Add-TI811X-support-for-Boot-Application.patch.gz
# patch -p1 < 0001-TI811x-PCIE-Add-TI811X-support-for-Boot-Application.patch

• Open file Makefile and set the path to kernel source directory as applicable. For more details, refer README file present in the directory.
• Build the application by typing 'make' at the shell. This will create an ELF file named 'saBootApp'.

Tuning Boot Application

In some cases, the defaults chosen by the boot application may require to be changed. Various such options are covered below with their default settings. Note that in such case, you cannot use the pre-built Boot Application ELF file and will need to build the application as mentioned above.

Following options are defined in saBootApp.c file: -

• CONFIG_IGNORE_BOOTFLAG_FAIL: This flag is enabled by default (set to '1') which directs the boot application to proceed even if the boot flag is not cleared. This is enabled currently since present U-Boot doesn't clear the boot flag and having this flag enables the boot application to proceed to load kernel and boot EP. You can set this flag to '0' if you update the U-Boot to clear the boot flag after setting DDR (refer Boot ROM spec for location of boot flag).
• CONFIG_BOOTFLAG_FAIL_DELAY: Time in seconds to wait for boot flag to be cleared. Default is 0 sec. You can lower/increase this value if you face booting issues

Additional configuration options such as load addresses for various images, etc., are present in Boot Driver header file ti81xx_pcie_bootdrv.h located in $PKGDIR/src/kernel/linux-04.00.00.09/driver/char directory and are used by the boot application.

PCIe EP U-Boot Boot Script

In PCIe boot scenario, the U-Boot environment storage may not be available on EP. In such cases loading a U-Boot script file (referred as 'bootscript') on EP with various environment variables set helps simulating environment storage. This also adds flexibility in a way that the various boot parameters such as kernel boot arguments, filesystem type, paths for EP can be changed though this script without any need to rebuild the U-Boot or need to explicitly set the U-Boot environment variables on EP. Also, setting U-Boot parameters on EP will require to have UART console connected to EP every time.

Creating U-Boot Script

• Refer sample script file named bootscript.txt is provided in $PKGDIR/src/examples/pcie directory.
• Edit/add parameters as per your requirement. The default script sets up kernel to use RAMDISK image.
• You will need U-Boot's 'mkimage' utility to build the script image from this file. Refer U-Boot User Guide for details about 'mkimage'
• Ensure that the PATH is set to point to 'mkimage'
• Build the boot script as

LINUXHOST$ cd $PKGDIR/src/examples/pcie
LINUXHOST$ mkimage -A arm -O linux -T script -C none -n 'PCIe Boot Script' -d bootscript.txt boot.scr

• Alternatively, for booting TI811X EP, you can use pre-built boot.scr present inside $PKGDIR/images/examples/pcie directory. This is built from sample bootscript.txt

Note: The sample boot script packaged in release is for TI811X EP only.

Auto Booting with the Bootscript

This approach requires modifying and re-building U-Boot. The release version numbers used in examples below will vary depending upon the TI811X PSP release being used.

Note: The current U-Boot default boot command is set to load the bootscript from MMC. Thus, pre-built u-boot.bin U-Boot binary from the release package, loaded on EP using boot application will not auto boot the kernel image downloaded over PCIe. The same issue applies if the U-Boot is built from source for SD boot. The 1st stage U-Boot modes select autoload of 2nd stage from respective boot medium/methods, thus will bypass loading U-Boot downloaded over PCIe.

This can be resolved by directing the U-Boot running on EP to load the bootscript from DDR location 0x80400000 (as loaded by the boot application) in case of TI811X 2nd stage U-Boot. In addition, the 1st stage U-Boot needs to be directed to autoload 2nd stage U-Boot from DDR.

This can be done as described in following sections:

TI811X

Extract the U-Boot source as per the approach described above and open following file to update:

• TI811X: include/configs/ti811x_evm.h

For 1st stage U-Boot, change bootcmd as part of CONFIG_EXTRA_ENV_SETTINGS for respective boot mode configuration being used:

E.g., for TI811X when using SD boot 1st stage U-Boot,

• Go inside the CONFIG_TI811X_MIN_CONFIG section in the above file and look for following

# elif defined(CONFIG_SD_BOOT)          /* Autoload the 2nd stage from SD */

• In CONFIG_EXTRA_ENV_SETTINGS, set the bootcmd (removing original line for bootcmd) as

"bootcmd=go 0x80800000\0" \

• Save the file and build U-Boot

LINUXHOST$ make clean 
LINUXHOST$ make ti811x_evm_min_sd
LINUXHOST$ make

• Use the u-boot.bin built in the above step as 1st stage U-Boot (MLO)

For 2nd stage U-Boot, change bootcmd as part of CONFIG_BOOTCOMMAND:

E.g., for TI811X,

• Go inside the #else part of CONFIG_TI811X_MIN_CONFIG section in the EVM header file mentioned above and look for following

# define CONFIG_BOOTCOMMAND \

• Replace the #define as follows to run bootscript

#define CONFIG_BOOTCOMMAND "source 0x80400000"

• Save and build the 2nd stage for desired medium (e.g., SD)

LINUXHOST$ make clean 
LINUXHOST$ make ti811x_evm_config_sd
LINUXHOST$ make

• The u-boot.bin built in above step can be used as 2nd stage U-Boot.

Note: bootcmd set explicitly in storage medium used will take precedence over the bootcmd set above in the source.

Saving Boot Command on EP Storage

This approach requires access to U-Boot console and availability of persistent storage such as NAND on EP.

In case of TI811X, this is only possible for 2nd stage U-Boot.

• Boot U-Boot on EP using PCIe Boot procedure described in next sections
• Halt U-Boot countdown by pressing ENTER at EP console
• Set 'bootcmd' at U-Boot prompt on EP as:

set bootcmd 'source 0x80400000'
saveenv

• Note that this is a one time operation and U-Boot will jump to correct script location on subsequent resets.

Building Kernel for EP

Note: The change mentioned below is not needed for TI811X when using release 04.01.00.05 or older and the kernel uImage built with ti811x_evm_defconfig can be used. For 04.01.00.06 and onwards release of TI811X kernel, the steps mentioned below should be followed since they have Root Complex support included.

As mentioned earlier in this section, the kernel built with PCIe RC support will result into conflicting configuration when booted on EP. To avoid this, it is recommended to rebuild the kernel with PCIe RC support disabled.

• As explained before, extract kernel source tarball and enter kernel configuration menu.

    General setup  --->
[*] Enable loadable module support  --->
-*- Enable the block layer  --->
   System Type  --->
   Bus support  --->
...

• Scroll down with the DOWN Arrow key till "Bus Support" gets selected. Press ENTER. This will show following menu:

[*] PCI support
[*] Message Signaled Interrupts (MSI and MSI-X)
...

• Disable "PCI support" by pressing 'n' key or SPACE key. This will automatically hide other dependent configurations and the above configuration menu will look like as shown below:

[ ] PCI support 
< > PCCard (PCMCIA/CardBus) support  --->

• Use RIGHT Arrow key to bring focus on 'Exit' and press ENTER. Repeat this to return to previous menu(s) and eventually back to the shell. Make sure to save the changes you have made when prompted at exit.
• Build the kernel.

Updating the Filesystem

The kernel may not boot or provide shell prompt when the boot driver is built into kernel. This issue is due to the device node numbers (major numbers) used for console device(s) not getting updated as udev in the filesystem still uses stale nodes from the device cache in the filesystem even on detection or removal of EP device across RC reboots.

This issue can be avoided by forcing the rebuild of device cache on detecting change in boot time device detection. This can be achieved by updating the filesystem as follows:

• Add following check in /etc/rcS.d/S03udev inside DEVCACHE block

[ -r /proc/devices ] && cat /proc/devices > /tmp/devices || touch /tmp/devices

• Similarly, add comparison for saved device list to detect changes in devices

cmp -s /tmp/devices /etc/udev/saved.devices && \

• Now the devcache block will look as follows

if [ "$DEVCACHE" != "" ]; then
        # Invalidate udev cache if the kernel or its bootargs/cmdline have changed
        [ -x /bin/uname ] && /bin/uname -mrspv > /tmp/uname || touch /tmp/uname
        [ -r /proc/cmdline ] && cat /proc/cmdline > /tmp/cmdline || touch /tmp/cmdline
        [ -r /proc/devices ] && cat /proc/devices > /tmp/devices || touch /tmp/devices
        [ -r /proc/atags ] && cat /proc/atags > /tmp/atags || touch /tmp/atags
        if [ -e $DEVCACHE ] && \
           cmp -s /tmp/uname /etc/udev/saved.uname && \
           cmp -s /tmp/cmdline /etc/udev/saved.cmdline && \
           cmp -s /tmp/devices /etc/udev/saved.devices && \
           cmp -s /tmp/atags /etc/udev/saved.atags; then
                (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
                not_first_boot=1
        fi
fi

• Update udev devcache script /etc/rcS.d/S12udev-cache to save device list by adding following line

mv /tmp/devices /etc/udev/saved.devices

• Update the same file with a line to delete temporary device list

rm -f /tmp/devices

• Now the dev cache code in this file will look like

if [ "$DEVCACHE" != "" ]; then
        echo -n "Populating dev cache"
        (cd /; tar cf $DEVCACHE dev)
        mv /tmp/uname /etc/udev/saved.uname
        mv /tmp/cmdline /etc/udev/saved.cmdline
        mv /tmp/atags /etc/udev/saved.atags
        mv /tmp/devices /etc/udev/saved.devices
        echo
else
        rm -f /tmp/uname
        rm -f /tmp/cmdline
        rm -f /tmp/atags
        rm -f /tmp/devices
fi

• The above changes will ensure that udev will rebuild the device cache depending upon detection of PCIe EP presence or removal across RC reboots.

Other simpler (but not recommended) option is to turn of the device cache altogether. This can be done by commenting the DEVCACHE line in /etc/default/udev of the filesystem.

Ensuring the PCIe System Initialization

Before proceeding to using the TI811X PCIe EP in the system, we must ensure that the RC is up and has detected and configured the PCIe EP. Steps mentioned below should be carried out before actually accessing the EP and proceeding for boot.

• Make sure that the board setup instructions mentioned in earlier section are followed.
• Power on EP
• Power on RC
• Once shell is available at RC, type 'lspci' command, it should show following two devices (EP in this example is TI811X):

ti811x-evm# lspci -v
00:00.0 Class 0604: Device 104c:b801 (rev 01)
       Flags: bus master, fast devsel, latency 0
       Memory at <ignored> (32-bit, non-prefetchable)
       Memory at <ignored> (32-bit, prefetchable)
       Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
       Memory behind bridge: 21c00000-21cfffff
       Prefetchable memory behind bridge: 20000000-21bfffff
       Capabilities: [40] Power Management version 3
       Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
       Capabilities: [70] Express Root Port (Slot-), MSI 00
       Capabilities: [100] Advanced Error Reporting
01:00.0 Class 0004: Device 104c:b801 (rev 01)
       Flags: fast devsel, IRQ 48
       Memory at 21c00000 (32-bit, non-prefetchable) [size=4K]
       Memory at 21000000 (32-bit, prefetchable) [size=8M]
       Memory at 20000000 (32-bit, prefetchable) [size=16M]
       Memory at 21800000 (32-bit, prefetchable) [size=4K]
       Memory at 21801000 (32-bit, prefetchable) [size=4K]
       Capabilities: [40] Power Management version 3
       Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
       Capabilities: [70] Express Endpoint, MSI 00
       Capabilities: [100] Advanced Error Reporting

• If 'lspci' is not available in your filesystem, look into /sys/bus/pci/devices directory to see if EP is detected. You should see two sub directories in this directory, e.g., see following command and its output:

cat /sys/bus/pci/devices/0000\:0*/device
0xb801
0xb801

• The first device is RC itself and the second device is EP
• Notice that the EP is allocated three BARs - BAR0, BAR1 and BAR2 of size 4KB, 8MB and 16MB respectively.

Load Addresses

The boot application loads various images at following addresses on EP:

Image	EP Device	Load Address
U-Boot 1st Stage	TI811X	0x40300000
U-Boot 2nd Stage
	TI811X	0x80800000
Kernel
	TI811X	0x80900000
Ramdisk
	TI811X	0x81000000
Bootscript
	TI811X	0x80400000

Booting EP

• Download PCIe EP Boot driver, U-Boot, U-Boot script, kernel image and ramdisk file (optional) on RC.
• These files can be downloaded over network using TFTP or can be copied to NFS mounted filesystem
• In our example, we will use following commands on RC:

ti8168-evm# tftp -m binary 192.168.0.1 -c get ti81xx_pcie_bootdrv.ko
ti8168-evm# tftp -m binary 192.168.0.1 -c get saBootApp
ti8168-evm# tftp -m binary 192.168.0.1 -c get u-boot.bin
ti8168-evm# tftp -m binary 192.168.0.1 -c get boot.scr
ti8168-evm# tftp -m binary 192.168.0.1 -c get uImage
ti8168-evm# tftp -m binary 192.168.0.1 -c get ramdisk.gz

Note: Transfer the 1st stage U-Boot MLO used for MMC/SD boot of TI811X EP.

• Make boot application as executable:

ti811x-evm# chmod a+x saBootApp

• Insert boot driver:

ti811x-evm# insmod ti81xx_pcie_bootdrv.ko
ti81xx_pcie_ep: Found TI81xx PCIe EP @0xcc827000, DEVICE ID = b801
pci 0000:01:00.0: This driver supports booting the first TI816x or TI814x target found on the bus
pci 0000:01:00.0: Major number 252 assigned
pci 0000:01:00.0: Added device to the sys file system
pci 0000:01:00.0: BAR Configuration -
          Start        |       Length  |       Flags
pci 0000:01:00.0: 0 index 0 bars
pci 0000:01:00.0:       0x21c00000      |       0x00001000      |       0x00040200
pci 0000:01:00.0: 1 index 1 bars
pci 0000:01:00.0:       0x21000000      |       0x00800000      |       0x00042208
pci 0000:01:00.0: 2 index 2 bars
pci 0000:01:00.0:       0x20000000      |       0x01000000      |       0x00042208
pci 0000:01:00.0: TI81XX registers mapped to 0xd0906000
pci 0000:01:00.0: TI81XX OCMC mapped to 0xd5000000
pci 0000:01:00.0: TI81XX DDR mapped to 0xd6000000

• The above log shows that three BARs on the EP are set up with 4KB, 8MB and 16MB size respectively.
• You may need to create node "/dev/ti81xx_pcie_ep" with the major number displayed when PCIe Boot Driver module is inserted. This depends on the filesystem and configuration, e.g., 'udev' should do this automatically.
• In case the device node is not created automatically, use following command sequence:

ti8168-evm# cat /proc/devices
 Character devices:
    1 mem
    :
    : 
    253 ti81xx_pcie_ep            <--- Our device major number is 253
    :
ti811x-evm# mknod /dev/ti81xx_pcie_ep c 253 0

• Run the boot application providing U-Boot, bootscript, kernel and ramdisk names as:

ti811x-evm# ./saBootApp u-boot.bin boot.scr uImage ramdisk.gz

• For TI811X EP, ensure that MLO binary is present in same directory as boot application
• Alternatively, you can use NFS root system on EP by updating the bootscript.txt accordingly and using the corresponding bootscript image. In this case, you need not download the ramdisk image.

ti811x-evm# ./saBootApp u-boot.bin boot.scr uImage

• The boot application will use boot driver to configure the EP and download the images to complete boot operation.

Note: You will see some messages about "Bootflag clear timed out" during the boot process. This is normal since the current U-Boot doesn't support clearing the boot flag in OCMC1 RAM at EP (location 0x4033B0FC for TI811X) as set by the PCIe Boot code running on RC. The application will still proceed to complete boot operation.