BeagleBone in a Board Farm

Objective

This article will describe and recommend methods to use BeagleBone in a board farm for automated software testing.

In this article we set a pretty high bar for testing and control. The end goals are:

• Recover use of the board regardless of what actions the test software performs
  • This includes things like accidentally erasing the boot code
• Simulate all normal user modes
  • This includes use of a uSD card
  • for BeagleBone Black, it includes the use of the eMMC and simulation of both states of the "Boot" button.
• Be able to test all target software
  • This includes testing new versions of bootloaders

If the goals above are relaxed, simpler methods could be employed and some of these will be described as alternatives.

Although the article is written for BeagleBone, many principals can be applied to other boards.

The Host system will be assumed to be a Linux system. It is a general recommendation that any true board farm use Linux as its control element. Some of this article could apply to a Windows host but such use is out of scope.

The majority of this article is applicable regardless of the target software under use. Specific issues with using U-boot will be discussed.

Hardware Setup: BeagleBone Black

BeagleBone Black with FTDI serial and ETH-RLY16 control of power, SYS_BOOT2/3, and RESET

BBB Serial Port Access

BBB showing FTDI Cable connected

The BeagleBone Black (BBB) has a header for UART0 that uses 3.3V TTL levels. This header is the primary reason BBB is easier to use in a board farm than the original BeagleBone.

Recommendation: The easiest way to get access to the serial port is to use a USB to TTL serial cable or converter. There are several available including the FTDI TTL-232R-3V3 cable and a converter board available from Special Computing (product # [20411]). The FTDI cable is a common method and has been tested by the author. The converter board is a little cheaper and has indicator lights for serial port activity. You will need to supply a standard mini-B USB cable for the converter board or buy one on the same page. (At this time the author has not tested the converter board.)

Both the recommendations above use USB as the interface to the computer. However, unlike some USB serial port solutions the USB port will not power the BBB so it will not prevent a clean power cycle. Since these cables are powered by the host or hub USB port, they will not disappear and reappear on target power cycle. This allows host serial port software to be run before board power and remain active during a target power cycle. However, if multiple of these cables/converters are used on the same host, udev rules may need to be written to ensure that they always use consistent device names on host system boots.

Pololu 23201a

Close up of connections

Alternative: If your board farm uses a ganged serial port concentrator or you have another reason to prefer a RS-232 level serial port, there is another option. The Pololu 23201a Serial Adapter can be wired to the serial port header and expansion header to get a real DB9 serial port.

The connections for this adapter are:

Adapter  BBB
RX       J1.5 TX
TX       J1.4 RX
GND      J1.1 GND
VCC      P9.4 VDD_3V3B

BBB Power Control

For BeagleBone Black (BBB), this article will make a simplifying assumption that all power will be applied via the 5V barrel connector and that the USB mini-B connector will not be used. This connector is normally used to connect the BBB to a PC. This violates the goal of being able to test all of the target software in the modes users will use. However, it makes things considerably simpler. There is no restriction on using the USB host connector to connect to a device.

To eliminate this assumption and use the USB mini-B, see the power control section for the original BeagleBone where USB power control can not be ignored.

Recommendation: Power control of boards is a basic tenet of board farms. Most board farms will already have a method to do this. The BBB is compatible with any such solution. Plug the AC plug of a 5V Adapter into a controlled outlet and plug the DC plug into the BBB barrel connector.

If you don't already have power control solution here are several references. In TI labs we use a lot of the APC AP7900 Ethernet controlled power strips ($450-550). For a less expensive option for the author's home lab, I have used the Digital Loggers Web Power Switch 7 ($129).

Relay power control on +5V conductor

Alternative: For a small lab you may want to double up on functions. As you will see in the next section, we will use an Ethernet controlled relay for bootmode control. These relays can be used to switch power as well. If you pursue this option it is strongly recommended that you put the relay on the +5V conductor. This avoids any 120/240 voltage in screw terminals and maintains a ground connection from the board to the outlet.

Caution: A loose or stray wire connection to the relay could be a safety hazard. Do not try to switch the AC side of the adapter. Even the 5V side can supply enough current to cause sparks, melting, or even fire. Ensure you have the proper skills before trying this method. Tin any stranded wire and secure the connection tightly. Do not leave extra exposed wire. Review the specs of your relay module to ensure it is rated for the current and voltages used.

BBB Boot Mode Control

In order to achieve all the goals defined above, it is required to be able to recover a board that software under test has "bricked". The normal bricked board recovery model for BBB is to insert a recover image into the uSD slot and hold the "boot" button while powering on the board. This model of recovery and others requires extra hardware.

Recommendation: Use two relays from an Ethernet controlled relay board to separately enable 4.7K ohm pull-downs on SYS_BOOT2/LCD_DATA2/P8.43 and SYS_BOOT3/LCD_DATA3/P8.44.

SYS_BOOT3 relay	SYS_BOOT2 relay	SYS_BOOT	Description	Sequence
open	open	0100_0000_0011_1100	BBB Normal	MMC1 (eMMC)	MMC0 (uSD)	UART0	USB0
open	closed	0100_0000_0011_1000	BBB w/ Boot switch	SPI0 (inactive)	MMC0 (uSD)	USB0	UART0*
closed	closed	0100_0000_0011_0000	recovery	XIP (inactive)**	UART0	ENET1	MMC0*

Note: * that the ENET1 and USB0 modes try for about 5 minutes before giving up and moving to the next boot mode. It may be impractical to use boot modes that appear in the sequence after the USB0 and ENET1 modes.

Note: ** It appears to be safe to pass through XIP mode to get to the other modes but it is not clear if the HW design accounted for this option. The active probe of the XIP memory will return FF but it is not clear if there are other bus cycles that could produce a short lived bus conflict. To avoid a bus conflict it should be ensured that the BBB USB Host connector is not in a current overload condition during boot.

Any software controllable relay should work fine. In TI labs we use a lot of the ETH-RLY16 modules that have 8 relays (~$80). The same company also has boards with smaller relay counts and that have USB control instead of Ethernet. For the authors' home lab, I have also tried the Digital Loggers DIN III relay unit, also with 8 relays (~ $140). You will need to supply your own power adapter for either option.

To perform a recovery operation with this configuration, both relays would be engaged and the BBB would be powered on. Either UART or Ethernet boot would be used to load a recovery image. The recovery image would then be instructed to write to the eMMC and/or uSD card. Then the BBB would be power cycled with the relays selecting one of the normal boot modes.

Note that the LCD module should be able to overdrive the 4.7K ohm pull downs and the HDMI output of the BBB should not be effected with these pull-downs active. This has been tested and no visible effects were seen. However, the pull-downs are only required for a few seconds after power up and can be disabled afterward if desired or if these signals are used on a cape.

Optional: Additional relays can be used to simulate the user pressing the reset button and power buttons. To simulate the reset button a relay controlled 100 ohm pull down can be connected to SYS_RESETn/P9.10. To simulate the power button a relay controlled 100 ohm pull-down can be connected to PWR_BUT/P9.9. Control of these buttons is not required for recovery but may be desired based on the target test cases.

Alternative: The Linaro LAVA lab uses a board called the SD-MUX. This board allows either the board under test or a PC Host (or neither) to have access to an SD card. With the right adapters this card could be used with the BBB.

To use this mode of recovery one of the following would need to be true:

• eMMC is blank and the lab trusts the software not to write to it
• a constant 4.7 K pull-down is added to LCD_DATA2 to force MMC0/uSD boot (boot button down)
• a relay controlled 4.7 k pull down is added to LCD_DATA2

(Only the last method achieves the goals defined for this article.)

To perform a SD card recovery in this model the board would be powered off and the SDMUX used to get host access to the SD card. The new test image would be written to the SD card and then the SDMUX would be set for BBB control and the board booted from MMC0/SD.

For the relay controlled model, an eMMC recovery would start by using the PC to write a recovery image to the SD card. The BBB would be booted from MMC0/SD and the recovery image would then write to the eMMC. Finally the board would be power cycled and brought back up in MMC1/eMMC boot mode.

Reduced Function Alternatives: If you are willing to sacrifice some of the goals of this article, there are options that require no hardware for boot mode control.

The methods below are different methods to load a bootloader. The idea is move the element of control to the bootloader. One example of this would be to have the bootloader do nothing at load and wait for commands from the UART. Another example would be to have the bootloader always boot via Ethernet and use host software to change the files that it gets.

• put your bootloader in eMMC and trust the software not to disturb it.
• use two constant 4.7K pull-downs on SYS_BOOT2 & SYS_BOOT3 to force the board to always boot in recovery mode.
  • Use UART or Ethernet boot mode to load the bootloader
• erase the eMMC and leave the uSD slot blank
  • Use UART boot mode to load the bootloader

Hardware Setup: Original BeagleBone (white)

This section is still being verified. Consider this information theoretical.

BB Serial Port Access

The original BeagleBone ties UART0 directly to the on-board FTDI serial-to-USB chip. The signals do not appear on any connector nor do they go through any resistors or other components that could be removed. The USB mini-B connector also supplies power to the board so it is not possible to maintain a serial connection across board power cycles.

Recommendation: Use USB serial access

Since all the standard software for BeagleBone uses UART0, the only way to test this standard software and still have serial port access is to use USB serial access. udev rules will need to be used to ensure consistent naming across power cycles if more than one BeagleBone is to be connected to the same host. The serial number in the BeagleBone may be used for this purpose or the physical topology may be used.

TODO: example udev rules

Alternative: Use other serial ports

Other serial ports are available on the expansion headers. If testing standard BeagleBone software is not a concern these could be used. However it may take a good bit of work to find all the places in the software that assumes UART0 is used. UART bootmode will not be available as the ROM only implements this for UART0.

BB Power Control

Two forms of power control will be needed for the original BeagleBone, 5V barrel control and USB power control. To get a full power cycle no residual power must be available from either source. To get access to UART0, USB must be powered. To get full operational speed, power from the 5V barrel must be available as the USB standard limits the current available from the USB source.

Recommendation: No recommendation yet. A readily available solution using non-obsolete device is yet to be found.

Alternative: Use a hub with per port power switching

The USB standard provides this for this capability of standard hubs. On paper it is very attractive as one hub can be used for many boards. However, it is hard to find hubs that actually implement this. In addition some hubs declare that they support this feature in their USB device descriptors but don't actually implement it correctly.

One hub that did provide this feature was the older version of the D-Link DUB-H7. This version can be identified by the silver and gray case and an LED next to each port. Unfortunately D-Link is now selling a new model using the same product name. This new product is all black and declares that it implements per port power switching but it does not work correctly.

Alternative: Use two switched outlets, one for the 5V adapter and one for the power adapter for a USB hub.

Another solution that sounds simple in theory but compatible hubs are unknown at this time.

Implement 5V barrel control as described for BeagleBone Black.

To control USB power plug the BeagleBoard into a powered USB hub. Do not connect unrelated devices into this hub. Plug the power adapter for this hub into another switch outlet. Make sure you do not use a hub designed to be used for mobile users as these hubs will supply current from the host if the adapter does not supply power. Unfortunately all USB 2.0 hubs I tested continue to supply power from the PC when the power adapter is unplugged or turned off.

Alternative: Insert a relay into the 5V conductor of a USB cable.

This should work but is untested by the author. Another variant of this option is to buy a hub that has manual power switches on each port such as this one and disassemble the hub to replace the power switches with wires for relay connection.

Alternative: Skip power control.

Use relay control of reset as described below to start new software test cycles.

This method does not test interactions of the board (SOC and PMIC) as a true power cycles would. However, most software testing at TI for the original BeagleBone is done this way.

BB Boot Mode Control

In order to achieve all the goals defined above, it is required to be able to switch software that is on the uSD card. As the original BeagleBone is stateless there really is no bricked state for the hardware to get into. However this assumes that the user can replace a uSD card whose image has been corrupted. This model of software switching requires extra hardware.

Recommendation: Use two relays from an Ethernet controlled relay board to separately enable 4.7K ohm pull-downs on SYS_BOOT4/GPIO2_10/P8.41 and SYS_BOOT1/GPIO2_7/P8.46. Use an additional relay to simulate the reset button push by controlling a 100 ohm pull down connected to SYS_RESETn/P9.10.

SYS_BOOT4 relay	SYS_BOOT1 relay	SYS_BOOT	Description	Sequence
open	open	0100_0000_0001_0111	BB Normal	MMC0 (uSD)	SPI0 (inactive)	UART0	USB0
closed	open	0100_0000_0000_0111	EMAC recovery	EMAC1	MMC0 (uSD)	XIP**	NAND**
closed	closed	0100_0000_0000_0101	UART recovery	UART0	XIP**	SPI0	NANDI2C**

Note: ** The behavior of these modes is not fully defined for the BeagleBone original hardware. They are not intended to be used and the time spent in them should be minimized.

Any software controllable relay should work fine. See the boot mode control section for BBB for more information.

Control of the warm reset SYS_RESET is required for UART bootmode. It takes time for the host to notice the new USB devices and enumerate them and for the host to start a download procedure on the new serial port. By this time the board has moved on to other boot modes. To ensure reliable UART bootmode it is required to power up the board, wait for device enumeration, place the board in reset, start the host download procedure and then release reset.

Alternative: Use LAVA SD-MUX

This alternative is similar to the description for BBB. However, since the original BeagleBone does not contain an eMMC, this configuration is simpler. No external relays or pulldowns are needed.

Reduced Function Alternatives: If you are willing to sacrifice some of the goals of this article, there are options that require less hardware for boot mode control.

The methods below are different methods to load a bootloader. The idea is move the element of control to the bootloader. One example of this would be to have the bootloader do nothing at load and wait for commands from the UART. Another example would be to have the bootloader always boot via Ethernet and use host software to change the files that it gets.

These methods require no extra hardware for boot mode control:

• put your bootloader in uSD and trust the software not to disturb it.
• use a constant pull-down on SYS_BOOT4 to force the board to always boot in ENET recovery mode.

The use of UART bootmode on the original BeagleBone will require relay control of the warm reset line for the reasons described above. These modes require only one relay per board:

• leave the uSD slot blank
• use constant pull-downs on SYS_BOOT4 & SYS_BOOT1 to force the board to always boot in UART recovery mode.

Target Software Setup: U-Boot and Linux

The normal bootloader used for Linux on BeagleBone is U-boot. The first stage loader is u-boot-spl which, for SD or eMMC boot, is contained in a file called MLO. This section describes how to control or modify U-boot for recovery mode. U-boot can be used to load other operating systems or applications than just Linux and this section can apply to any such use as well.

U-boot SPL/MLO

The default BeagleBone build of MLO/u-boot-spl from BeagleBone.org/Angstrom is smart enough to act correctly regardless of which ROM boot mode is used to load it. If MLO is loaded from MMC1/uSD card then the MMC1 will be used to load u-boot.img. Likewise for MMC0/eMMC.

If UART is used to load u-boot-spl then it will print a banner and start another X/Y Modem transfer to get the U-boot image.

If Ethernet is used to load u-boot-spl then another DHCP/TFTP sequence will be used to get the U-boot image. A different vendor class ID will be used for this second round so the DHCP server can distinguish the 2nd stage from the first and give a different filename.

The AM335x ROM bootloader requires a header for the images it loads from uSD/MMC and the MLO file has this header. For loading from UART or Ethernet, this header is not expected by the ROM and should not be present. If you are rebuilding U-boot, the SPL image without the header is found in the file spl/u-boot-spl.bin. Unfortunately, this file is not commonly distributed. However, as a quick hack, it is easy to remove the header from the MLO image using the following command:

dd if=MLO of=spl.bin bs=1 skip=520

U-boot default boot

Unfortunately, unlike U-boot SPL, U-boot upstream and as built by Beagleboard.org/Angstrom (as of May 2013) does not change its behavior based on the bootmode used to load it. To deal with this situation there are several possible solutions:

1. Send characters on the UART while U-boot is booting so that it cancels its default bootcmd
2. Patch and Rebuild U-boot to do nothing at initial boot
3. Patch and Rebuild U-boot to start a new sequence of TFTP loads
4. Patch and Rebuild U-boot to perform default boot actions appropriate for the boot mode used to load it

Method 1 is pretty easy to deal with for UART bootmode as we will write a script to do the whole sequence anyway and it requires no patching or rebuilding. Method 1 can be used for Ethernet booting as well but does not gain a lot for the extra complication.

Method 2 would be the ideal mode for U-boot that was loaded via UART. Since UART is used to load it is natural to then instruct U-boot, command by command, what to do using the UART. Method 2 could be done for Ethernet also but requires two points of control: one at the TFTP directory and another via the UART. Again there is very little gain for the extra complication.

Method 3 would be the ideal mode for U-boot that was loaded via Ethernet. A 3rd round of DHCP and TFTP could be used to transfer a uEnv.txt file and that file could define which exact files were to be transfered and what command line options were to be used to complete the boot process. This allows the lab to control everything by altering the files in a board specific TFTP directory.

Method 4 would be the ideal patch to enhance U-boot so that one U-boot binary could be distributed for all boot modes. The creation of such a patch will be out of scope of this article but in the author's opinion should be a longer term goal. U-boot SPL does appear to pass on the information about boot mode to the U-boot image. This information could be used to set a bootdev environment variable and the default bootcmds altered to use this var.