DM816x AM389x PSP 04.01.00.06 Optimize Linux Boot Time

Introduction

Boot-time i.e. the time taken by the system to show its "availability" since the power button was pushed on; is a becoming a key differentiator in the usability vector.

The definition of availability varies across the devices. For example:

• Appearance of home screen for devices containing a display e.g. cellphone, media player
• An audible tone / LED turning on or changing color for devices without display
• Appearance of shell prompt on development systems with console.

As these examples suggest, there isn't an objective measure that can be used across. This article doesn't attempt to define boot-time. It recognizes the fact that boot-time must be measured in context of the device, its intended usage and associated user expectations.

Objectives

Specific usage of term 'optimizing' - instead of 'reducing' - sets the overall direction for this article. The reduction can be achieved by adding hacks/ quirks/ taking custom shortcuts - that are difficult to maintain across component versions. Optimizations are generic and easily maintainable.

This article describes a typical boot sequence and identifies the opportunities for optimization. It also walks through different techniques that can be used to optimize the boot time. These optimizations are illustrated with specific patches that currently apply against the latest PSP 04.00.01.13 release for Netra device.

This article is augmented with actual patches across the boot-loader to illustrate the techniques discussed here. Therefore, discussion is limited to select techniques and changes. The commit messages in the individual patches provide more detailed description. The differences between SDK and an end-product will be visible in these patches as well.

• The SDK tends to generic and inclusive. It is intended to demonstrate maximum features available on the platform and tools to leverage these features.
• An end-product is specific and exclusive. It implements only defined set of use-cases.

To provide a direction to the optimizations, these use-cases are being defined:

Use-case 1: Boot from SDHC Card

• Boot from SDHC card
• Mount the filesystem from SDHC card itself
• Shell on serial console.
• A graphical application on the LCD

Understanding the boot sequence

A typical (and simplified) boot sequence in Linux based systems is illustrated below:

NOTE

* Actual boot sequence depends upon the device used viz. NAND flash, MMC/SD Card, NOR flash etc.

• Two stage bootloading is required when the bootloader cannot fit into the internal memory of the processor.
• There are multiple inter-dependencies between loading modules and starting services. This picture is presenting a simplified view for illustration purpose.

Identify markers

As discussed above, the overall boot process involves boot-loader(s), Linux kernel and the filesystem. We must identify the markers in the boot log that can be used as delimiters for each stage of the boot process. This helps in determining the time spent in each stage.

• u-boot

  The banner containing the U-Boot version indicates the start of u-boot.

  U-Boot 2010.06 (Apr 16 2011 - 15:22:19)

• Linux kernel

  First line after this indicates the start of Linux kernel:
  Uncompressing Linux... done, booting the kernel.

• File System

  INIT: version 2.86 booting

Developers with intimate knowledge of the source code can point that some of the strings are printed much after the start (e.g. U-boot banner) or much before (e.g. filesystem marker) the actual event. While this is definitely true, these markers serve the purpose well.

Techniques for optimization

Shedding the bulk

The boot-loaders and the Linux kernel contain a default configuration that decides which components are included in the build. These default configurations become starting point for creating a product specific configuration. These configurations must be tailored to ensure that unnecessary components aren't being included in the final build.

Some of the features may be required only by developers and don't have much use in the end-product. These features can also be removed.

Here are few examples of simple decisions that can result in a tailored configuration for a product.

u-boot

• Avoid long help text for the u-boot commands.

  #undef CONFIG_SYS_LONGHELP

• Use simple parser - instead of hush

  #undef CONFIG_SYS_HUSH_PARSER

• If ETHERNET support is not required, remove it from the configuration:

  #undef CONFIG_CMD_NET

Linux kernel

The drivers and subsystems that are not required in the product can simply be left out of the kernel configuration. Ones that are not "essential" at the system boot-up can be built as insertable modules. These modules can be inserted into the kernel when they are really needed.

• If the product doesn't support any of MTD devices, the corresponding drivers and support for JFFS2 filesystem can be removed from the kernel configuration.
• Initialization of the capture driver can be delayed until it is really required.

Decision on "What is essential?" and "What is extra?" depends upon the actual product definition.

NOTE

#undef is used here to illustrate the configuration options that can be removed in u-boot. To exclude them from product specific configuration, not defining them is sufficient.

Deferred initialization

As discussed above, initcalls for some modules may take quite long delaying the boot process. A short patch had been created (against kernel version 2.6.27) to defer selected initcalls until after the system is booted - without having to explicitly define them as modules.

• Identify the modules that are not essential for the system at boot-up
• Locate the initcall definition of these modules in the source code.
• Change the declaration of the initcalls from module_init() to deferred_module_init()
• Once the system has booted, deferred initcalls can be executed by:

  $ echo 1 >/proc/deferred_initcalls

Refer to this page for details and link to the original patch.

Build for product

The default configuration of u-boot and the Linux kernel is friendly for development systems - not the end products. This includes, building with debug information, additional code for traceability, etc.

Once the development is complete and the individual components have been well tested, much of the debug infrastructure can be removed.
Here are some suggestions:

u-boot

• Remove option -g from the compiler.
• u-boot is build for armv5 for compatibility reasons. Update compiler flags to build it for armv7-a.

Linux kernel

• Remove option -g from the compiler.
• Disable Kernel debugging
• Disable Debug Filesystem
• Disable Tracers

When choosing the compiler options, a common dilemma is to optimize for speed or 'size. Here are few data points that should help make the decision faster:

• Unless executing-in-place, the executable images have to be loaded into the memory from the storage media e.g. while booting from MMC/SD. Bigger means longer time to load the image... which impacts the boot time.
• When optimizing for size, the image would be smaller but the execution would is expected to be little slower.

It is common to see the balance shift on either side during the optimization cycles.

Filesystem

A heavy filesystem can negate all the efforts in optimizing the components. Most of the filesystems are derived from the desktop based systems that may not apply to embedded systems.

IMPORTANT

Initialization scripts need to be reviewed to ensure that only necessary initializations are done.

It is quite common for the filesystems to be built for generic architectures. This may not provide best performance. For example, ARMv7 based system using filesystem built for ARMv5.

Be quiet

In the default configuration, boot-loaders and the Linux kernel print many verbose messages on the serial console. Though quite useful in the development stages, these messages are not required in the end product. Depending upon the number of prints, turning off these messages can save 1 - 2 seconds on the overall boot process.

u-boot

Turning the prints off is a 2 step process:

• Add the following line to board specific configuration:

  #define CONFIG_SILENT_CONSOLE 1

• Set the environment variable silent to 1.

  TI8168_EVM # set silent 1

  TI8168_EVM # saveenv

Linux kernel

The prints in the Linux kernel can be turned off by adding bootarg quiet to the kernel command-line.

IMPORTANT

As we have seen above, most of the data used for optimizations is available via these prints. This should, therefore, be delayed till the last iteration. If the desired boot time isn't achieved, enable the prints again and look for more opportunities.

Building sources

Build u-boot 1st stage for SD

Quick-boot enable a faster booting using the SD card images . Quick boot removed the boot delay and suppress the console message from the u-boot. It uses a separate configuration file "include/configs/ti8168_config_quick_mmc.h" for building the quick-boot images.Please follow the steps to build 1st stage quick-boot images for mmc.

$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm distclean
$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm ti8168_config_quick_mmc_min
$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm u-boot.ti

Build u-boot 2nd stage for SD

The 2nd stage quick-boot images for mmc is generated from the same quick-boot config file "include/configs/ti8168_config_quick_mmc.h". The u-boot image will load the kernel from the sd card.The env "quite" is passed by default from the u-boot . Normal messages about hardware detection at boot are suppressed ,only important and system critical kernel messages are printed to the console. It also avoid the boot delay by setting it to 0 .

$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm distclean
$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm ti8168_config_quick_mmc
$ make CROSS_COMPILE=arm-none-linux-gnueabi- ARCH=arm u-boot.ti

This will generate binaries named u-boot.noxip.bin and u-boot.bin. Use u-boot.bin for boot from mmc.

This will generate a file u-boot.min.sd which needs to be renamed to MLO.

Build Linux kernel

$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- ti8168_evm_defconfig
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- uImage

References

1. http://elinux.org/Boot_Time
2. http://elinux.org/Deferred_Initcalls
3. http://www.linux-mtd.infradead.org/doc/ubifs.html
4. http://free-electrons.com/blog/lzo-kernel-compression/
5. http://processors.wiki.ti.com/index.php/Optimize_Linux_Boot_Time