TI811x UBIFS Support

UBIFS

UBIFS may be considered as the next generation of the JFFS2 file-system.

JFFS2 file system works on top of MTD devices, but UBIFS works on top of UBI volumes and cannot operate on top of MTD devices. In other words, there are 3 subsystems involved:

• MTD subsystem, which provides uniform interface to access flash chips. MTD provides an notion of MTD devices (e.g., /dev/mtd0) which basically represents raw flash;
• UBI subsystem, which is a wear-leveling and volume management system for flash devices; UBI works on top of MTD devices and provides a notion of UBI volumes; UBI volumes are higher level entities than MTD devices and they are devoid of many unpleasant issues MTD devices have (e.g., wearing and bad blocks); see here for more information;
• UBIFS file system, which works on top of UBI volumes.

For more information on MTD, refer <http://www.linux-mtd.infradead.org/doc/general.html>

For more information on UBI, refer <http://www.linux-mtd.infradead.org/doc/ubi.html>

For more information on UBIFS, refer <http://www.linux-mtd.infradead.org/doc/ubifs.html>

UBIFS User-space tools

UBI user-space tools, as well as other MTD user-space tools, are available from the the following git repository: git://git.infradead.org/mtd-utils.git

The repository contains the following UBI tools:

   ubinfo - provides information about UBI devices and volumes found in the system;
   ubiattach - attaches MTD devices (which describe raw flash) to UBI and creates corresponding UBI devices;
   ubidetach - detaches MTD devices from UBI devices (the opposite to what ubiattach does);
   ubimkvol - creates UBI volumes on UBI devices;
   ubirmvol - removes UBI volumes from UBI devices;
   ubiupdatevol - updates UBI volumes; this tool uses the UBI volume update feature which leaves the volume in 
                  "corrupted" state if the update was interrupted; additionally, this tool may be used to wipe 
                  out UBI volumes;
   ubicrc32 - calculates CRC-32 checksum of a file with the same initial seed as UBI would use;
   ubinize - generates UBI images;
   ubiformat - formats empty flash, erases flash and preserves erase counters, flashes UBI images to MTD devices;
   mtdinfo - reports information about MTD devices found in the system.

All UBI tools support "-h" option and print sufficient usage information.

UBIFS Implementation Details

• The UBIFS file system is supported with 8-bit BCH ECC scheme by default. The 8-bit BCH ECC scheme is aligned across ROM, U-boot and Kernel.
  
• Both UBIFS and 8-bit BCH scheme are enabled in the Uboot and Kernel by default.

Configuration

Note: The following configuration is enabled in the kernel by default.

To enable UBIFS support, start the Linux Kernel Configuration tool:

$ make menuconfig 

(1)

• Select Device Drivers from the main menu.

    ...
    ...
    Power management options --->
[ ] Networking support --->
    Device Drivers --->
    ...

Select Memory Technology Device (MTD) Supportas shown here:

    ...
    ...
< > Connector - unified userspace <-> kernelspace linker  --->
<*> Memory Technology Device (MTD) support  --->
< > Parallel port support  --->
    ...

Select Enable UBI - Unsorted block images as shown here:

    ...
<*> Enable UBI - Unsorted block images  --->

(2)

• Select Device Drivers from the main menu.

    ...
    ...
    Power management options --->
[ ] Networking support --->
    Device Drivers --->
    ...

Select Memory Technology Device (MTD) Supportas shown here:

    ...
    ...
< > Connector - unified userspace <-> kernelspace linker  --->
<*> Memory Technology Device (MTD) support  --->
< > Parallel port support  --->
    ...

Select NAND Device Support as shown here:

    ...
    ...
[ ] NAND ECC Smart Media byte order
<*> NAND Device Support  --->
    ...

Deselect Verify NAND page writes from the main menu.

[ ] Verify NAND page writes
    ...
    ...

(3)

• Select File Systems from the main menu.

    ...
    ...
    Device Drivers --->
    File systems  --->
    ...

Select Miscellaneous filesystemsas shown here:

    ...
    ...
    Pseudo filesystems  --->
[*] Miscellaneous filesystems  ---> 
[*] Network File Systems  ---> 
    ...

Select UBIFS file system support' as shown here:

    ...
    ...
<*> UBIFS file system support
[ ] Extended attributes support
[ ] Advanced compression options
[ ] Enable debugging support
    ...

Compilling UBIFS Tools

The MTD and UBI user-space tools are available from the the following git repository:

git://git.infradead.org/mtd-utils.git

As of writing this wiki, the mtd-utils version is 1.4.8.

For instructions on compiling MTD-utils, refer TI811x MTD-Utils Compilation.

Creating UBIFS file system

From information on how to create a UBIFS image. refer create an UBIFS image

• mkfs.ubifs

 [root@localhost]# cd /home/<user>/mtd/install/sbin
 [root@localhost]# ./mkfs.ubifs -r <target_fs> -F -o ubifs.img -m 2048 -e 126976 -c 1580

Where:

-m 2KiB (or 2048)
The minimum I/O size of the underlying UBI and MTD devices. In our case, we are running the flash with no sub-page writes, so this is a 2KiB page.
-e 124KiB (or 126976)
Erase Block Size: UBI requires 2 minimum I/O units out of each Physical Erase Block (PEB) for overhead: 1 for maintaining erase count information, and 1 for maintaining the Volume ID information. The PEB size for the XO flash is 128KiB, so this leads to each Logical Erase Block (LEB) having 124KiB available for data.
-c 1580
The maximum size, in LEBs, of this file system. See calculation below for how this number is determined.
-r filesystem
Use the contents of the 'filesystem/' directory to generate the initial file system image.

-F

File-system free space has to be fixed up on first mount (http://www.linux-mtd.infradead.org/faq/ubifs.html#L_free_space_fixup)
-o ubifs.img
Output file. 

The output of the above command, ubifs.img is fed into the 'ubinize' program to wrap it into a UBI image.

  The images produced by mkfs.ubifs must be further fed to the ubinize tool to create a UBI image which must be put to the raw flash to be used a UBI partition. 

• Create ubinize.cfg file and write the contents into it

 
 [root@localhost]# cd /home/<user>/mtd/install/sbin
 [root@localhost]# vi ubinize.cfg

  [ubifs]               <== Section header
  mode=ubi              <== Volume mode (other option is static)
  image=ubifs.img       <== Source image
  vol_id=0              <== Volume ID in UBI image
  vol_size=192MiB       <== Volume size
  vol_type=dynamic      <== Allow for dynamic resize
  vol_name=rootfs       <== Volume name
  vol_flags=autoresize  <== Autoresize volume at first mount

[See calculations below to determine the value associated with 'vol_size']

• ubinize

 
 [root@localhost]# cd /home/<user>/mtd/install/sbin
 [root@localhost]# ./ubinize -o ubi.img -m 2048 -p 128KiB -s 512 -O 2048 ubinize.cfg

Where:
-o ubi.img
Output file
-m 2KiB (or 2048)
Minimum flash I/O size of 2KiB page
-p 128KiB

Size of the physical eraseblock of the flash this UBI image is created for

-O 2048 offset if the VID header from start of the physical eraseblock

The output of the above command, 'ubi.img' is the required image.

Calculations

Usable Size Calculation

As documented here, UBI reserves a certain amount of space for management and bad PEB handling operations. Specifically:

• 2 PEBs are used to store the UBI volume table
• 1 PEB is reserved for wear-leveling purposes;
• 1 PEB is reserved for the atomic LEB change operation;
• a % of PEBs is reserved for handling bad EBs. The default for NAND is 1%
• UBI stores the erase counter (EC) and volume ID (VID) headers at the beginning of each PEB. 1 min I/O unit is required for each of these.

To calculate the full overhead, we need the following values:

Symbol	Meaning	Value for XO test case
SP	PEB Size	128KiB
SL	LEB Size	128KiB - 2 * 2KiB = 124 KiB
P	Total number of PEBs on the MTD device	200MiB / 128KiB = 1600
B	Number of PEBs reserved for bad PEB handling	1% of P = 16
O	The overhead related to storing EC and VID headers in bytes, i.e. O = SP - SL	4KiB

UBI Overhead = (B + 4) * SP + O * (P - B - 4) 
      = (16 + 4) * 128Kib + 4 KiB * (1600 - 16 - 4)
      = 8880 KiB 
      = 69.375 PEBs (round to 69)

This leaves us with 1531 PEBs or 195968KiB available for user data.

Note that we used "-c 1580" in the above mkfs.ubifs command line to specify the maximum filesystem size, not "-c 1531" The reason for this is that mkfs.ubifs operates in terms of LEB size (124 KiB), not PEB size (128Kib). 195968KiB / 124 Kib = 1580.39 (round to 1580).

Volume size = 195968KiB (~192MiB)

Using UBIFS file system

Flashing UBIFS image to a NAND partition

We can Flash UBIFS image from either Linux Kernel or U-Boot.

Follow steps mentioned here to create an UBIFS image.

From U-Boot,

Get the UBIFS image to U-Boot from tftp or MMC/SD or UART. Lets consider an example of MMC card.

Since we copy the data to NAND, Empty/Erase the required RAM. Then, get the UBIFS image to U-Boot

 u-boot# mw.b 0x83000000 0xFF <filesystem_image_size> <=== filesystem image size is upward aligned to NAND block size,
                                                           This is required to get rid of "Empty Flash" JFFS2 during kernel boot.
 u-boot# mmc rescan
 u-boot# fatload mmc 0 0x83000000 ubi.img

Next, erase and write to the NAND.

NOTE

On flashing UBIFS image from U-Boot, make sure that ECC selected is in sync with Linux

Example

Assuming

1. NAND partition to be erased starts from "0x6C0000",
2. NAND partition of size "0xC820000" and
3. File system image size to be flashed is 0xFC0000 which is upward aligned to NAND block size

 
 u-boot# nand erase 0x6C0000 0xC820000
 u-boot# nand write 0x83000000 0x6C0000 0xFC0000

Using UBIFS image as root file system

• Set up the bootargs environment variable as below to use the UBIFS file system image present in a MTD partition:

setenv bootargs 'console=ttyO0,115200n8 noinitrd ip=off mem=256M rootwait=1 rw ubi.mtd=X,YYYY rootfstype=ubifs root=ubi0:rootfs init=/init'

Where X is the MTD partition number being used for file system and YYYY is the NAND page size. make sure that an UBI file system is flashed into this partition before passing it as a boot partition for Linux.

Assuming 9th mtd partition,

#setenv bootargs 'console=ttyO0,115200n8 noinitrd ip=off mem=256M rootwait=1 rw ubi.mtd=9,2048 rootfstype=ubifs root=ubi0:rootfs init=/init'

The value of <X> depends on memory device which holds the rootfs.

NOTE

On booting with UBIFS as rootfs, the first boot happens successfully. Before subsequent boot-ups, it is recommended to do a manual "sync" from the console. This allows UBIFS meta data properly updated on the partition. This initial sync will help later recovery.

NOTE:: 

   By default the kernel is compiled with both SPI and NAND support
   If SW6.1 is ON ubi.mtd=4,2048 should be used for NAND.
   If SW6.3 is ON ubi.mtd=4,2048 should be used for SPI.
   If both SW6.1 and SW6.3 are ON then ubi.mtd=9,2048 should be used for NAND or SPI.