RowboPERF User Guide

RowboPERF User Guide

RowboPERF User Guide - December 09, 2010

About this manual

This document describes how to install and work with rowboPerf applications, which includes:

• TI's Android FroYo DevKit release for OMAP35x, AM37x, AM35x platforms running Android.
• TI's Android GingerBread DevKit-2.x releases for OMAP35x, AM37x, AM35x platforms running Android.

This document describes how to build, install and work with rowboPerf applications, which includes:

• TI's Android GingerBread DevKit-2.1.1 release for AM335x platform running Android.

RowboPERF is the collection of TI's performance and Demo applications for Android. The document contains instructions to:

• Build the rowboPerf under rowboat source code
• Install the rowboPerf to android root file system
• Running RowboPERF application

What is RowboPerf?

RowboPerf is an android application based on Android’s Webview, javascript and native code that provides a common entry point to all the apps/benchmarks provided by TI that showcase the features and performance of a given TI part. RowboPerf’s UI was designed to provide the same look and feel as the demo provided by TI’s Linux SDK releases, originally based on Matrix GUI and QT, so that previous Linux users will find a familiar User interface for the demos provided with an Android release. However, RowboPerf is completely based on Android’s application framework and does not use QT or any of it components.

Get and Build RowboPerf

Note: This section is applicable for TI's Android GingerBread DevKit-2.1.1 release for AM335x platform running Android.

Source Location

TI provides RowboPerf sources for all the supported devices, developers can download the sources from the RowboPerf Gitorious repository.

Clone RowboPerf from Gitorious:
Clone the rowboPerf source under rowboat source directory.

  $ cd $(ROWBOAT_ROOT)
  $ git clone git://gitorious.org/rowboat/rowboperf.git

Dependency:
Android SDK need to be downloaded and installed from Android SDK on host machine to build the rowboPerf source.

• Download installer_r15-windows.exe for Windows Host Machine.
• Download android-sdk_r15-linux.tgz for Linux Host Machine.

Build RowboPerf

Android SDK path need to be exported for building rowboPerf source.

  $ export PATH=<android_sdk_path>/platform-tools/:<android_sdk_path>/tools/:$PATH

Check mm command is working in android build system.

  $ source build/envsetup.sh

To build rowboPerf under rowboat source, run below commands:

  $ cd rowboperf
  $ mm TARGET_PRODUCT=<target_name>

e.g. For AM335x EVM:

  $ mm TARGET_PRODUCT=am335xevm

For BeagleBone:

  $ mm TARGET_PRODUCT=beaglebone

This step will build the rowboPerf and install packages at

• out/target/product/<platform>/system/app, and
• out/target/product/<platform>/system/bin

Note: Below Installation section is not required as above steps build and installs RowboPerf package.

Installation

The RowboPERF application is already integrated to the pre-built file systems provided as part of Android FroYo DevKit. When building from sources, RowboPERF applications need to be integrated to the filesystem before preparing the rootfs tarball.

• Extract the Performance_Apps.tar.gz to some suitable directory

  # tar -xvzf ~/Performance_Apps.tar

• Assume <rowboat-source>/out/target/product/omap3evm/android_rootfs is the android rootfs directory, Invoke the install_rowboperf.sh script provided inside the RowboPERF package pr android_rootfs directory as argument.

  # cd ~/Performance_Apps
  # ./install_rowboperf.sh <rowboat-source>/out/target/product/omap3evm/android_rootfs

Package Contents

The directory structure of rowboPERF package is listed below

Performance_Apps
  |-- 0xbench
  |   |-- armeabi
  |   |   |-- bench_bind
  |   |   |-- bench_cascade_fcntl
  |   |   |-- bench_cascade_flock
  |   |   |-- bench_cascade_mutex
  |   |   |-- bench_chdir    
  |   |   |-- bench_close
  |   |   |-- bench_close_tcp
  |   |   |-- bench_connection
  |   |   |-- bench_dup
  |   |   |-- bench_exec
  |   |   |-- bench_exit
  |   |   |-- bench_exp
  |   |   |-- bench_fcntl
  |   |   |-- bench_fcntl_ndelay
  |   |   |-- bench_file_lock
  |   |   |-- bench_fork
  |   |   |-- bench_getenv
  |   |   |-- bench_getpeername
  |   |   |-- bench_getpid
  |   |   |-- bench_getrusage
  |   |   |-- bench_getsockname
  |   |   |-- bench_gettimeofday
  |   |   |-- bench_isatty
  |   |   |-- bench_listen
  |   |   |-- bench_localtime_r
  |   |   |-- bench_log
  |   |   |-- bench_longjmp
  |   |   |-- bench_lrand48
  |   |   |-- bench_lseek
  |   |   |-- bench_malloc
  |   |   |-- bench_memcpy
  |   |   |-- bench_memmove
  |   |   |-- bench_memrand
  |   |   |-- bench_memset
  |   |   |-- bench_mktime
  |   |   |-- bench_mmap
  |   |   |-- bench_mprotect
  |   |   |-- bench_msync
  |   |   |-- bench_munmap
  |   |   |-- bench_mutex
  |   |   |-- bench_nop
  |   |   |-- bench_open
  |   |   |-- bench_pipe
  |   |   |-- bench_poll
  |   |   |-- bench_pread
  |   |   |-- bench_pthread_create
  |   |   |-- bench_pwrite
  |   |   |-- bench_read
  |   |   |-- bench_realpath
  |   |   |-- bench_recurse
  |   |   |-- bench_select
  |   |   |-- bench_setsockopt
  |   |   |-- bench_sigaction
  |   |   |-- bench_siglongjmp
  |   |   |-- bench_signal
  |   |   |-- bench_sigprocmask
  |   |   |-- bench_socket
  |   |   |-- bench_socketpair
  |   |   |-- bench_stat
  |   |   |-- bench_strcasecmp
  |   |   |-- bench_strchr
  |   |   |-- bench_strcmp
  |   |   |-- bench_strcpy
  |   |   |-- bench_strftime
  |   |   |-- bench_strlen
  |   |   |-- bench_strtol
  |   |   |-- bench_system
  |   |   |-- bench_time
  |   |   |-- bench_times
  |   |   |-- bench_ubench_arithoh
  |   |   |-- bench_ubench_context1
  |   |   |-- bench_ubench_dhry2
  |   |   |-- bench_ubench_dhry2reg
  |   |   |-- bench_ubench_double
  |   |   |-- bench_ubench_execl
  |   |   |-- bench_ubench_float
  |   |   |-- bench_ubench_fstime
  |   |   |-- bench_ubench_hanoi
  |   |   |-- bench_ubench_int
  |   |   |-- bench_ubench_long
  |   |   |-- bench_ubench_looper
  |   |   |-- bench_ubench_pipe
  |   |   |-- bench_ubench_register
  |   |   |-- bench_ubench_short
  |   |   |-- bench_ubench_spawn
  |   |   |-- bench_ubench_syscall
  |   |   |-- bench_ubench_whetstone-double
  |   |   |-- bench_write
  |   |   |-- bench_writev
  |   `-- Benchmark-release.apk
  |-- 3D
  |   |-- libPVRShell_ChameleonMan.so
  |   |-- libPVRShell_Coverflow.so
  |   |-- libPVRShell_Shaders.so
  |   |-- libPVRShell_Vase.so
  |   |-- OGLES2ChameleonMan-release.apk
  |   |-- OGLES2Coverflow-release.apk
  |   |-- OGLES2Shaders-release.apk
  |   `-- OGLESVase-release.apk
  |-- install_rowboperf.sh
  |-- Launcher2
  |   `-- Launcher2.apk
  |-- rowboatBench
  |   `-- rowboat.benchmark.runner-debug.apk
  |-- RowboPERF
  |   |-- bin
  |   |   |-- armeabi
  |   |   |   |-- dhrystone
  |   |   |   |-- dhrystonereg
  |   |   |   |-- linpack
  |   |   |   `-- whetstone
  |   |   `-- armeabi-v7a
  |   |       |-- dhrystone
  |   |       |-- dhrystonereg
  |   |       |-- linpack
  |   |       `-- whetstone
  |   `-- RowboPERF.apk
  `-- StorageIO
      `-- StorageIO.apk

Detailed descriptions about each component are listed below

0xbench                                

0xbench is a comprehensive benchmark suite by 0xlab and it integrates a series of benchmarks for Android system.

There are currently 16 included benchmarks. Two for arithmetic performance, seven for 2D rendering, four for 3D rendering, one for Dalvik, and two native benchmarks for system performance

Arithmetic

1. Linpack

The Linpack Benchmark make use of a general numerical linear algebra operation to test the device's ability to perform floating point operations (MFLOPS). For detail please refer to Wikipedia, netlib.org, cs.cmu.edu. The original source can be found online here: Source code.

2. Scimark2

The Scimark2 Benchmark Suite includes several scientific calculation (including FTT, MonteCarlo ..etc) for testing the device's abilitity to perform floating point operations (MFLOPS). For detail please refer to nist.gov.

2D

1. Canvas Redraw

Use random color to redraw canvas repeatedly, and calculate the refresh rate. Note that the refresh rate of this benchmark should (may) be bounded by the device's vertical synchronization (vsync). (60fps for many Android phones, including Nexus One, HTC Hero, HTC Desire)

2. Draw Circle

A simple 2D animation program. Calculate the refresh rate.

3. DrawRect

Repeatedly add random colored, size rectangles on canvas.

4. DrawCircle2

Repeatedly render random colored, size circles on canvas.

5. DrawArc

Simple 2D animation.

6. DrawText

Calculate text rendering speed.

7. DrawImage

Calculate picture rendering speed.

3D

1. GL Cube

A sample program from the Android API Demo that uses OpenGL ES to render a rotating Rubik's Cube.

2. GL Teapot

Use OpenGL ES to render a rotating Utah Teapot (Wikipedia). Adapted from android-utah-teapot project (which uses source code from Android SDK sample programs and iPhone SDK sample programs).

3. NeHe Lesson08

A rotating 3D cube with textured applied and alpha blending enabled. For detail please refer to nehe.gamedev.net(blending). For source code and explanation please refer to NeHe lesson08.

4. NeHe Lesson16

A rotating 3D cube with textured applied and GLFog enabled. For detail please refer to nehe.gamedev.net(Cool look fog). For source code and explanation please refer to NeHe lesson16.

Dalvik

1. Garbage Collection

Test the performance of the garbage collection mechanism on the Dalvik VM. For detail please refer to hpl.hp.com. The original source can be found online here: GCBench.java

Native

1. LibMicro

A portable set of microbenchmarks that executes abundant system calls to measure the performance of different system and library calls (bionic). Since calling system functions directly is required, this benchmark depends on a set of executable files that needs to be deployed in advance. (Which requires root permission of your device) More information on LibMicro can be found here: Project LibMicro, OpenSolaris

2. BYTE UnixBench

A set of benchmark cases originally started development in 1983 updated by many over the years. Byte UnixBench is intended to provide indication on the performance of unix-like system. More information on Byte UnixBench can be found on Google Code.

ARM Benchmarks  

Three important benchmark applications Dhrystone, Whetstone and Linpack are included to test CPU performance

Dhrystone

The Dhrystone benchmark was designed to test performance factors important in non numeric systems programming (operating systems, compilers, wordprocessors, etc.). Some Important fetures of Dhrystone are,

• it contains no floating point operations;
• a considerable percentage of time is spent in string functions making the test very dependent upon the way such operations are performed (e.g. by in-line code, routines written in assembly language, etc.) making it susceptible to manufacturers 'tweaking' of critical routines;
• it contains hardly any tight loops so in the case of very small caches the majority of instruction accesses are will be misses; however, the situation changes radically as soon as the cache reaches a critical size and can hold the main measurement loop;
• only a small amount of global data is manipulated (as opposed to Whetstone).

Whetstone

The Whetstone benchmark was the first intentionally written to measure computer performance and was designed to simulate floating point numerical applications:

• it contains a large percentage of floating point data and instructions;
• a high percentage of execution time (approximately 50%) is spent in mathematical library functions;
• the majority of its variables are global and the test will not show up the advantages of architectures such as RISC where the large number of processor registers enhance the handling of local variables;
• Whetstone contains a number of very tight loops and the use of even fairly small instruction caches will enhance performance considerably;

Linpack

The Linpack benchmark was derived from a real application which originated as a collection of linear algebra subroutines in Fortran. As one would expect it tests floating point performance and results are presented in Mflops (millions of floating point instructions per second):

• it has a large percentage of floating point operations (note that division is not used);
• it uses no mathematical functions (in contrast to Whetstone);
• there are no global variables; operations being carried out on local variables or an array passed to subroutines as a parameter;
• it operates on a two-dimensional array and when comparing results care must be taken to ensure that the same array size was used;
• results are for single or double precision operations (which should be specified);
• a large percentage (over 70%) of the execution time is spent within a single function where even a small instruction cache can alter results considerably;

rowboatBench                        

rowboatBench is another set of performance test applications for the Android Platform. It is developed by mentor graphics and it contains the following tests

   * 2D graphics:
        1. Archs
        2. Fill Rate
        3. Circles
        4. Rectangles
        5. Bitmap Animation
        6. Text 

   * 3D graphics:
        1. Colored Cube
        2. Lighting
        3. Textures
        4. AlphaBlending
        5. glFog
        6. Reflection effect
        7. Multitexturing
        8. Utah teapot
        9. GlxGears
       10. Stars
       11. Sphere mapping
       12. Bezier surface 

   * CPU:
        1. Dhrystone
        2. Whitestone
        3. Himeno
        4. Spectral normalization 

3D Demos

Four 3D demos from Imagination Tec are included to show the 3D performance of Android system.

Chameleon Man                               

The Chameleon Man demo shows a matrix skinned character in combination with DOT3 per pixel lighting. Matrix skinning is the act of animating a vertex over time given a set (palette) of matrices and a known set of blend weights assigned to those matrices. The Chameleon Man model has 19 bones and an animation cycle of 16 frames.

Shader Library Browser                 

Demo illustrating a collection of Shader Effects including procedural texturing effects, environment based effects and complex lighting applied to a collection of procedurally generated geometry objects.

Cover Flow                                      

In this demo, the highly popular Cover Flow-style concept of browsing through a music collection is implemented at 60+ FPS rates with minimal power consumption and maximal eye-candy. This demo shows the typical mirror effects and a per-pixel specular highlight together with a variety of transition effects including a high-performance blur effect.

Vase                                                 

This demo shows a transparent vase with dynamic reflections on its metallic sections.

Storage IO

This Application measures the read and write speed of the selected mounted device.

The SD card partition which is used as the EXTERNAL STORAGE is already mounted on the FIle System.  So for measuring the SD card speed, select the device node corresponding the SD card partition from the list and run the test. Other external storage devices need to be mounted via adb shell or console before executing this test.

Limitations:

1. If not already mounted, you have to manually mount the node you want to test (via adb) because mount operations are not allowed within the app environment.

2. You have to manually mount the node, even if it is already mounted by vold. This is because when vold automounts a node the node lists as /dev/block/vold/<partition>, and the app is not able to figure out how to map this device information to the block device associated with it so it is not able to determine the mount point.

Running RowboPERF

After booting up Android, you can find the RowboPERF launcher on the home screen, left to the launcher button.

Else you can find the RowboPERF icon on the Launcher Screen

After you launch RowboPERF the resulting window looks like below

From this menu you can navigate all the RowboPERF packages described in the previous section.

Versioning

This is Version 1.0

Compiling LMBench Binaries for Android

LMBench is a command line suite that allows the user to benchmark the latency and bandwidth in a system. Although, it is written based on libraries typically found in Linux systems (glibc, rpc, etc) it is possible to compile some of the binaries used in LMBench for Android. This sections explains the steps needed to compile some of the LMBench binaries for Android.

1. Dowload and install the Android NDK from http://developer.android.com/sdk/ndk/index.html
2. Create a standalone version of the android NDK toolchain, follow the steps explained in the “Standalone Toolchain” section of <ndk install folder>/documentation.html ($NDK/build/tools/make-standalone-toolchain.sh --platform=android-14)
3. Download LMBench source from http://sourceforge.net/projects/lmbench/?source=directory and untar the sources
4. cd to <LMbBench source folder>/src
5. Edit bench.h to remove entries not supported by the Android libraries, comment out line 39 //#include <rpc/rcp.h> , and line 44 //#include <rpc/types.h>
6. Add the path of the bin folder of your Standalone NDK to your PATH environment variable, export PATH=<ndk install folder>/bin:$PATH
7. Compile LMBench's lib_timing, lib_mem, lib_stats, lib_debug, lib_sched, and getopt using the following commands (<output folder> refers to the folder where the output files will be placed, the folder must exist before compiling):
   • arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -c lib_timing.c -o <output folder>/lib_timing.o
   • arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -c lib_mem.c -o <output folder>/lib_mem.o
   • arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -c lib_stats.c -o <output folder>/lib_stats.o
   • arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -c lib_debug.c -o <output folder>/lib_debug.o
   • arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -c lib_sched.c -o <output folder>/lib_sched.o
   • arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -c getopt.c -o <output folder>/getopt.o
8. Create archive lmbench.a with the binaries compiled in the previous step by issuing the following command:
   • arm-linux-androideabi-ar src <output folder>/lmbench.a <output folder>/getopt.o <output folder>/lib_sched.o <output folder>/lib_debug.o <output folder>/lib_stats.o <output folder>/lib_mem.o <output folder>/lib_timing.o
9. Compile the desired LMBench executable with the following command:
   • arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -o <output folder>/<lmbench exe> <exe source file (bw_mem.c, lat_mem_rd.c, etc)> <output folder>/lmbench.a. For example to compile bw_mem one can use, "arm-linux-androideabi-gcc -O3 -fmessage-length=0 -mlittle-endian -march=armv7-a -mtune=cortex-a8 -o <output folder>/bw_mem bw_mem.c <output_folder>/lmbench.a"

Technical Support and Product Updates

For further information or to report any problems, contact http://e2e.ti.com "Android" or http://support.ti.com.
For community support join http://groups.google.com/group/rowboat
For IRC #rowboat on irc.freenode.net