WL18xx TI Bluetooth Stack VoLE Demo App

Return to WL18xx AM335x TI Bluetooth Stack

Note: This page is currently under construction.

Demo Overview

The Voice over BLE (VoLE) demo application uses the Dual Mode CC2564X or Wilink8 devices in conjunction with the CC2650RC remote control to stream voice data, using notifications, over the BluetopiaPM GATM Module (GATT profile). This voice data is saved as a wav file, on the Linux host, where it may be analyzed for specific voice commands. The VoLE profile, defined by the VoLETyp.h header file, is a custom service that sends compressed audio over over GATT using the Bluetooth Low Energy protocol. It is not a standard Bluetooth Low Energy profile, and will only be useful if both the local and remote devices are using the custom protocol.

Note: Only the Client side of the profile is supported on the Sitara platforms.

This application allows the user to use Bluetooth Low Energy (BLE) to establish connection between two BLE devices (The CC2650RC remote and the Wilink8/CC2564x) and send raw data from the GATT server to the client.

The companion application, which runs on the CC2650RC remote control is the hid_adv_remote_cc2650rc_app and can be found as a part of the extra examples for the TI BLE-Stack 2.2.x SDK release found with other BLE Examples. The hid_adv_remote_cc2650rc_app application is responsible for using its PDM driver to take audio samples at 16 kHz, compress the audio using 4:1 ADPCM compression and send the audio samples over GATT. For details on this process, visit the CC2650RC Developers guide.

Hardware Requirements

• CC2650RC Remote
• TMDXEVM3358 or TMDSEVM437x Sitara EVM
• CC256XCQFN-EM or WL1837MODCOM8I
• Headphones or speakers to hear audio over the EVM's 3.5mm jack

Software Requirements

As this application is designed to show a full use case, there are extra software requirements, besides those mentioned in the Getting Started Guide, for the ADPCM decoding and voice recognition.

• WL18xx TI Getting Started Guide for Environment and Bluetooth stack setup.

Note: When using the CC2564x device, you must update your build environment for that platform. See Configuring the Linux Environment for more information.

• Up to date HID Adv Remote firmware for the CC2650RC remote. Instructions on performing the Over the Air Download (OAD) can be found in the CC2650RC User's Guide using the CC2650RC 1.10 (BLE) for the audio service.

• Python > 2.7 (Comes pre-installed with the Processor SDK)

• Python modules: Installed via the command line using the Python pip installer. See Installing Python Modules
  • Wave File Generation:
    • Wave Module
  • (Optional) Open source local speech recognition:
    • Pocketsphinx Module
    • SpeechRecognition Module /* Python Port for pocket sphinx */

Software Overview

This sample application (LinuxVoLE) differs from others in the BluetopiaPM structure as it has a companion python script designed to run concurrently with it. This allows for the audio data that is captured over GATT to be processed and saved using a simple python interface. This can be seen in the diagram below as the output (STDOUT) from LinuxVoLE is piped to the Python application. For implementation details on the (Optional) blocks within the diagram, see the relevant appendices.

Software components

LinuxVoLE C Application

LinuxVoLE is a C application that includes code to connect to the CC2650RC remote, receive the voice data in the form of notifications and receive incoming HID reports. The LinuxVoLE application is designed to run without the use of a terminal and as such will perform the necessary setup, such as powering on the device and scanning for the remote, automatically. LinuxVoLE builds upon the existing LinuxHOGM application in order to allow for the acceptance of HID (Human Interface Device) key press indication reports. More information on the HID Over GATT profile can be found on the following demo page HOGM Demo

See the flowchart below for details on how the LinuxVoLE thread runs.

Note: The VoLE_initialize() function powers on the device and handles registering the necessary callbacks for authentication, HID and GATT events.

Audio_frame_linux Python Application

audio_frame_linux.py is a python script which is responsible for decoding the incoming ADPCM data and saving the input to a wav file. When the correct speech recognition modules are installed, the script will attempt to decode a subset of phrases and output them to the console. This script also spawns sub processes to run the Bluetooth Daemon and LinuxVoLE application. As such, all IO is piped through the python script.

Note: : These instructions can be used to run this demo on the AM335x or AM437x Sitara Platforms.

hid_adv_remote C Application

hid_adv_remote is a C application designed to run on the CC2650RC Remote control. It is responsible for sampling an input microphone (Using the TI-RTOS Pulse Density Modulation(PDM driver) and converting the input stream to Pulse Code Modulation (PCM) data. That data is then compressed using a 4:1 ADPCM algorithm and transmitted, in the form of GATT notifications, over BLE. This process can be observed in the diagram below.

Please see the CC2650RC Getting Started Guide for more information about building and debugging the hid_adv_remote application from either IAR or CCS

See the CC2650 Voice over BLE Developers Guide for details regarding the Audio streaming encoding format and application throughput requirements.

See the Voice over Remote Control Application Note for a detailed sequence diagram of the audio sampling process.

Running the Bluetooth Code

Note: When using a CC2564x controller you will need to change the init script in order to enable BLE. You can use the following command in the AM335x device terminal

cp -f /lib/firmware/TIInit_6.12.26_ble_on.bts /lib/firmware/TIInit_6.12.26.bts

Enter to the Sample Application folder, for this example cd BluetopiaPM/bin

 root@am335x-evm:~# cd BluetopiaPM/bin/
 root@am335x-evm:~/BluetopiaPM/bin#

Start the python script by running the following command:

 root@am335x-evm:~# python audio_frame_linux.py

The following will be printed to the console.

root@am335x-evm:~/BluetopiaPM/bin_4_2_cc256x# python audio_frame_linux.py

Starting Voice Over LE Demo!
Please press the power button to start pairing.
echo 16 > /sys/class/gpio/export
sh: line 0: echo: write error: Device or resource busy
echo out > /sys/class/gpio/gpio16/value
echo 0 > /sys/class/gpio/gpio16/value
echo 1 > /sys/class/gpio/gpio16/value
echo 0 > /sys/class/gpio/gpio16/value
echo 1 > /sys/class/gpio/gpio16/value
BT COMM PORT (/dev/ttyS3): 1
Changing HCI baud rate to 3000000
Status: Executing BTS Script /lib/firmware/TIInit_6.12.26.bts
Status: BTS Script successfully executed.

If no error messages are printed to the console, then the CC2650RC remote is ready to be paired. Click the power Button on the remote device to start the pairing procedure.

Attempting to Pair With Remote Device:
 Waiting on Remote...

After approximately 5 seconds a passkey will appear on the terminal, press the corresponding keys on the remote control in order to pair with the device.

Please enter the following authentication passkey on the Remote: 514976
Attempting to Pair With Remote Device:
 Waiting on Remote...

Note: If a pairing error occurs and less than 6 numbers appear for the passkey, press any 6 digits to have a new one generated.

Note: Pressing the pair button twice sequentially will remove all previous bonding data.

The following user messages will be printed on the screen while the device completes the pairing procedure.

Configuring Remote...
 Waiting on Remote...

Configuring Remote...
 Waiting on Remote...

Finally, press the power button once again in order to establish a connection to the device and configure the link to receive audio.

Attempting to Connect With Remote Device
 Waiting on Remote...

When the “Ready to receive audio data” message appears, this means that the remote’s services have been discovered and connection parameters have been updated and is ready to transmit audio data.

Note: Remote configuration can take around 5 seconds and may disconnect after pairing in this time. If still not connected, try pressing the power button to reconnect.

Ready to receive Audio commands!
 Waiting on Remote...

Press and hold the Mic button on the remote to start streaming a voice command, releasing when complete. For example say Pause Music and a wav file will automatically be generated.

Saving new Audio file...
...DONE...

The new wav file will be saved in the directory that the python script was executed in, in this case the LinuxVoLE directory, and is of the form:

pdm_test_YYYY-MM-DD-HR-MIN-SEC_adpcm.wav

Eg. pdm_test_2016-10-03_21-48-06_adpcm.wav

If the correct speech recognition python modules have been installed, then the Sphinx speech recognition software will parse the wav file and print the command to the console as seen below. If not, the command that was spoken will simply be played back to the user through the audio port on the Sitara EVM. See the following appendix for steps on setting up speech recognition.

Processing Wav file...

Sphinx thinks you said:
PAUSE MUSIC

The supported speech recognition commands are as follows and can be changed by updating the analyzePhrase() function within the python script:

Command	Usage
Set Volume to N	Change the current volume to an arbitrary amount (1-10) where 10 is the loudest.
Pause Music	Pause the current track.
Play Music	Play the current track.
Stop Music	Stop the current track. Track must be re-loaded to being playing again.
Load Music	Load the track from the beginning.

Memory/CPU usage

In order to estimate the current CPU usage for the VoLE Demo app, the linux "top" command was used. The findings are below for the memory and CPU usage while the application is idle and when transmitting voice data.

Application	Idle Times %	Memory %
audio_frame_linux.py	1.3%	.6%
SS1BTPM	~ 0%	.2%
LinuxVoLE	~ 0%	.1%

Application	Active Times %	Memory %
audio_frame_linux.py	76.2%	.7%
SS1BTPM	8.4%	.2%
LinuxVoLE	5.5%	.1%

Appendix A: Installing Python Modules

In order to install the python modules, the pip (python module installer) executable must be present on the device.

Note: For the installation of python modules, the Sitara platform must be connected to the internet. This can be over WiFi (Wilink8) or Ethernet.

Setup:

1. Download the get-pip.py python script from the following Python page.

2. Copy the script to your target Sitara platform, where [target-root] is the home directory on the SD card and <DOWNLOADS_DIR> is where the get-pip.py script was downloaded.

sudo cp <DOWNLOADS_DIR>/get-pip.py [target-root]/home/root/BluetopiaPM/

3. Run the python script to install pip:

python get-pip.py

4. After installation is complete, install the wave module (Required for wav file generation):

 pip install wave

5. (Optional) Install the open source speech recognition modules:

pip install pocketsphinx
pip install SpeechRecognition /* Python Port for pocket sphinx */

Appendix B: Speech Recognition add-on

As an optional add-on, the pocket sphinx speech recognition library is conditionally executed in the audio_frame_linux.py python script if the correct python modules have been installed using step 5 from the following Appendix.

The wav file generated within the audio_frame_linux.py application can be immediately processed, using the processWav(filename) function for commands. This function uses the Speech Recognition API, which is a port of the pocketsphinx C API's to a python module. This API allows for both online and offline speech recognition, with the offline being used in the add-on.

Once both the pocketsphinx and SpeechRecognition modules have been installed, the application will automatically print out it's best guess for the input phrase spoken into the CC2650RC microphone.

The application uses a smaller custom dictionary (en-US-custom) in order to speed up speech processing time, see the following link on steps to build your own Building a custom language model using the lmtool.

Using the smaller and faster custom speech library

1. Open the audio_frame_linux.py script in the editor on the evm using the following command:

vi audio_frame_linux.py

2. Copy the en-US-custom library, found in the Software Requirements to the following directory on the Sitara.

cp -R <DOWNLOADS_DIR>/en-US-custom /usr/lib/python2.7/site-packages/speech_recognition/pocketsphinx-data/

3. Run the audio_frame_linux.py script as normal.

python audio_frame_linux.py

Using the original speech library

1. Open the audio_frame_linux.py script in the editor on the EVM using the following command:

vi audio_frame_linux.py

2. In the processWav(filename) function, edit the following line replacing en-US-custom with en-US for the full English library.

# A much smaller custom dictionary is used here, use "en-US-custom" for the full dictionary.
result = r.recognize_sphinx(audio, language = "en-US-custom", keyword_entries=None)

3. Run the audio_frame_linux.py script as normal.

python audio_frame_linux.py

Appendix C: Local MP3 Player add-on

An option for the python application is to use the voice commands in order to control a local stream of music. This can be accomplished by defining certain key-phrase pairs in the python script which control certain actions for an MP3 player on the Sitara platform.

By downloading, installing and building the Mpg123 open source command line MP3 player, from the following link, user's can use the voice recognition commands to control the music stream.

The default python script will automatically attempt to use the mpg123 player if it is installed and will try to play a song defined by the following variable:

# Path to generic MP3 File
mp3FilePath = "/home/root/BluetopiaPM/bin_4_2_cc256x/LinuxVoLE/JoshWoodward-Ashes-01-LetItIn.mp3"

User's can update this path to any song on the device.

References

• CC2564C Application Note

• CC256x QFN-EM User's Guide

• CC2650 Getting Started Guide

• Voice over Remote Control Application Note

• CC2650 Voice over BLE Developers Guide

Licensing Information

Pocketsphynx - BSD License

Speech recognition library - BSD License