RingIO Users Guide

Overview

SysLink provides runtime software, analysis tools, and an associated porting kit to simplify the development of embedded applications in which a host processor controls and communicates with one or more slave processors. SysLink provides control and communication paths between high-level OS (HLOS) threads and SYS/BIOS (RTOS) tasks.

The RingIO module is included as part of SysLink. It provides data streaming between different cores or within the same core on a device using a ring buffer as the transport.

This document gives a detailed description of the Ring Input/Output (RingIO) design.

Terms & Abbreviations

• RingIO: Ring Input/Output
• SR: Shared Region of memory
• HLOS: High-level OS (Linux, QNX)
• RTOS: Real-time OS (SYS/BIOS)

References

• RingIO API Reference Guide
• SysLink User Guide
• IPC User Guide

Assumptions

The RingIO design makes the following assumptions:

• The hardware provides a shared region of memory. This region is accessible directly or indirectly to the cores involved in the data transfer.
• In the case of indirect access to the memory regions ­the shared region of memory can be synchronized across processors through DMAs or some other mechanism.

For the purpose of this document, actual physically sharable memory architecture shall be used.

High Level Design

In a multiprocessor system with shared access to a memory region, an efficient mode of data transfer can be implemented, which uses a ring buffer created within the shared memory. The reader and writer of the ring buffer can be on different processors.

Figure 1. Basic architecture of system supporting RingIO transfer

The RingIO component on each processor shall provide the ability to create RingIO buffers within the memory provided by the application. The memory provided may be within a shared memory region (SR) between two processors. The RingIO buffer is accessible to reader and writer present on the two processors between which the memory is shared. The application can obtain a handle to the RingIO through a call to open it, by passing the name that was used when creating the RingIO.

RingIO provides the ability for the writer to acquire empty regions of memory within the data buffer. The contents of the acquired region are committed to memory when the data buffer is released by the writer.

RingIO provides the ability for the reader to acquire regions of memory within the data buffer with valid data within them. On releasing the acquired region, the contents of this region are marked as invalid.

The conceptual view of the RingIO is demonstrated in Figure 2. A single writer, single reader shall be supported for each RingIO buffer.

Figure 2. Conceptual view of the RingIO data buffer

RingIO supports APIs for enabling synchronous transfer of attributes with data. End of Stream (EOS), Time Stamps, Stream offset etc. are examples of such attributes and these shall be associated with offsets in the ring buffer.

Features

Generic features

• A client using RingIO is a single unit of execution. It is typically a thread or task on a given processor.
• The RingIO instance can be created between a client on the a host processor and a client on a slave processor or between two slave cores.
• Either the reader or writer can create or delete the RingIO instance.
• When creating a RingIO instance, a handle is returned. This is a special handle that can be used to delete the RingIO instance during cleanup. It is different from the handle that is returned when opening the instance and does not provide access to data.
• The RingIO instance should be created in a shared memory region which can be accessed directly by both the reader and the writer.
• Both the reader and the writer need to open the RingIO instance and get a handle. Any data access on the RingIO instance should be made using these handles.
• Each RingIO can have a single writer client and a single reader client. A RingIO handle may not be shared between multiple clients on a given core. For example, the following scenario is not permitted: One thread acquires from the RingIO, passes the buffer pointer to another thread, which then releases the buffer. This scenario is a multi-­reader/writer scenario, which is not supported.
• Each RingIO instance is associated with a unique RingIO name. This RingIO name is specified while creating and opening the RingIO.
• The RingIO client can be closed only if there is no currently acquired data or attributes. If there is any unreleased data or attributes, they must be released or cancelled before the RingIO client can be closed.
• The RingIO can be deleted only when both reader and writer clients have successfully closed their RingIO clients.
• Each RingIO instance has an associated footer area, if configured. The foot­-buffer can be configured to be of zero size if not required. If configured, the foot-­buffer is physically contiguous with the data buffer.

Cache-related information

• A cache-­maintenance flag is provided to the writer and reader clients while opening the RingIO. This flag enable the user to get the maximum performance from the system and customize it for their own use. It indicates whether cache coherence operations are to be performed for the RingIO data buffer. The flags need not be specified when opening the RingIO for the following application scenarios:
  • When both reader and writer resides on the same processor and data buffer is accessed directly using the CPU
  • If the RingIO data buffer is specified to be placed into an internal memory pool.

• Cache maintenance operations for the Control structures and Attribute buffer of a RingIO instance are taken cared of internally based on the cacheability of the SR they reside in.
• Cache coherency issues are addressed in this Cache Coherency in RingIO article

Acquiring and releasing data

• The writer/reader client can acquire data buffers of any arbitrary size. RingIO does not maintain the acquired data as separate buffers, but as the complete acquired size.
  • Each buffer received from the acquire call is guaranteed to be a contiguous data buffer.
  • However, buffers received from multiple consecutive acquire calls may not be contiguous.
  • No assumption should be made that consecutively acquired buffers are contiguous in memory.
  • The writer/reader client can acquire multiple buffers and release the size completely, or in smaller chunks of varying sizes.
• The data is released into the RingIO by specifying the size to be released. Buffer pointers are not provided to the release call.
• As long as the size to be released does not exceed the total acquired size, the data can be released in any granularity. The sequence of release calls does not need to match the acquire calls.
• Cancel: Any acquired data that is not required can be cancelled back to the RingIO through the RingIO_cancel () API.
  • The cancel call removes all acquired but un­released data from the RingIO for the calling client.
  • In case of writer, any attributes that were set within this acquired but un­released region are also removed.
  • In case of reader, any attributes that were removed within the acquired region are replaced back into the RingIO.

Writer

• The writer writes data into the RingIO data buffer by first acquiring a contiguous data buffer, writing data into the acquired buffer, and then releasing the filled up data to the RingIO.
• The behavior of acquire varies depending on the NEED_EXACT_SIZE specified while opening the writer client. The NEED_EXACT_SIZE flag indicates whether the writer always needs buffers only of a specific size, and buffers of lesser size are not acceptable.
  • NEED_EXACT_SIZE is TRUE
    • If the requested empty size is not available within the RingIO as a contiguous data buffer, error is returned.
    • If the requested empty size is not available till the end of the RingIO buffer, but is available from the top of the buffer, a wraparound occurs, and a contiguous buffer is returned from the top of data buffer.
  • NEED_EXACT_SIZE is FALSE: If the requested buffer size is not available, RingIO returns the amount of empty contiguous data buffer that is available till the end of the data buffer, with a status code indicating this.
• Five different types of notification mechanisms are supported. Details of the notification types are present in later sections.
• The writer can flush data that it has written into the RingIO in two different modes. In the case of hard-flush, all data and associated attributes present in the RingIO will be removed. In the case of soft-­flush, all data and associated attributes after the first readable attribute will be flushed, and the attribute is also removed.

Reader

• The reader reads data from the RingIO data buffer by first acquiring a contiguous data buffer, reading data from the acquired buffer, and then releasing the empty buffer to the RingIO.
• The behavior of acquire varies depending on the NEED_EXACT_SIZE specified while opening the reader client. The NEED_EXACT_SIZE flag indicates whether the reader always needs buffers only of a specific size, and buffers of lesser size are not acceptable.
  • NEED_EXACT_SIZE is TRUE
    • If the requested valid size is not available within the RingIO as a contiguous data buffer, error is returned.
    • If the requested valid size is not available till the end of the RingIO buffer, but is available from the top of the buffer, the behavior varies depending on whether a foot-­buffer has been configured.
      • If non-­zero size foot-­buffer is configured, the required amount of valid data is copied from the top of the data buffer into the foot­-buffer (assuming foot-­buffer size is sufficient). A contiguous data buffer is then returned to the user as requested. Further acquires will happen from the specific offset from the top of the buffer. If foot-­buffer size is not sufficient to return a contiguous data buffer of specified size, error is returned.
      • If foot­-buffer is not configured, error is returned in this case.
  • NEED_EXACT_SIZE is FALSE: If the requested buffer size is not available, RingIO returns the amount of valid contiguous data buffer that is available till the end of the data buffer, with a status code indicating this. Foot-­buffer is not used in this scenario.
• Five different types of notification mechanisms are supported. Details of the notification types are present in later sections.
• The reader can flush the data available from the RingIO in two different modes. In the case of hard-­flush, all data and associated attributes present in the RingIO will be removed. In the case of soft-flush, all data and associated attributes before the first readable attribute will be flushed.

Attributes

Generic information

• Attributes are used to communicate in-­band information from the writer to the reader.
• Typical attributes could be the EOS marker at the end of the stream that’s being written, or an attribute to indicate changes in the stream’s status.
• Attributes can be of two types:
  • Fixed attributes: Fixed attributes have an attribute type and an optional parameter
  • Variable attributes: Variable attributes can be provided a data buffer as payload data in addition to the attribute type and the optional parameter. The attributes are copy­-based. The information in writer­-provided buffer is copied into the attribute buffer. The size of provided buffer in variable attributes must be a multiple of 4 bytes.

Setting attributes

• If a data buffer has been previously acquired, attributes can be set by the writer at the position right before the start of the acquired buffer.
• When no data has been acquired, the writer can set attributes at the position before the start of the next buffer to be acquired.
• The writer commits attributes to the attribute buffer immediately upon setting them.

Getting attributes

• The attributes written by the writer should be “read” before the reader can read any more data after the position at which the attribute is set.
• If the reader could not read data due to presence of attributes at the current read location, an error code mentioning the presence of an attribute is returned.
• When a variable attribute is being read, a valid buffer must be provided to the getAttribute function. The attribute information is copied into this application buffer.
• Attributes are removed from the attribute buffer when the reader gets the attributes or the writer flushes valid data which will clear associated attributes.
• Fixed and variable attributes can be set and received using different APIs. In case a fixed attribute get function is called when a variable attribute is present, an error code is returned informing of the presence of the variable attribute.

Notification

The notification mechanism as well as other configuration parameters for the notification can be set by the reader or writer through an API call to set the notifier. Parameters that can be configured include the notification type, watermark (threshold in data size beyond which notifications are issued), callback function, and fixed parameter to the callback function. In general, notifications needs to be first 'enabled' by a pre-condition before it can be triggered by a change in data size. Five different types of notification are supported:

RINGIO_NOTIFICATION_NONE: No notification is required.

RINGIO_NOTIFICATION_ALWAYS:

• The notification is enabled when an attempt to acquire data by the client has failed.
• Once enabled, the notification remains enabled till:
  • For writer client, empty data size falls below the watermark.
  • For reader client, valid data size falls below the watermark.

  At this point, the notification is disabled again. Only a RingIO_release() call will disable the notification. RingIO_cancel() and RingIO_flush() will not disable the notification.

• Notifications are sent each time when the other client releases data, as long as the data size is above the watermark:
  • Empty data size for writer – This condition can be met in the functions RingIO_release() and RingIO_flush().
  • Valid data size for reader
• Cancel call does not enable or disable notification.
• Flush call does not enable or disable notification. Flush called by the reader client may cause empty data size to go above the watermark and cause a notification to be sent to the writer client.

As a design decision, it has been decided that cancel and flush call will not affect the state of notification. This is done irrespective of previous acquire call state. It is not possible to maintain the state log for previous acquire call failures.

RINGIO_NOTIFICATION_ONCE:

• The notification is enabled when an attempt to acquire data by the client has failed.
• The notification is sent when the other client releases data, when the below condition is true:
  • For writer client, empty data size is above the watermark – This condition can be met in the functions RingIO_release() and RingIO_flush().
  • For reader client, valid data size is above the watermark. As soon as the notification is sent, it is disabled.
• The notification is re-­enabled, only when the first condition is met again (acquire attempt fails).
• Cancel call does not enable or disable notification.
• The notification is disabled only once the notification is sent by the other client.
• Flush call does not enable or disable notification. Flush called by the reader client may cause empty data size to fall above the watermark and cause a notification to be sent to the writer client.

RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS:

• Notifications are sent each time when the other client releases data, as long as the data size is above the watermark:
  • Empty data size for writer. ­This condition can be met in the function RingIO_release() and RingIO_flush().
  • Valid data size for reader.
• This notification is always enabled. Unlike RINGIO_NOTIFICATION_ALWAYS, this notification does not require buffer to get full/empty or acquire to fail to get enabled.
• Cancel call does not enable or disable notification.
• Flush call does not enable or disable notification. Flush called by the reader client may cause empty data size to go above the watermark and cause a notification to be sent to the writer client.

RINGIO_NOTIFICATION_HDWRFIFO_ONCE:

• This notification type will send a notification only once when a watermark condition is satisfied and then it is disabled.
• Unlike RINGIO_NOTIFICATION_ONCE, this notification does not require buffer to get full/empty or acquire to fail to get enabled.
• The notification is sent when the other client releases data, when the below condition is true:
  • For writer client, empty data size is above the watermark. ­This condition can be met in the function RingIO_release() and RingIO_flush().
  • For reader client, valid data size is above the watermark.

  As soon as the notification is sent, it is disabled.

• The notification is re-­enabled when the data size crosses the watermark:
  • For writer client, empty data size falls below the watermark.
  • For reader client, valid data size falls below the watermark.
• Cancel call will affect the notification state. If the notification has been enabled earlier either because of a failed acquire call or a low watermark condition is satisfied, this notification will be disabled if the low watermark condition is no longer true.
  • The notification will be disabled when the data size crosses the watermark:
    • For writer client, empty data size falls above the watermark.
    • For reader client, valid data size falls above the watermark.
• Flush call will affect the notification state.
  • For writer client, the notification will be disabled when the data size goes above the watermark i.e. empty data size is greater than the watermark.
  • For reader client, the notification will be enabled when the data size falls below the watermark i.e. valid data size is lesser than the watermark.

Notification related details

Writer Client calls Acquire

The below table discusses the effect of the acquire call on the writer client's notification depending upon the notification type.

The acquire call called by the writer does not affect the reader client's notification status.

API Writer RingIO_acquire()	Watermark crossed (size falls below watermark and current acquire has passed)			Current acquire call has failed
Client Notification Type	Enable Notify	Disable Notify	Send Notify	Enable Notify	Disable Notify	Send Notify
RINGIO_NOTIFICATION_ONCE: size = emptySize	No	No	No	Yes	No	No
RINGIO_NOTIFICATION_ALWAYS: size = emptySize	No	Yes	No	Yes	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE: size = emptySize	Yes	No	No	Yes	No	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS: size = emptySize	n/a	n/a	No	n/a	n/a	No

Reader Client calls Acquire

The below table discusses the effect of the acquire call on the reader client’s notification state depending upon the notification type.

The acquire call called by the reader does not affect the writer client’s notification status.

API Reader RingIO_acquire()	Watermark crossed (size falls below watermark and current acquire has passed)			Current acquire call has failed
Client Notification Type	Enable Notify	Disable Notify	Send Notify	Enable Notify	Disable Notify	Send Notify
RINGIO_NOTIFICATION_ONCE: size = validSize	No	No	No	Yes	No	No
RINGIO_NOTIFICATION_ALWAYS: size = validSize	No	Yes	No	Yes	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE: size = validSize	Yes	No	No	Yes	No	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS: size = validSize	n/a	n/a	No	n/a	n/a	No

Writer Client calls Release

The below table discusses the effect of the release call on the reader client notification state depending upon the notification type.

If notification is sent for RINGIO_NOTIFICATION_ONCE/ RINGIO_NOTIFICATION_ HDWRFIFO_ONCE the notification is disabled for the reader client.

API Writer RingIO_release()	Notification state for the Reader client after writerRelease call.
	Watermark crossed (valid size goes above watermark and current release has passed)			Watermark not crossed (valid size remains below watermark and current release has passed)
Reader Client Notification Type	Enable Notify	Disable Notify	Send Notify (to Reader)	Enable Notify	Disable Notify	Send Notify (to Reader)
RINGIO_NOTIFICATION_ONCE: size = validSize	No	Yes	Yes	No	No	No
RINGIO_NOTIFICATION_ALWAYS: size = validSize	No	No	Yes	No	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE: size = validSize	No	Yes	Yes	No	No	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS: size = validSize	n/a	n/a	Yes	n/a	n/a	No

Reader Client calls Release

The below table discusses the effect of the release call on the writer client notification state depending upon the notification type.

If notification is sent for RINGIO_NOTIFICATION_ONCE/ RINGIO_NOTIFICATION_ HDWRFIFO_ONCE the notification is disabled for the writer client.

API Reader RingIO_release()	Notification state for the Writer client after readerRelease call.
	Watermark crossed (empty size goes above watermark and current release has passed)			Watermark not crossed (empty size remains below watermark and current release has passed)
Writer Client Notification Type	Enable Notify	Disable Notify	Send Notify (to Writer)	Enable Notify	Disable Notify	Send Notify (to Writer)
RINGIO_NOTIFICATION_ONCE: size = emptySize	No	Yes	Yes	No	No	No
RINGIO_NOTIFICATION_ALWAYS: size = emptySize	No	No	Yes	No	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE: size = emptySize	No	Yes	Yes	No	No	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS: size = emptySize	n/a	n/a	Yes	n/a	n/a	No

Writer Client calls Cancel

The below table discusses the effect of the cancel call on the writer client’s notification state depending upon the notification type.

API Writer RingIO_cancel()	Watermark crossed (size goes above watermark and notification was enabled)		Watermark crossed (size goes above watermark and notification was disabled)
Client Notification Type	Disable Notify	Send Notify	Enable Notify	Send Notify
RINGIO_NOTIFICATION_ONCE: size = emptySize	No	No	No	No
RINGIO_NOTIFICATION_ALWAYS: size = emptySize	No	No	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE: size = emptySize	Yes	No	No	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS: size = emptySize	n/a	No	n/a	No

Reader Client calls Cancel

The below table discusses the effect of the cancel call on the reader client’s notification state depending upon the notification type.

API Reader RingIO_cancel()	Watermark crossed (size falls goes above watermark and notification was enabled)		Watermark crossed (size falls goes above watermark and notification was disabled)
Client Notification Type	Disable Notify	Send Notify	Enable Notify	Send Notify
RINGIO_NOTIFICATION_ONCE: size = emptySize	No	No	No	No
RINGIO_NOTIFICATION_ALWAYS: size = emptySize	No	No	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE: size = emptySize	Yes	No	No	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS: size = emptySize	n/a	No	n/a	No

Writer Client calls Flush

The below table discusses the effect of the flush call on both the reader and writer client’s notification state depending upon the notification type.

API Writer RingIO_flush()	Notification state for the RingIO client after writerFlush call.
	Reader client			Writer client
Client Notification Type	Enable Notify	Disable Notify	Send Notify to Writer	Enable Notify	Disable Notify	Send Notify to Reader
RINGIO_NOTIFICATION_ONCE	No	No	No	No	No	No
RINGIO_NOTIFICATION_ALWAYS	No	No	No	No	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE	Yes (if validSize falls below watermark), No (if validSize stays above watermark)	No	No	No	Yes (if emptySize goes above watermark), No (if emptySize stays below watermark)	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS	n/a	n/a	No	n/a	n/a	No

Reader Client calls Flush

The below table discusses the effect of the flush call on both the reader and writer client’s notification state depending upon the notification type.

API Reader RingIO_flush()	Notification state for the RingIO client after readerFlush call.
	Reader client			Writer client
Client Notification Type	Enable Notify	Disable Notify	Send Notify to Writer	Enable Notify	Disable Notify	Send Notify to Reader
RINGIO_NOTIFICATION_ONCE	No	No	Yes (if emptySize goes above watermark), No (if emptySize is below watermark)	No	No	No
RINGIO_NOTIFICATION_ALWAYS	No	No	Yes (if emptySize is above watermark), No (if emptySize is below watermark)	No	No	No
RINGIO_NOTIFICATION_ HDWRFIFO_ONCE	Yes (if validSize falls below watermark), No (if validSize stays above watermark)	No	Yes (if emptySize goes above watermark), No (if emptySize is below watermark)	No	Yes (if emptySize goes above watermark), No (if emptySize is below watermark)	No
RINGIO_NOTIFICATION_HDWRFIFO_ALWAYS	No	No	Yes (if emptySize is above watermark), No (if emptySize is below watermark)	No	No	No

Control flow

The typical control flow of an application using the RingIO includes the following activities:

1. Create a RingIO

   A RingIO in shared memory is identified by a RingIO name. Hence, to create a RingIO, the application has to specify a RingIO name. This name should be used to open the RingIO buffer for reading or writing. At any point in time, only a fixed number of RingIO can be created. This fixed number can be modified through configuration.

   The memory to be used for creation of the RingIO buffers is specified by the application writer through a SharedRegion interface. The application specifies the SharedRegion ID to be used for allocating the buffers. The SharedRegion may be memory for a RingIO shared between two processors, or within local memory if the RingIO is for intra­-processor transfer. SharedRegion IDs for three different buffers must be provided, the data buffer memory, attribute buffer memory, and the control structure memory.

2. Get a handle to the RingIO

   To write and read data from the RingIO, a handle needs to be acquired. This can be done using the RingIO_open call. The reader and writer will get different handles using which they can read and write data to the RingIO. A set of open flags can also be provided which will inform the RingIO whether cache maintenance is needed for the data buffer and whether read/write requests have to be satisfied to the exact size (If requested size is not available, partial buffer is not returned).

3. Write into an acquired buffer

   To write data into the RingIO, an empty data buffer has to be first acquired. After this data can be written into the acquired buffer and the filled up buffer can be released back to the RingIO. Any sized data buffer can be acquired and released. Data buffer acquired using a single acquire call can also be released in parts. Similarly data buffers acquired in multiple calls can be released using a single release call. When the writer releases a buffer, any attributes set in the acquired buffer also gets released.

4. Read from an acquired buffer

   To read data from the RingIO, a filled up data buffer has to be first acquired. After this the data buffer can be used up and the empty buffer can be released back to the RingIO. Any sized data buffer can be acquired and released. Data buffer acquired using a single acquire call can also be released in parts. Similarly data buffers acquired in multiple calls can be released using a single release call.

5. Set attributes

   The writer can set attributes into the RingIO to communicate any synchronous information to the reader. Attributes can be set at the beginning (write offset 0) of the acquired buffer. If no buffer has been acquired, the attribute is set at the next write location.

6. Get attributes

   The reader can read the attributes set by the writer using the get attribute function. The attributes can be read only at the current read offset.

7. Flush

   Both the reader and writer can call the flush function in two different modes soft­-flush and hard­-flush. In hard­-flush, all valid data and associated attributes are removed from the RingIO. In soft­-flush, the functionality of flush is different depends on the caller. For the writer, flush will remove all valid data after the first readable attribute. This call removes the attribute as well. For the reader, flush will remove all valid data before the first readable attribute. The attribute is left in the stream itself. The flush call will also return the first readable attribute type and the optional parameter to the caller.

8. Close RingIO

   Closing the writer or the reader allows another writer or reader to open the RingIO again. Closing of the RingIO can be done only if no data buffers have been acquired.

9. Delete the RingIO

   This removes the RingIO instance from shared memory. Both the reader and writer will not be able to use the RingIO after it is deleted.

RingIO module

The RingIO component utilizes the shared memory between the various cores for implementing the data transfer protocol. The RingIO component uses the functionality provided by the Notify module for notification of events across the processor boundary.

On an HLOS (host-side)

The component interaction diagram indicates the interaction of RingIO with the various components within SysLink. Internally the part where it interacts directly with the shared memory is partitioned into a transport module named RingIOShm.

Figure 3. HLOS-side component interaction

On an RTOS (slave side)

The RTOS-side of the RingIO component is implemented as part of the SysLink library over the base inter­-processor communication functionality provided by IPC. Internally the part where it interacts directly with the shared memory is partitioned into a transport module named RingIOShm.

Figure 4. RTOS-side component interaction