MemoryProtectionOnKeystoneDevices

		Wiki Note
MemoryProtectionOnKeystoneDevices
Author(s):Matti Rintamäki		Telecom Infrastructure Field Applications
SPRWIKI9012		April 2011
Abstract
This document describes how to configure memory protection for catching SW issues on TMS320C66x (KeyStone) devices. Memory protection on KeyStone devices is implemented in a distributed manner, so that each subsystem can implement their own memory protection features. For the same reason, the documentation on memory protection is also distributed across the documentation of the various subsystems. This application note aims to collect the documentation together to make it easier for the developers to implement memory protection features to their application SW. While the examples are given for the TMS320TCI6616 device, the ideas are applicable to other KeyStone devices as well. An example application is accompanied, showing how to configure the various memory protection features, route the corresponding events as exceptions to alert the system of violations, and finally showing how the memory protection works in practice by doing a series of offensive accesses by various masters in the system.

Purpose of memory protection

Early detection of unintended memory accesses is vital in SW development on today’s complex system on chip (SoC) devices such as the TMS320TCI6616 with four DSP cores, several master DMA ports and HW accelerators (see TMS320TCI6616 Communications Infrastructure Digital Signal Processor Data Sheet (SPRS624)). Such memory accesses arise from programming errors such as uninitialized pointers, uncoordinated accesses to shared resources, peripheral configuration errors, etc. The symptoms of such programming errors can be anything from no issues to fatal errors leading to system crash. They may be very hard to find as there can be a long time between the point where an erroneous access is made to the moment when it produces detectable consequences. Memory protection in HW allows for detecting such unintended accesses at the moment when they occur, and provides means to block the access and notify the system.

Memory protection provides many benefits to a system. Memory protection functionality can:

• Protect operating system data structures from poorly behaving code.
• Aid in debugging by providing greater information about illegal memory accesses.
• Allow the operating system to enforce clearly defined boundaries between supervisor and user mode accesses, leading to greater system robustness.

Of course this behavior doesn’t come for free. You’ll have to design the SW and configure the memory protection of various modules appropriately to make the system less susceptible for fails. These considerations we’ll discuss in the following chapters.

Figure 1 - TMS320TCI6616 Functional Block Diagram

Memory Protection Documentation

As we will observe in the next chapter, the memory protection architecture is distributed. So is the documentation. Here is a list of documents to look for memory protection documentation:

• SPRUGW0 – C66x CorePac reference guide
  • General memory protection architecture description
  • Descriptions of memory protection features of each of the CorePac internal modules
• SPRUGW5 – C66x Memory Protection Unit User’s Guide
  • Description and usage of the system-level memory protection units
• Device datasheet
  • Description of the implementation of various memory protection features on the particular device
• Module User’s Guides
  • The module user’s guides describe memory protection features for modules (such as EDMA3) that implement them.

Memory Protection Architecture

Memory protection on the KeyStone architecture is implemented in a distributed manner. Each module that provides memory protection features implements a set of commonly defined registers for setting the permissions and capturing violations. In addition, the CorePac implements some registers for capturing bus access errors. Access violations also trigger system events that can be routed as interrupts or exceptions to the CorePacs. In this chapter we look at the memory protection architecture in detail.

Privilege Levels

Code running on the DSP executes in one of two privilege modes: supervisor mode or user mode. The default after reset is supervisor mode. The mode can be changed via a SW exception procedure that is documented in the TMS320C66x DSP CPU and Instruction Set Reference Guide (literature number SPRUGH7) and not described here. Supervisor code is considered more trusted than user code. Examples of supervisor threads include operating system kernels and hardware device drivers. Examples of user threads include vocoders and end applications. Supervisor mode is generally granted access to peripheral registers and the memory protection configuration. User mode is generally confined to the memory spaces that the operating system (OS) specifically designates for its use.

DSP accesses as well as internal and external DMA accesses have a privilege level associated with them. The DSP privilege level is determined as described above. The Internal DMA accesses that are initiated by the DSP inherit the DSP's privilege level at the time they are initiated. The privilege level for external DMA accesses is device-dependent and described in the device datasheet. The privilege level assignment of the TMS320TCI6616 device is provided in Table 1 as an example.

Table 1. - Device master privilege level settings on the TMS320TCI6616

Privilege ID	Master	Privilege Level
0	CorePac 0	SW dependent
1	CorePac 1	SW dependent
2	CorePac 2	SW dependent
3	CorePac 3	SW dependent
4	AIF	User
5	TAC	User
6	RAC	User
7	FFTC	User
8	PA_SS	User
9	SRIO	User / Driven by SRIO block. User mode and supervisor mode is determined by per transaction basis. Only the transaction with source ID matching the value in SupervisorID register is granted supervisor mode
10	Multicore Navigator	User
11	PCIe	Supervisor
12	Debug Access Point	Driven by debug subsystem
13	Reserved	Supervisor
14	Reserved	Supervisor
15	Reserved	User

Privilege ID

Each requestor on a KeyStone device (CorePacs, SRIO, EMAC, etc.) has an ID for privilege purposes. This ID accompanies all memory accesses made on behalf of that requestor. The privilege IDs on the TMS320TCI6616 device are shown in Table 1.

The privilege IDs of the device masters are mapped to the Allowed IDs in the MPPA registers either directly (as is the case with the MPU units on the TMS320TCI6616 device), or via a mapping table. In the case of CorePac local memory protection settings, the CorePac implements a set of mapping registers PAMAP0...PAMAP15 that map the Privilege IDs of the system into a smaller set of Allowed IDs that the CorePac understands (see SPRUGW0).

Memory Protection Pages and Page Attribute Registers

For memory protection purposes, memory is divided into pages. The page sizes on the TMS320TCI6616 device are shown in Table 2.

Table 2. - Memory Protection Page Sizes on the TMS320TCI6616 Device.

Memory	Number of Pages	Page Size
L1P	16	2 kB
L1D	16	2 kB
L2	32	32 kB
Shared L2	Configurable via MPAX	Configurable via MPAX
DDR3	Configurable via MPAX	Configurable via MPAX
MPU-protected peripherals (CORE/3 CFG TeraNet, PacketDMA config and data ports, Semaphore and RAC)	Configurable via MPU	Configurable via MPU

For each page, there is a corresponding Memory Protection Page Attribute Register (MPPA) of the form described in Figure 2 and Table 3. Note that not all of the bits might be implemented on a particular module. For example, the C66x CorePac only implements AID0-AID5 (and AIDX which covers the rest of the Allowed IDs) of the allowed ID bits. The meaning of each individual bit is that when set, access is granted, and when cleared, access is denied.

Figure 2 - Memory Protection Page Attribute (MPPA) Register.

Table 3. - Meanings of the bits in the MPPA register.

Bit Field	Meaning When Set	Meaning When Cleared
AIDn (n=0…5)	Allow accesses from ID = n	Deny accesses from ID = n
AIDx	Allow accesses from ID > 5	Deny accesses from ID > 5
LOCAL	Allow access from CPU to its local memories (L1/L2 only)	Deny access from CPU to its local memories (L1/L2 only)
SR	Allow supervisor reads	Deny supervisor reads
SW	Allow supervisor writes	Deny supervisor writes
SX	Allow supervisor execute	Deny supervisor execute
SR	Allow user reads	Deny user reads
SW	Allow user writes	Deny user writes
SX	Allow user execute	Deny user execute

Memory Protection Fault Registers

Each module that supports memory protection implements the following registers:

• Memory protection fault address register (MPFAR)
  • Captures the address of the access causing the fault
• Memory protection fault status register (MPFSR)
  • Captures the type of access (User/Supervisor Read/Write/Execute) and the privilege ID of the master that performed the access
• Memory protection fault command register (MPFCR)
  • Used to clear a previous fault so that subsequent faults can be detected

SPRUGW0 provides more details on these registers.

What is an invalid access?

Invalid accesses include:

• Offending access to a memory page that is protected by an MPPA register or MPAX memory protection settings
• Access to unimplemented memory space (for example, some reserved area in the memory map)
• Write access to a read-only register, or read access to a write-only register
• Write access to a register that is locked from write accesses.

Actions taken on access violation

Hardware memory protection performs the following when an access violation is detected:

• The access is denied (reads return garbage, writes are blocked)
• Faulting address is captured in the MPFAR register
• Type of faulting access is captured in the MPFSR register
• Event is generated to alert the system on the violation

The software should perform the following to handle the violations:

• At initialization, route the memory protection events that you want to monitor as interrupts / exceptions to the CorePac
  • Some events are directly routed to CorePac interrupt controller (INTC), some events are routed via the system-level interrupt controller INTC0
• In the interrupt service routine (ISR):
  • Read the MPFAR and MPFSR registers to see the reason for the violation
  • Write 0x1 to the MPFCR register to clear the fault (required so that subsequent faults can be reported)

Example use cases for MPPA settings

Using the MPPAs of the CorePac local memories as an example, some example use cases and corresponding MPPA settings are given in Table 4.

Table 4. - Example use cases and MPPA register settings.

Register	Protected Memory range	Address	Value	Meaning of the setting	Application uses this page for...
L1DMPPA16-L1DMPPA31	0x00F00000 – 0x00F07FFF	0x0184AE40 - 0x0184AE7C	0x00000000	Deny all accesses	Cache (32 kB)
L1PMPPA20	0x00E02000 – 0x00E027FF	0x0184A650	0x00000108	Allow only local DSP core execute in supervisor mode, deny all other accesses	Critical operating system routine
L2MPPA10	0x00850000 – 0x00857FFF	0x0184A228	0x00007D06	Allow user read+write for local DSP core and AID0…3 (assumed that Core 0...3 are mapped to AID0…3, and Multicore Navigator to AID4), deny all other accesses	Buffers for inter-core communication by EDMA3 and Multicore Navigator, accessed by the local DSP by user mode tasks.
L2MPPA11	0x00858000 – 0x0085FFFF	0x0184A22C	0x00000109	Allow only local DSP core execute in user and supervisor mode, deny all other accesses	User code and OS kernels
L2MPPA24-L2MPPA31	0x008C0000 – 0x008FFFFF	0x0184A260 - 0x0184A27C	0x00000000	Deny all accesses	Cache (256 kB)

Memory Protection Coverage on the TMS320TCI6616

In this chapter we go through each module that contains memory protection features on the TMS320TCI6616 device. Due to the distributed nature of the memory protection architecture, it is possible that an invalid memory access is protected by multiple modules and thus also causes multiple system events to get asserted on the same access. This chapter attempts to collect together the information on which modules provide protection support for which memory ranges, which system events get generated, and which registers are used to capture information on the faulting accesses. The events and fault reporting are summarized at the end of the chapter.

C66x CorePac

Local memory space (Addresses 0x00000000-0x00FFFFFF)

The local memory space consists of the L1P, L1D and local L2 memories, and reserved areas.

In general, the local memory controllers report violations that are directly served by that controller. For example, if a data read by the local DSP core is made to local L2 memory and the access results in a violation (due to the L2 MPPA settings), the violation is reported by L1D controller (and not L2 controller), because the L1D controller will be the one finally servicing the read and returning it to the DSP core. On the other hand, if the access was a write instead of a read, then the violation would be reported by the L2 controller as it is the one servicing the write.

Protection on accesses to reserved areas is discussed in section 4.1.5.

The L1P, L1D and L2 controllers have two output events each; one for local DSP core access faults and one for DMA access faults. These events (120-125) are input to the CorePac interrupt controller. Also, each of the controllers implements its own memory protection registers (MPPAn, MPFAR, MPFSR, MPFCR), which are documented in SPRUGW0.

As an additional level of protection, the L1P, L1D and L2 controllers implement a simple locking mechanism to protect against disallowed accesses to the memory protection registers themselves. This is also documented in SPRUGW0.

Internal configuration space (Addresses 0x01000000-0x01BFFFFF)

The internal configuration space contains the memory mapped configuration registers of entities internal to the CorePac (e.g., IDMA, local memory controllers, interrupt controller, etc.).

There are two events associated with access faults in this memory space. These events (119 and 126) are input to the CorePac interrupt controller.

Event 119 captures access faults from the interrupt controller module itself and the Power and Sleep Controller (PSC) module (mainly within address range 0x01800000-0x0180FFFF). There are no access fault reporting registers associated with that event.

Event 126 captures access faults to the rest of the internal configuration space (mainly within address range 0x01810000-0x0185FFFF). Invalid writes are trapped in the internal configuration memory protection fault registers described in Table 5. Invalid reads are reported by the initiator (L1P for program fetch accesses, L1D for data reads) in their corresponding memory protection fault registers.

Table 5: Internal Configuration Space Memory Protection Fault Registers.

Register	Address	Description
ICFG MPFAR	0x01820300	Memory Protection Fault Address Register
ICFG MPFSR	0x01820304	Memory Protection Fault Status Register
ICFG MPFCR	0x01820308	Memory Protection Fault Command Register

External configuration space (Addresses 0x01C00000-0x0BFFFFFF)

The external configuration space is divided into two parts:

The first part is address range 0x01C00000-0x07FFFFFF, and contains the memory mapped configuration registers of modules external to the CorePac (e.g., EDMA3, SRIO, Multicore Navigator, etc.). Accesses to this space are handled by the CFG port of the EMC controller of the CorePac (see SPRUGW0). The EMC does not implement the MPFAR, MPFSR, and MPFCR registers. Instead, access violations are trapped in the ECFGERR register (see SPRUGW0).

The second part is address range 0x08000000-0x0BFFFFFF. These addresses are presented to the XMC controller, and contains accesses to memory mapped configuration registers (including those of the XMC itself). The XMC and its Memory Protection and Address Extension Unit (MPAX) are discussed further in section 4.1.4.

Note that accesses to the range 0x00000000-0x07FFFFFF are decoded by the CorePac itself, and are not presented to the MPAX unit of the XMC controller. Furthemore, accesses to the range 0x08000000-0x0BFFFFFF are considered accesses to memory mapped control registers. The MPAX unit does not modify these addresses, nor does it perform a segment-based protection check for accesses in this range, regardless of whether a segment overlaps this range. The MPAX unit is further discussed in section 4.1.4.1

External Master DMA (MDMA) space (Addresses 0x0C000000-0xFFFFFFFF)

This memory space includes accesses to the shared L2 memory as well as external memory via the Multicore Shared Memory Controller (MSMC). Accesses to this space are handled in the CorePac by the XMC controller. Memory protection and address extension are provided together in a unit called MPAX, which is contained in the XMC. The MPAX unit thus provides memory protection for the shared L2 memory and external memory.

Memory Protection and Address Extension Unit (MPAX)

The MPAX defines 16 segments of runtime-selectable size that project C66x CorePac’s 32-bit address space into a larger 36-bit address space. In addition, each segment has a corresponding set of permissions (Supervisor/User Read/Write/Execute) to control accesses to that segment. The segment sizes can be individually configured and can range from 4 kB to 4 GB per segment. The segments can also overlap each other. When there are multiple segments matching the incoming logical address, the highest numbered segment takes precedence over the lower numbered segments. See SPRUGW0 for a detailed description of the MPAX unit.

The MPAX unit will report memory protection violations on accesses to the programmed segments based on the access privilege settings in the PROT field of the XMPAXLn segment registers. If the incoming logical address does not match any of the segments (but is within the MDMA space), the XMC controller will consider this as an access with no permissions and report this back to the L2 controller, which will flag it as the MDMAERREVT (CorePac interrupt controller input event #110) and report it in the MDMAERR register (see SPRUGW0).

Memory Protection registers of the XMC

The XMC does not implement MPPA registers. Instead, the MPAX unit segment registers contain an 8-bit permission field similar to the 8 LSBs of the MPPA register (see Figure 2).

The XMC controller performs specific checks on accesses on its own configuration registers. This is not done as part of the MPAX segment checks. Violations are captured in the XMC memory protection fault registers and reported back to L2 controller, which flags them by asserting CorePac interrupt controller input event #110.

The XMC controller implements its own memory protection fault registers (XMPFAR, XMPFSR, XMPFCR), which are documented in SPRUGW0.

Operation when an access does not match any MPAX segment

In the case when the accessed address does not match any of the programmed segments, the XMC considers the access as not allowed. In this case, the XMC considers this as an access with no permissions and report this back to the L2 controller, which will flag it as the MDMAERREVT (CorePac interrupt controller input event #110) and report it in the MDMAERR register (see SPRUGW0]).

Operation when an access matches multiple MPAX segments

In the case when the accessed address matches two or more programmed segments, the XMC will assume the permission settings of the highest numbered segment and ignores the permission settings of the lower numbered segments.

Reserved areas

Accesses to the reserved areas in the memory map generate memory protection events by virtue of the area not being implemented. Note, however, that this behavior is not guaranteed, as there could be parts of the reserved area implemented for use, e.g., in internal testing.

Multi-Core Shared Memory Controller (MSMC)

The MSMC controller is described in detail in SPRUGW7. A brief introduction is given here, focus being on the memory protection features.

The MSMC handles accesses from all system masters and CorePacs to its own configuration registers, the shared L2 memory and the external memory. It has dedicated ports for the CorePacs, and two additional ports called the SMS and SES.

The SMS port handles accesses from system masters (other than CorePacs) to the shared L2 logical memory space and the MSMC configuration registers. Any accesses through the SMS port that does not address either shared L2 or MSMC configuration registers will result in an addressing error returned to the requesting master.

The SES port handles accesses from system masters (other than CorePacs) to the external logical memory space and the EMIF memory mapped registers. Any accesses through the SES port that does not address either the external memory logical space or the EMIF memory mapped registers result in an addressing error returned to the requesting master.

Both system slave interfaces (SES and SMS) feature a Memory Protection and Address Extension (MPAX) unit similar to the MPAX unit inside the C66x CorePac (see section 4.1.4.1) that combines these functions into a single process. This allows the creation of a programmer’s view of the memory that is consistent between the C66x CorePac and other system masters (particularly DMA masters).

Segment-based protection

The MPAX units of the SMS and SES ports contain registers for programming 8 segments for each of the 16 privilege IDs. Each of the segments can be programmed with allowed access types (User/Supervisor Read/Write/Execute). Violations will trigger system event MSMC_mpf_errorn, where n is the privilege ID of the requestor making the offending access. The mapping of these events is described in the device datasheet. Addresses that match multiple segments, or not matching any segments while being within the range controlled by the MPAX units, are treated the same way as described in section 4.1.4.1, except that the event raised is the MSMC_mpf_errorn, and the fault is reported by MSMC. The MSMC implements its own memory protection fault registers SMPFAR and SMPFSR, which are documented in SPRUGW7. It also implements a SMPFXR (Shared Memory Protection Fault Extension Register). The NM bit (bit 0) of that register is set when the fault is caused by the address not matching any of the segments.

Protection of the MSMC registers

Accesses to the segment registers through system masters is automatically protected as these registers are addressed by Privilege ID such that each Privilege ID has access to only its set of eight pairs of segment registers. Accesses to MPAX segment registers with a mismatched Privilege ID will generate a protection error.

In addition, the MSMC features a simple lock mechanism to protect against runaway pointer writes to its registers. This lock mechanism is described in SPRUGW7.

Operation when an access does not match any segment

In the case when the accessed address does not match any of the programmed segments, the MSMC considers this as an access with no permissions and raise the MSMC_mpf_errorn event. The faulting address and status are reported in MSMC’s SMPFAR and SMPFSR registers. In addition, The NM bit (bit 0) of the SMPFXR register is set to indicate that the fault is caused by the address not matching any of the segments.

Operation when an access matches multiple segments

In the case when the accessed address matches two or more programmed segments, the XMC will assume the permission settings of the highest numbered segment and ignores the permission settings of the lower numbered segments.

System-Level Memory Protection Units (MPUs)

On the TMS320TCI6616 device there are five system-level MPUs that protect the address ranges as described in Table 6. Each MPU contains a number of programmable-size ranges (as described in Table 6) within the memory range that the particular MPU is protecting. Each range has its corresponding MPPA register similar to that shown in Figure 2. For example, accesses directed to the main configuration crossbar (CFG SCR) memory space 0x01D00000-0x026203FF can be divided into 16 ranges with programmable sizes (1 kB granularity) and each range can be given access privileges according to the associated MPPA register.

The MPUs implement their own memory protection fault registers FLTADDRR, FLTSTAT and FLTCLR.

The MPU is described in detail in SPRUGW5, and the device specific parts are documented in the device datasheet.

Table 6: MPU memory regions on the TMS320TCI6616.

	Memory Protection	Start Address	End Address	Number of programmable ranges	Range granularity
MPU0	Main CFG SCR	0x01D00000	0x026203FF	16	1 kB
MPU1	Multicore Navigator Data Port	0x34000000	0x340BFFFF	5	1 kB
MPU2	Multicore Navigator CFG Port	0x02A00000	0x02ABFFFF	16	1 kB
MPU3	Semaphore	0x02640000	0x026407FF	1	1 kB
MPU4	RAC	0x01F80000	0x0215FFFF	2	1 kB

Note on the protection checks

The MPUs treat the protection checks somewhat differently than other memory protection modules (such as the CorePac local memory protection and MPAX units). The MPU units assume accesses to be allowed by default, if not specifically denied by the MPPA settings. In particular, the MPU first checks the transfer’s privilege ID against the AID bit settings of the MPPA registers. If the AID bit corresponding to the privilege ID is 0, then the range will not be checked (and access is allowed). This is different to, for example, the CorePac local memory protection operation where the access is denied if the AID bit corresponding to the privilege ID is 0. See also the following two subsections for further differences.

Operation when an access does not match any range

In the case when the accessed address does not match any of the programmed ranges, the MPU considers the access as allowed. Note that this is different from the MPAX units, which consider access to a non-matching segment as violations (see sections 4.1.4.3 and 4.2.3).

Operation when an access matches multiple ranges

In the case when the accessed address matches two or more ranges, the MPU assumes the privilege rights of each type to be the lowest among the matching range. Therefore, in order for the access to be allowed, all the matching ranges need to allow it. Note that this is different from the MPAX units; when an access matches multiple segments, the MPAX units assumes the privilege rights of the highest priority (highest-numbered) segment register among the group of matching segments (see sections 4.1.4.4 and 4.2.4).

EDMA3

The EDMA3 module implements two kinds of memory protection: active and proxied. This is documented in SPRUGS5. A brief overview is given here.

Active memory protection covers protection of invalid accesses to the EDMA3 channel controller configuration registers and parameter RAM (PaRAM). For this purpose, EDMA3 implements nine MPPA registers, one to protect accesses to the global region registers and one each to protect accesses to each of the eight shadow regions (registers and PaRAM sets). EDMA3 also implements MPFAR, MPFSR and MPFCR registers for fault reporting.

Proxied memory protection means that each EDMA3 transfer inherits the privilege level and privilege ID of the master that programs the transfer. These values will travel along with the transfer and will be used for potential memory protection checks at various modules within the path of the transfer.

Summary of modules that support memory protection

L1P

The L1P controller monitors:

• DSP core instruction fetches from L1P and L2
• Accesses to L1P memory protection and cache control registers
• DMA accesses to L1P RAM

There are two interrupt event outputs: DSP core violation and DMA violation. These are routed to CorePac interrupt controller input events 120 and 121, respectively.

L1D

The L1D controller monitors:

• DSP core data accesses that are directly serviced by L1D (includes reads from L2)
• Accesses to L1D memory protection and cache control registers
• DMA accesses to L1D RAM

There are two interrupt event outputs: DSP core violation and DMA violation. These are routed to CorePac interrupt controller input events 122 and 123, respectively.

L2

The L2 controller monitors:

• DSP core data accesses that are directly serviced by L2 (e.g. writes to L2)
• Accesses to L2 memory protection and cache control registers
• DMA accesses to L2 RAM
• MDMA Bus Transaction Errors

There are three interrupt event outputs: DSP core violation, DMA violation, and MDMA error. These are routed to CorePac interrupt controller input events 124, 125 and 110, respectively.

EMC Controller

The EMC controller monitors transaction errors that occur in the Configuration Bus when accessing global configuration space from the CorePac DSP core. Bus errors trigger an event that is routed to CorePac interrupt controller input event 127.

CorePac Internal Configuration System

The CorePac monitors transaction errors in accesses to its internal configuration registers. Two events capture access faults. One captures access faults to the configuration registers of the Interrupt Controller and Power and Sleep Controller (PSC) modules, and the other captures access faults to other internal configuration registers. These are routed to CorePac interrupt controller input events 119 and 126, respectively.

XMC Controller

The XMC controller monitors:

• Accesses to XMC configuration registers
• Accesses to the segments programmed with the Memory Protection and Address eXtension Unit (MPAX)

MSMC Controller

The MSMC controller has the following monitoring capabilities:

• MPAX units at SMS and SES control memory protection for non-CorePac system master accesses to SL2 and external memory
• Protection checks when a master tries to program some other master’s segment registers

Memory Protection Units (MPUs)

The five chip-level MPUs in the TMS320TCI6616 have the following monitoring capabilities:

• Chip-level MPUs for CFG Teranet, Multicore Navigator Data Port, Multicore Navigator Configuration port, Semaphore and RAC
• Protection check of master accesses to programmable address ranges within the slave endpoints
• The MPU generates one external interrupt that combines the following two internal interrupt signals:
  • An address error interrupt (MPU_ADDR_ERR_INT), generated when there is an addressing violation due to an access to a non-existent location in the MPU register space.
  • A protection interrupt (MPU_PROT_ERR_INT), generated when there is a protection violation of the either in the defined ranges or to the MPU registers.

EDMA3

The EDMA3 has the following monitoring capabilities:

• Protection checks on its own configuration registers
• Proxied protection: PrivID and Privilege level of the programmer are inherited by an EDMA3 transfer

Summary of memory protection fault events and reporting

Table 7 and Table 8 summarize the memory protection events and reporting registers when offending accesses are made to various locations in the memory map.

Table 7: Memory Protection Faults reported by the CorePac.

Address range	Event triggered on invalid access	Fault Capture Register	Fault Capture Register Address	Comments
0x00000000 - 0x00FFFFFF	120: core to L1P 121: DMA to L1P 122: core to L1D 123: DMA to L1D 124: core to L2 125: DMA to L2	L1PMPFAR / L1PMPFSR / L1PMPFCR, L1DMPFAR / L1DMPFSR / L1DMPFCR, L2MPFAR / L2MPFSR / L2MPFCR	0x0184A400 / 0x0184A404 / 0x0184A408, 0x0184AC00 / 0x0184AC04 / 0x0184AC08, 0x0184A000 / 0x0184A004 / 0x0184A008
0x01000000 - 0x01BFFFFF	119 (SYS_CMPA), 126 (EMC_CMPA)	ICFG MPFAR/ ICFG MPFSR/ ICFG MPFCR	0x01820300 / 0x01820304 / 0x01820308	Note: The fault reporting registers are associated with event 126. Event 119 captures faults from the interrupt controller and PSC modules. There are no fault reporting registers associated with that event.
0x01C00000 - 0x07FFFFFF	127 (EMC_BUSERR)	ECFGERR / ECFGERRCLR	0x01820408 / 0x0182040C
0x08000000 - 0x0BFFFFFF	110 (MDMAERREVT)	MDMAERR / MDMAERRCLR	0x01846020 / 0x01846024
0x0C000000 - 0xFFFFFFFF	110 (MDMAERREVT), 10 (MSMC_mpf_errorn)	MDMAERR / MDMAERRCLR, XMPFAR / XMPFSR / XMPFCR	0x01846020 / 0x01846024, 0x08000200 / 0x08000204 / 0x08000208

Table 8: Memory Protection Faults reported by other modules.

Address range	Event triggered on invalid access	Fault Capture Register	Fault Capture Register Address	Comments
0x0C000000 - 0xFFFFFFFF	INTC0: 102-109 (MSMC_mpf_error8-15) INTC0: 170-173 (MSMC_mpf_error4-7)	MSMC SMPFAR / SMPFXR / SMPFR / SMPFCR	0x0BC0001C / 0x0BC00020 / 0x0BC00024 / 0x0BC00028	MSMC memory protection faults for PrivIDs 4-15. NOTE: Fault events for PrivID 0-3 (i.e., the CorePacs) are directly routed to the CorePac interrupt controllers.
0x01D00000 - 0x026203FF	INTC0: 90 (MPU0)	MPU0 FLTADDRR / FLTSTAT / FLTCLR	0x02360300 / 0x02360304 / 0x02360308	MPU0: Main CFG SCR address space, 16 ranges. Note: covers also invalid accesses to MPU0 config registers at address range 0x02360000 - 0x023603FF
0x34000000 - 0x340BFFFF	INTC0: 92 (MPU1)	MPU1 FLTADDRR / FLTSTAT / FLTCLR	0x02368300 / 0x02368304 / 0x02368308	MPU1: QM_SS DATA PORT, 4 ranges. Note: covers also invalid accesses to MPU1 config registers at address range 0x02368000 - 0x023683FF
0x02A00000 - 0x02ABFFFF	INTC0: 94 (MPU2)	MPU2 FLTADDRR / FLTSTAT / FLTCLR	0x02370300 / 0x02370304 / 0x02370308	MPU2: QM_SS CFG PORT, 16 ranges. Note: covers also invalid accesses to MPU2 config registers at address range 0x02370000 - 0x023703FF
0x02640000 - 0x026407FF	INTC0: 96 (MPU3)	MPU3 FLTADDRR / FLTSTAT / FLTCLR	0x02378300 / 0x02378304 / 0x02378308	MPU3: Semaphore, 1 range. Note: covers also invalid accesses to MPU3 config registers at address range 0x02378000 - 0x023783FF
0x01F80000 - 0x0215FFFF	INTC0: 174 (MPU4)	MPU4 FLTADDRR / FLTSTAT / FLTCLR	0x02380300 / 0x02380304 / 0x02380308	MPU4: RAC, 2 ranges. Note: covers also invalid accesses to MPU4 config registers at address range 0x02380000 - 0x023803FF
0x02700000 - 0x02707FFF, 0x02720000 - 0x02727FFF, 0x02740000 - 0x02747FFF	INTC0: 33 (TPCC0 EDMACC_MPINT) INTC0: 1 (TPCC1 EDMACC_MPINT), INTC0: 17 (TPCC2 EDMACC_MPINT)	CC0 MPFAR / MPFSR / MPFCR CC1 MPFAR / MPFSR / MPFCR CC2 MPFAR / MPFSR / MPFCR	0x02700800 / 0x02700804 / 0x02700808 0x02720800 / 0x02700804 / 0x02700808 0x02740800 / 0x02700804 / 0x02700808	Each CC contains one MPPA per each shadow region and the global region. Protects accesses to EDMA3 channel controller registers and PaRAM sets.

Considerations for memory protection configuration

Some considerations are presented in this chapter on what should be taken into account when designing the application to leverage the memory protection features of the TMS320TCI6616 device.

Application layout

Efficient use of the memory protection features requires careful planning of the application layout. When designing the layout, one should keep in mind the memory protection page sizes and granularity and aim for keeping similar code and data areas together to ensure that the orthogonality of access rights is maximized.

Routing of events

Each memory protection violation will result in a system event being asserted. Tables 7 and 8 show the events generated by various modules. To make sure the SW is notified when a violation is made, these events need to be routed to the DSP core as an interrupt or exception. Some of the events are directly routed to the interrupt controller of the CorePacs, while other events are routed via the system level interrupt controller INTC0. Consult the device datasheet for specific routing of the events.

The DSP cores could of course poll the events instead of using interrupts/exceptions. However, due to the nature of memory protection, the DSP cores should get notified as soon as possible of memory protection violations, while the violations should not occur at all in normal SW operation. Therefore it is recommended to use interrupts or exceptions instead of polling.

Interrupts vs. Exceptions

The C66x DSP core supports interrupts as well as exceptions. The main conceptual difference between these is that interrupts are considered normal disruptions of the processing flow, while exceptions are considered serious error situations that are sometimes non-recoverable. In particular, the C66x DSP core begins the processing of exceptions as soon as it detects them, even if the pipeline of the DSP core is in a non-interruptible state. In such cases it is not safe to return back to the interrupted program flow. A comprehensive description of the C66x interrupts and exceptions is given in SPRUGH7.

Any events that correspond to faulting accesses done by the DSP core should typically be considered as fatal and routed as exceptions. Events corresponding to faulting DMA accesses could be routed as interrupts or exceptions depending of the intended use of the data, but in most cases exceptions is the right choice. The only reason to use interrupts instead is if it is known that the fault can be somehow taken care of in SW, and it is desired to always return to the original program flow after the interrupt.

Exception handler considerations

When a memory protection violation results in an exception, the DSP core immediately saves the processor context and branches to the NMI vector for exception handling as described in SPRUGH7. The exception handler should find out which events caused the exception, and then read the associated memory protection fault registers to find out details of the fault. Note that due to the distributed memory protection architecture, it may be that a particular access causes memory protection events of several modules to be asserted, and thus also several fault registers may contain useful information of the fault. It can therefore be useful to dump and clear the fault registers of all modules (instead of only the one associated with the event that caused the exception) and save them to system logs for further analysis, along with register dumps, memory dumps, and other information that might be useful for post-mortem analysis.

Example application and memory protection configuration

In this chapter we present an example application layout and memory protection configuration at various levels for maximal robustness on the TMS320TCI6616 device. The test application will then perform a number of invalid accesses to various places in the memory map to demonstrate memory protection event generation. A simple exception handler queries the various memory protection fault status registers and prints the information out via CIO. For the purpose of the example we will assume the following:

• CorePac 0 is the master for all the HW configurations
• CorePac 0 also receives all the memory protection events from shared resources
• Local internal memory layout is the same for all cores
• The PAMAPn registers of the CorePacs are set with the following mapping
  • PrivID 0…3 (CorePac0…3) maps to AID0…3, respectively
  • All other PrivIDs map to AIDX
• Inter-core communication is performed via EDMA3. EDMA3 also has access to the private data areas in shared memory
• The the following event mapping is assumed:
  • CorePac INTC input events 9, 10, 22-25, 56, 96, 97, 110, 113, 117 and 119-127 are routed as exceptions to the DSP core
  • INTC0 input (host) events are routed to INTC0 output channels as follows:

Table 9: INTC0 event routing in the example application.

System Interrupt	Host Channel	CorePac event
1 (CSL_INTC0_CPU_3_1_EDMACC_MPINT)	64	CorePac 0 input event 22
17 (CSL_INTC0_CPU_3_2_EDMACC_MPINT)	64	CorePac 0 input event 22
33 (CSL_INTC0_CPU_2_EDMACC_MPINT)	64	CorePac 0 input event 22
170 (CSL_INTC0_MSMC_MPF_ERROR4)	65	CorePac 0 input event 23
171 (CSL_INTC0_MSMC_MPF_ERROR5)	65	CorePac 0 input event 23
172 (CSL_INTC0_MSMC_MPF_ERROR6)	65	CorePac 0 input event 23
173 (CSL_INTC0_MSMC_MPF_ERROR7)	65	CorePac 0 input event 23
102 (CSL_INTC0_MSMC_MPF_ERROR8)	65	CorePac 0 input event 23
103 (CSL_INTC0_MSMC_MPF_ERROR9)	65	CorePac 0 input event 23
104 (CSL_INTC0_MSMC_MPF_ERROR10)	65	CorePac 0 input event 23
105 (CSL_INTC0_MSMC_MPF_ERROR11)	65	CorePac 0 input event 23
106 (CSL_INTC0_MSMC_MPF_ERROR12)	65	CorePac 0 input event 23
107 (CSL_INTC0_MSMC_MPF_ERROR13)	65	CorePac 0 input event 23
108 (CSL_INTC0_MSMC_MPF_ERROR14)	65	CorePac 0 input event 23
109 (CSL_INTC0_MSMC_MPF_ERROR15)	65	CorePac 0 input event 23
90 (CSL_INTC0_MPU0_INTD)	66	CorePac 0 input event 24
92 (CSL_INTC0_MPU1_INTD)	66	CorePac 0 input event 24
94 (CSL_INTC0_MPU2_INTD)	66	CorePac 0 input event 24
96 (CSL_INTC0_MPU3_INTD)	66	CorePac 0 input event 24
174 (CSL_INTC0_MPU4_INTD)	66	CorePac 0 input event 24
0 (CSL_INTC0_CPU_3_1_EDMACC_ERRINT)	67	CorePac 0 input event 25
16 (CSL_INTC0_CPU_3_2_EDMACC_ERRINT)	67	CorePac 0 input event 25
32 (CSL_INTC0_CPU_2_EDMACC_ERRINT)	67	CorePac 0 input event 25
38 (CSL_INTC0_CPU_2_TPCCINT0)	68	CorePac 0 input event 26
39 (CSL_INTC0_CPU_2_TPCCINT1)	78	CorePac 1 input event 26
40 (CSL_INTC0_CPU_2_TPCCINT2)	88	CorePac 2 input event 26
41 (CSL_INTC0_CPU_2_TPCCINT3)	98	CorePac 3 input event 26

Example Application Layout

The application will configure memory protection of the following modules:

• CorePac L1P, L1D and L2 memories
• CorePac XMC MPAX units
• MSMC MPAX units
• MPU0...MPU4 modules
• EDMA3 active memory protection

Table 10 shows the layout of the example application, along with CorePac local and MSMC memory protection configuration. The EDMA3 active memory protection is set so that the areas protected by MPPA0 are given access by CorePac0, MPPA1-->CorePac1, and so on. The MPU modules are programmed in order to allow demonstration of their protection capabilities, please refer to the example project to see their configuration values.

Table 10: Example Application Layout.

Memory	Range	Usage	Memory Protection Configuration	Comments
L1P	0x00E00000 - 0x00E07FFF	Full cache	L1PMPPA16…L1PMPPA32 = 0x00000000	Deny all accesses to protect cache
L1D	0x00F00000 - 0x00F07FFF	Full cache	L1DMPPA16…L1DMPPA32 = 0x00000000	Deny all accesses to protect cache
L2	0x00800000 - 0x0081FFFF	Platform + OS Code	L2MPPA0… L2MPPA3 = 0x00000108	Allow local supervisor execute
	0x00820000 - 0x0082FFFF	Platform + OS Data	L2MPPA4… L2MPPA5 = 0x00000103	Allow local supervisor read/write
	0x00830000 - 0x0086FFFF	Application Code	L2MPPA6… L2MPPA13 = 0x00000109	Allow local user/supervisor execute
	0x00870000 - 0x0087FFFF	Application user data	L2MPPA14…L2MPPA15 = 0x00000106	Allow local user read/write
	0x00880000 - 0x0088FFFF	Application general data	L2MPPA16…L2MPPA17 = 0x00000136	Allow local user/supervisor read/write
	0x00890000 - 0x0089FFFF	Application user scratchpad	L2MPPA18…L2MPPA19 = 0x00000106	Allow local user read/write
	0x008A0000 - 0x008AFFFF	Application general scratchpad	L2MPPA20…L2MPPA21 = 0x00000136	Allow local user/supervisor read/write
	0x008B0000 - 0x008BFFFF	Inter-core comm.	L2MPPA22…L2MPPA23 = 0x00003D30	Allow read/write from all cores (only supervisor mode)
	0x008C0000 - 0x008FFFFF	Cache (256 kB)	L2MPPA24…L2MPPA31 = 0x00000000	Deny all accesses to protect cache
Shared L2	0x0C000000 - 0x0C07FFFF	Shared code	XMPAXH1 = 0x0C000012, XMPAXL1 = 0x00C00089	XMC: 512kB segment, Allow User/Supervisor Execute MSMC: No matching segment (Note: XMPAXH0/XMPAXL0 are set to 0x0000001E/0x000000BF to map the 0x00000000..0x7FFFFFFF CorePac logical address space as such into physical address space by default. The higher-numbered XMPAX segments will override this as necessary.)
	0x0C080000 - 0x0C0FFFFF	Read-only data	XMPAXH2 = 0x0C080012, XMPAXL2 = 0x00C080A4	XMC: 512kB segment, Allow User/Supervisor Read MSMC: No matching segment
	0x0C100000 - 0x0C11FFFF	Core0 private data	XMPAXH3(core0) = 0x0C100010, XMPAXL3(core0) = 0x00C100B6 SMS_MPAXH_0_0 = 0x0C100010, SMS_MPAXL_0_0 = 0x00C100B6	XMC & MSMC: 128kB segment, Allow User/Supervisor Read/Write for core0
	0x0C100000 - 0x0C11FFFF	Core1 private data	XMPAXH3(core1) = 0x0C100010, XMPAXL3(core1) = 0x00C120B6 SMS_MPAXH_1_0 = 0x0C100010, SMS_MPAXL_1_0 = 0x00C120B6	XMC & MSMC: 128kB segment, Allow User/Supervisor Read/Write for core1
	0x0C100000 - 0x0C11FFFF	Core2 private data	XMPAXH3(core2) = 0x0C100010, XMPAXL3(core2) = 0x00C140B6 SMS_MPAXH_2_0 = 0x0C100010, SMS_MPAXL_2_0 = 0x00C140B6	XMC & MSMC: 128kB segment, Allow User/Supervisor Read/Write for core2
	0x0C100000 - 0x0C11FFFF	Core3 private data	XMPAXH3(core3) = 0x0C100010, XMPAXL3(core3) = 0x00C160B6 SMS_MPAXH_3_0 = 0x0C100010, SMS_MPAXL_3_0 = 0x00C160B6	XMC & MSMC: 128kB segment, Allow User/Supervisor Read/Write for core3
	0x0C180000 - 0x0C1FFFFF	Shared data (fast path)	XMPAXH4 = 0x0C180012, XMPAXL4 = 0x00C180B6	XMC: 512kB segment, Allow User/Supervisor Read/Write MSMC: No matching segment
	0xE0180000 - 0xE01FFFFF	Shared data (non-cached)	XMPAXH5 = 0xE0180012, XMPAXL5 = 0x00C180B6	XMC: 512kB segment, Allow User/Supervisor Read/Write MSMC: No matching segment
DDR3	0x80000000 - 0x807FFFFF	Shared code	XMPAXH6 = 0x80000016, XMPAXL6 = 0x80000089	XMC: 8MB segment, Allow User/Supervisor Execute MSMC: No matching segment
	0x80800000 - 0x80FFFFFF	Read-only data	XMPAXH7 = 0x80800016, XMPAXL7 = 0x800800A4 SES_MPAXH_[0..3]_3 = 0x80800016, SES_MPAXL_[0..3]_3 = 0x800800A4	XMC & MSMC: 8MB segment, Allow User/Supervisor Read
	0x81000000 - 0x81FFFFFF	Core0 private data	XMPAXH8(core0) = 0x81000017, XMPAXL8(core0) = 0x801000B6 SES_MPAXH_0_0 = 0x81000017, SES_MPAXL_0_0 = 0x801000B6	XMC & MSMC: 16MB segment, Allow User/Supervisor Read/Write (Core0)
	0x81000000 - 0x81FFFFFF	Core1 private data	XMPAXH8(core1) = 0x81000017, XMPAXL8(core1) = 0x802000B6 SES_MPAXH_1_0 = 0x81000017, SES_MPAXL_1_0 = 0x802000B6	XMC & MSMC: 16MB segment, Allow User/Supervisor Read/Write (Core1)
	0x81000000 - 0x81FFFFFF	Core2 private data	XMPAXH8(core2) = 0x81000017, XMPAXL8(core2) = 0x803000B6 SES_MPAXH_2_0 = 0x81000017, SES_MPAXL_2_0 = 0x803000B6	XMC & MSMC: 16MB segment, Allow User/Supervisor Read/Write (Core2)
	0x81000000 - 0x81FFFFFF	Core3 private data	XMPAXH8(core3) = 0x81000017, XMPAXL8(core3) = 0x804000B6 SES_MPAXH_3_0 = 0x81000017, SES_MPAXL_3_0 = 0x804000B6	XMC & MSMC: 16MB segment, Allow User/Supervisor Read/Write (Core3)
	0x85000000 - 0x88FFFFFF	Shared data (cacheable)	XMPAXH9 = 0x85000019, XMPAXL9 = 0x805000B6 SES_MPAXH_[0..3]_1 = 0x85000019, SES_MPAXL_[0..3]_1 = 0x805000B6	XMC & MSMC: 64MB segment, Allow User/Supervisor Read/Write
	0x89000000 - 0x89FFFFFF	Shared data (non-cacheable)	XMPAXH10 = 0x89000017, XMPAXL10 = 0x809000B6 SES_MPAXH_[0..3]_2 = 0x89000017, SES_MPAXL_[0..3]_2 = 0x809000B6	XMC & MSMC: 16MB segment, Allow User/Supervisor Read/Write

Questions and answers

This section contains frequently asked questions and answers to them.

Question - If I don’t clear (acknowledge) a memory protection fault, what happens when I get a subsequent fault?

The subsequent fault will not be captured in the memory protection fault registers, and the corresponding event is not latched.

Question - I have protected a segment in L2 memory from direct DSP core reads. But when the DSP core reads from this segment, nothing is captured in the L2MPFSR/L2MPFAR registers. Why?

Faults are reported by the module that ultimately services the access. DSP core reads are ultimately serviced by the L1D controller, therefore the fault is also reported in L1DMPFSR/L1DMPFAR. In this case the CorePac interrupt controller input event 122 (L1D_CMPA) is asserted.

Question - I made an access to a reserved area in the memory map, but I'm not getting any memory protection faults. Why?

Memory protection is not guaranteed for all addresses in the device memory map. It could be that the address you are poiting to is actually implemented for internal testing purposes, but not documented. Such an access may not generate a memory protection fault.

Question - What would cause the special cache victim write-back memory protection fault?

1. Map a location external to CorePac as read-write.
2. Allocate the line.
3. Make it dirty.
4. Change the mapping that covers the line to read only, or remove the mapping.
5. Evict the line.

Question - Consider the following situation. Before configuring any memory protection features, I make an access to address 0x80000000 (beginning of external memory) so it gets cached in both L1D and L2 caches. Then I configure XMC MPAX to deny accesses from this core to the page containing address 0x80000000. When I make a subsequent access from the same core to 0x80000000, will I get a memory protection error?

Any cache hits for data cached before the change won't fault due to cached permissions. If you dirty the line but have removed write permissions, the XMC will record a fault when the cache sends the victim. If the line gets dropped from the cache and is later refetched, XMC will apply the new permissions. To keep MPAX and the cache in sync, you should clean the cache out first, and then change MPAX. This avoids faults due to victims.

Question - On prefetching and memory protection. Consider the following situation. I have enabled pre-fetch for an address range X. Then I have configured memory protection to deny accesses to address range Z which is contained in X as follows

X
Y	Z

If I now make an access to the end of Y (which is allowed), the pre-fetch mechanism will make accesses to the beginning of Z. Will this trigger a memory protection violation?

MPAX will kill the prefetches. No fault will be recorded.

Conclusions

Memory protection offers significant benefits to application development and product deployment. It helps to shorten the development cycle by catching SW issues at an early stage, and can help to make the SW robust against programming errors, misuse, etc. It can also significantly speed up post-mortem analysis by providing information of faulting accesses (address, privilege id, etc.) The benefits offered by the memory protection HW are maximized by carefully designing the application layout with memory protection HW features in mind.

References

1. TMS320TCI6616 Communications Infrastructure KeyStone SoC Data Manual (SPRS624)
2. TMS320C66x DSP CorePac User Guide (SPRUGW0)
3. KeyStone Architecture Enhanced Direct Memory Access (EDMA3) Controller User Guide (SPRUGS5)
4. TMS320C66x DSP CPU and Instruction Set Reference Guide (SPRUGH7)
5. KeyStone Architecture Multicore Shared Memory Controller User Guide (SPRUGW7)
6. KeyStone Architecture Memory Protection Unit User Guide (SPRUGW5)

Appendix A. Example code

The Example Code can be downloaded from here:

• MemoryProtectionExample.zip

Appendix B. Abbreviations

Table B1. - Abbreviations

AID	Allowed ID
AIF	Antenna Interface
CCn	Channel Controller n
DMA	Direct Memory Access
DSP	Digital Signal Processor
EDMA3	Enhanced Direct Memory Access 3
EMAC	Ethernet Media Access Controller
EMC	External Memory Controller
FFTC	Fast Fourier Transform Coprocessor
HW	Hardware
INTC	Interrupt Controller
MDMA	Master Direct Memory Access
MPAX	Memory Protection and Address Extension
MPFAR	Memory Protection Fault Address Register
MPFCR	Memory Protection Fault Command Register
MPFSR	Memory Protection Fault Status Register
MPPA	Memory Protection Page Attribute
MPU	Memory Protection Unit
MSMC	Multi-Core Shared Memory Controller
OS	Operating System
PA_SS	Packet Accelerator Subsystem
PCIe	Peripheral Component Interconnect Express (PCI Express)
RAC	Receive Accelerator
SoC	System on Chip
SRIO	Serial Rapid IO
SW	Software
TAC	Transmit Accelerator
XMC	Extended Memory Controller