Sitara Linux Dual Emac Mode

This page describes some of the CPSW switch hardware capabilities and how they are used by the Linux driver in dual EMAC mode.

CPSW Switch Hardware Capabilities

Adding VLAN Headers

When the CPSW is placed into VLAN_AWARE mode then all packets that enter the switch must have a VLAN header. If a VLAN header already exists then that header remains on the packet during CPSW ingress. If a packet entering the switch is untagged (no VLAN header) then a VLAN header will be added using the values in the Pn_PORT_VLAN register (where n is the port number of packet ingress). For an untagged packet entering the switch through host port 0 the values in P0_PORT_VLAN are used, for port 1 ingress P1_PORT_VLAN is used, and for port 2 ingress P1_PORT_VLAN is used. In our Linux dual emac case if you check those three registers after boot you will find that the VLAN ID for the port 0 is 0, the VLAN ID for port 1 is 1, and the VLAN ID for port 2 is 2.

Removing VLAN Headers

During the Ingress Filtering Process (14.3.2.7.2.1 from the TRM) there are 4 variables that get set for each incoming packet. These 4 variables are force_untagged_egress, reg_mcast_flood_mask, unreg_mcast_flood_mask, and vlan_member_list. These variables are used for the packet throughout the rest of the interaction with the switch (ALE Lookup Process, Egress Process, Learning Process, and Updating Process; check the sections for those processes in the TRM and you will see conditions that rely on those variables). For our purposes, I'll focus on the force_untagged_egress variable. This variable is a port mask (bit 0 for port 0, bit 1 for port 1, and bit 2 for port 2) that determines whether or not the VLAN header of a packet should get removed during the switch egress process. From the Ingress Filtering Process we can see how the force_untagged_egress variable gets set:

• if the switch is VLAN_UNAWARE
  • the variable is hardcoded to "000000" so that VLAN headers are not removed during switch egress from any port,
• if the switch is VLAN_AWARE and a VLAN Table Entry in the ALE table has a VLAN ID that matches the packet's VLAN ID
  • then the variable is set with the value from the VLAN Table Entry (section 14.3.2.7.1.7)
• if the switch is VLAN_AWARE and there is no VLAN ID match in the ALE table with a VLAN Table Entry
  • the value comes from the UNKNOWN_VLAN register (section  14.5.1.4)

Learning/Updating Process

The Learning Process and Updating Process described in sections 14.3.2.7.3.1 and 14.3.2.7.3.2 are based on the Source MAC address of the packet. These processes allow the switch to keep the ALE table up to date with which port a given MAC address should be found at. So, if a packet arrives at the switch from a device with MAC1 through port 2 then the switch makes a note in the ALE table that the device with MAC1 currently resides at port 2. If that device is then unplugged and moved to a network connected to port 1, then the next time that device sends a packet through the switch the ALE table will be updated to reflect that the device with MAC1 is now in the network connected to port 1. If the switch is in VLAN_AWARE mode then the Source MAC address as well as the VLAN ID are learned to identify future packets from that particular device from that particular VLAN. 

Lookup Process

The Lookup Process described in sections 14.3.2.7.2.2 and 14.3.2.7.2.3 is based on the Destination MAC address of the packet. The switch uses the Lookup Process to determine which ports the packet should be directed to based on the information that it has in the ALE table. So, continuing from the example above, if a packet arrives with a destination of MAC1 the lookup process will determine that the packet currently resides at port 1 (because the address was updated when the device was moved to the new network) and the packet will be sent out of port 1. If the switch is in VLAN_AWARE mode then the destination address as well as the VLAN ID of the packet is used during the lookup process. 

Directed Packets

A directed packet is a packet that is sent from the ARM host toward a specific external Ethernet port. This means that directed packets skip the ALE Lookup Process since the port of device egress is already established by the ARM host (look at the second if statement in the Ingress Filtering Process, 14.3.2.7.2.1). Directed packets still go through the Learning/Updating Process. Directed packets are also still appended with a VLAN header on switch ingress if they are untagged and the switch is currently in VLAN_AWARE mode. 

Linux Driver Background

Now a little background on how our Linux driver uses these switch functions to create the dual emac mode. 

Dual EMAC Mode

In order to have our device represent a different MAC address for each port here is how the switch and the ALE are configured:

• VLAN Aware Mode: The switch is placed into VLAN_AWARE mode in order to assign different VLAN IDs to packets entering the switch through different external Ethernet ports. The VLAN_AWARE mode also allows for a MAC address for the host port to be tied to a VLAN ID that corresponds to an external port. 
• Port VLAN IDs: Each port uses the Pn_PORT_VLAN registers (mentioned above) to assign a VLAN ID to each port. Port 0 is VLAN ID 0, Port 1 is VLAN ID 1, and Port 2 is VLAN ID 2. Any untagged packet entering the switch through a port will have a VLAN header added that matches the ingress port's VLAN ID. 
• Port 0 ALE Table Entries: As the driver comes up there is a set of default ALE Table Entries that configure the rules for the packets that are allowed to exit the switch at port 0 destined for Linux on the ARM host. These rules are how we tell the switch which packets we want to receive on the ARM host (and filter out all of the other unwanted traffic using hardware). Keep in mind that since we are in VLAN_AWARE mode every packet will be VLAN tagged inside the switch (with the VLAN ID of the port that they entered the switch through, if they were not previously VLAN tagged).   
  • Unicast/VLAN entry: MAC address d4:94:a1:8c:5e:b0 and VLAN ID 1 tied to port 0 (this is a default MAC address that can be changed)
  • Multicast/VLAN entry: MAC address ff:ff:ff:ff:ff:ff and VLAN ID 1 tied to port 0
  • Unicast/VLAN entry: MAC address d4:94:a1:8c:5e:b1 and VLAN ID 2 tied to port 0 (this is a default MAC address that can be changed)
  • Multicast/VLAN entry: MAC address ff:ff:ff:ff:ff:ff and VLAN ID 2 tied to port 0
• VLAN Table Entries: As the driver comes up there is also a set of VLAN Table Entries that are added to the ALE. These entries determine which ports belong in the member list of each VLAN as well as which VLAN headers should be removed on switch egress.
  • VLAN entry: VLAN ID 0, force_untagged_egress = 0x7, member_list = 0x7 (this means that any packet with a VLAN ID of 0 should have its VLAN header removed on switch egress from all three ports, since 0x7 is 0b111 corresponding to a port mask of all ports).
  • VLAN entry: VLAN ID 1, force_untagged_egress = 0x3, member_list = 0x3 (this means that any packet with a VLAN ID of 1 should have its VLAN header removed on switch egress from port 0 and port 1. This also means that port 0 and port 1 are both members of the VLAN ID 1)
  • VLAN entry: VLAN ID 2, force_untagged_egress = 0x5, member_list = 0x5 ( this means that any packet with a VLAN ID of 2 should have its VLAN header removed on switch egress from port 0 and port 2. This also means that port 0 and port 2 are both members of the VLAN ID 2)
• Directed Packets: The Linux driver uses directed packets when it sends a packet that is destined to exit through the external Ethernet ports. These directed packets that are sent by Linux are also untagged, which means that as they enter the switch at host port 0, a VLAN header with VLAN ID 0 is appended to the packet. As mentioned above, directed packets skip the Lookup Process and go straight to the Egress Process of the port they are destined for.

Dual EMAC Mode Intent

Given everything we now know from above, the intent of all of those settings from the Linux driver in dual emac mode is relatively straight forward.

• The Unicast/VLAN and Multicast/VLAN table entries that are added are meant to be the only matching rules that will allow packets to pass up to the Linux driver through host port 0. This allows the hardware to filter out a lot of unwanted traffic that would otherwise flood into the device and slow Linux down. 
• Placing the switch into VLAN_AWARE mode, setting the Pn_PORT_VLAN registers, and adding the VLAN table entries are all used to accomplish a similar set of tasks.
  • In order to represent two different MAC addresses (each tied to a different external Ethernet port) we needed the ALE to be aware of which port the packet entered the switch through. This is accomplished with the VLAN tagging based on Pn_PORT_VLAN during ingress. This allows us to have a MAC address rule that will only match for a specific external port, which is what we want. 
  • The VLAN table entries use the force_untagged_egress variable to then immediately remove those VLAN addresses that we just added since they have already served their purpose. The VLAN ID 0 rule is also used to remove the VLAN ID 0 header from packets that are destined for external Ethernet ports (since this header is appended at host port 0 ingress only because the switch is in VLAN_AWARE mode).
• Since the ARM host is aware of which MAC address belongs to which external port then we do not need the ALE's help to route packets to the correct external port. This is why directed packets can be used from the ARM host perspective. A side effect is the fact that the VLAN header with VLAN ID 0 is appended to the packet on host port 0 switch ingress and then removed on external Ethernet port egress.  

Other Resources

Linux Core CPSW User's Guide:Dual Standalone EMAC mode

Archived

Sitara Linux SDK 08.00.00.00