Parallel Redundancy Protocol (PRP) for IE 4000, IE 4010, and IE 5000 Switches

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Configuring PRP

This document provides details about configuring Parallel Redundancy Protocol (PRP) on the Cisco Industrial Ethernet 4000 Series, Cisco Industrial Ethernet 4010 Series, and Cisco Industrial Ethernet 5000 Series switches.

PRP is supported on multiple IE platforms, and PRP feature support may vary by platform. For details, refer to Feature History. Be sure to use the configuration guide for your IE platform.

Information About PRP

Parallel Redundancy Protocol (PRP) is defined in the International Standard IEC 62439-3. PRP is designed to provide hitless redundancy (zero recovery time after failures) in Ethernet networks. To recover from network failures, redundancy can be provided by network elements connected in mesh or ring topologies using protocols like RSTP, REP, or MRP, where a network failure causes some reconfiguration in the network to allow traffic to flow again (typically by opening a blocked port). These schemes for redundancy can take between a few milliseconds to a few seconds for the network to recover and traffic to flow again. PRP uses a different scheme, where the end nodes implement redundancy (instead of network elements) by connecting two network interfaces to two independent, disjointed, parallel networks (LAN-A and LAN-B). Each of these Dually Attached Nodes (DANs) then have redundant paths to all other DANs in the network. The DAN sends two packets simultaneously through its two network interfaces to the destination node. A redundancy control trailer (RCT), which includes a sequence number, is added to each frame to help the destination node distinguish between duplicate packets. When the destination DAN receives the first packet successfully, it removes the RCT and consumes the packet. If the second packet arrives successfully, it is discarded. If a failure occurs in one of the paths, traffic continues to flow over the other path uninterrupted, and zero recovery time is required. Non-redundant endpoints in the network that attach only to either LAN-A or LAN-B are known as Singly Attached Nodes (SANs). A Redundancy Box (RedBox) is used when an end node that does not have two network ports and does not implement PRP needs to implement redundancy. Such an end node can connect to a RedBox, which provides connectivity to the two different networks on behalf of the device. Because a node behind a RedBox appears for other nodes like a DAN, it is called a Virtual DAN (VDAN). The RedBox itself is a DAN and acts as a proxy on behalf of its VDANs.

Role of the Switch

The IE 4000, IE 4010, and IE 5000 switches implement RedBox functionality using Gigabit Ethernet port connections to each of the two LANs.

PRP Channels

Mixed Traffic and Supervisory Frames

Traffic egressing the RedBox PRP channel group can be mixed, that is, destined to either SANs (connected only on either LAN-A or LAN-B) or DANs. To avoid duplication of packets for SANs, the switch learns source MAC addresses from received Supervision frames for DAN entries and source MAC addresses from non-PRP (regular traffic) frames for SAN entries and maintains these addresses in the node table. When forwarding packets out the PRP channel to SAN MAC addresses, the switch looks up the entry and determines which LAN to send to rather than duplicating the packet.

A RedBox with VDANs needs to send supervisory frames on behalf of those VDANs. For traffic coming in on all other ports and going out PRP channel ports, the switch learns source MAC addresses, adds them to the VDAN table, and starts sending Supervisory frames for these addresses. Learned VDAN entries are subject to aging.

You can add static entries to the node and VDAN tables as described in Adding Static Entries to the Node and VDAN Tables. You can also display the node and VDAN tables and clear entries. See Verifying Configuration and Clearing All Node Table and VDAN Table Dynamic Entries.

PTP over PRP

Precision Time Protocol (PTP) can operate over Parallel Redundancy Protocol (PRP) on IE 4000, IE 4010, and IE 5000 switches. PRP provides high availability through redundancy for PTP. For a description of PTP, see Precision Time Protocol Software Configuration Guide for IE 4000, IE 4010, and IE 5000 Switches.

The PRP method of achieving redundancy by parallel transmission over two independent paths (see Information About PRP) does not work for PTP as it does for other traffic. The delay experienced by a frame is not the same in the two LANs, and some frames are modified in the transparent clocks (TCs) while transiting through the LAN. A Dually Attached Node (DAN) does not receive the same PTP message from both ports even when the source is the same. Specifically:

Previously, PTP traffic was allowed only on LAN-A to avoid the issues with PTP and parallel transmission described above. However, if LAN-A went down, PTP synchronization was lost. To enable PTP to leverage the benefit of redundancy offered by the underlying PRP infrastructure, PTP packets over PRP networks are handled differently than other types of traffic. The implementation of the PTP over PRP feature is based on the PTP over PRP operation detailed in IEC 62439-3:2016, Industrial communication networks - High availability automation networks - Part 3: Parallel Redundancy Protocol (PRP) and High-availability Seamless Redundancy (HSR). This approach overcomes the problems mentioned above by not appending an RCT to PTP packets and bypassing the PRP duplicate/discard logic for PTP packets.

PTP over PRP Packet Flow

The following figure illustrates the operation of PTP over PRP.

In the figure, VDAN 1 is the grandmaster clock (GM). Dually attached devices receive PTP synchronization information over both their PRP ports. The LAN-A port and LAN-B port use a different virtual clock that is synchronized to the GM. However, only one of the ports (referred to as SLAVE) is used to synchronize the local clock (VDAN 2 in the figure). While the LAN-A port is the SLAVE, the LAN-A port’s virtual clock is used to synchronize VDAN-2. The other PRP port, LAN-B, is referred to as PASSIVE_SLAVE. The LAN-B port’s virtual clock is still synchronized to the same GM, but is not used to synchronize VDAN 2.

If LAN-A goes down, the LAN-B port takes over as the SLAVE and is used to continue synchronizing the local clock on RedBox 2. VDAN 2 attached to RedBox 2 continues to receive PTP synchronization from RedBox 2 as before. Similarly, all DANs, VDANs and Redboxes shown in the figure continue to remain synchronized. Note that for SANs, redundancy is not available, and in this example, SAN 1 will lose synchronization if LAN-A goes down.

Due to the change, VDAN 2 may experience an instantaneous shift in its clock due to the offset between the LAN-A port’s virtual clock and the LAN-B port’s virtual clock. The magnitude of the shift should only be a few microseconds at the most, because both clocks are synchronized to the same GM. The shift also occurs when the LAN-A port comes back as SLAVE and the LAN-B port becomes PASSIVE_SLAVE.

Supported Location of GM

The GM can be located in a PTP over PRP topology as one of the following:

The GM cannot be a SAN attached to LAN-A or LAN-B, because only the devices in LAN-A or LAN-B will be synchronized to the GM.

Configuration

PTP over PRP does not require configuration beyond how you would normally configure PTP and PRP separately, and there is no user interface added for this feature. The difference is that prior to the PTP over PRP feature, PTP worked over LAN_A only; now it works over both LANs. Before implementing PTP over PRP, refer to Guidelines and Limitations.

The high-level workflow to implement PTP over PRP in your network is as follows:

  1. Refer to PRP RedBox Types to determine the location of the PRP RedBox. Refer to Precision Time Protocol Software Configuration Guide for IE 4000, IE 4010, and IE 5000 Switches to determine PTP mode and profile.
  2. Configure PTP as described in Precision Time Protocol Software Configuration Guide for IE 4000, IE 4010, and IE 5000 Switches, following the procedure for the PTP profile determined in step 1 above.
  3. Configure PRP as described in Creating a PRP Channel and Group.

Supported PTP Profiles and Clock Modes

The following table summarizes PTP over PRP support for the various PTP profiles and clock modes. In unsupported PTP profile/clock mode combinations, PTP traffic flows over LAN-A only. LAN-A is the lower numbered interface. See PRP Channels for PRP interface numbers.

PRP RedBox type as per IEC 62439-3

PRP RedBox Types

The switch plays the role of a RedBox in PRP networks. This section describes the types of PRP RedBoxes supported for PTP over PRP as defined in IEC 62439-3.

PRP RedBox as a Doubly Attached BC (DABC) with E2E

In the configuration shown below, two RedBoxes (for example, M and S) are configured as Boundary Clocks (BCs) that use the End-to-End delay measurement mechanism and IEEE1588v2 Default Profile. The Best Master Clock Algorithm (BMCA) on RedBox M determines port A and port B to be MASTER. The PTP protocol running on Redbox M treats both ports A and B individually as MASTER ports and sends out Sync and Follow_Up messages individually on both the ports.

On Redbox S, the regular BMCA operation determines port A to be a SLAVE and port B to be PASSIVE. However, with the knowledge that ports A and B are part of the same PRP channel, port B is forced into PASSIVE_SLAVE state. Port A and Port B on Redbox S operate as follows:

PRP RedBox as Doubly Attached BC (DABC) with P2P

The following figure shows an example where Redbox M and Redbox S are configured to run in Power Profile as Boundary Clocks that use Peer-to-Peer (P2P) delay measurement mechanism. In this example, the GM is the ordinary clock attached through LAN C. All the clocks are configured to run Peer-to-Peer Delay measurement and the peer delay is regularly calculated and maintained on every link shown in the figure.

The BMCA on Redbox M determines ports A and B to be MASTER. The PTP protocol running on Redbox M treats both ports A and B individually as MASTER ports and sends out Sync and Follow_Up messages individually on both the ports.

On Redbox S, the regular BMCA operation determines port A to be SLAVE and port B to be PASSIVE. However, with the knowledge that ports A and B are part of the same PRP channel, port B is forced into PASSIVE_SLAVE state. Port A and Port B on Redbox S operate as follows:

PRP RedBox as Doubly Attached TC (DATC) with P2P

The following figure shows an example where Redbox M and Redbox S are configured to run in Power Profile mode as Transparent Clocks. In this example, the GM is the ordinary clock attached through LAN C. All the clocks are configured to run Peer-to-Peer Delay measurement and the peer delay is regularly calculated and maintained on every link shown in the figure.

Redbox M and Redbox S run BMCA even though it is not mandatory for a P2P TC to run BMCA. On Redbox M, the BMCA determines ports A and B to be MASTER. Redbox M forwards all Sync and Follow_Up messages received on port C out of ports A and B.

On Redbox S, port A is determined to be SLAVE and port B to be PASSIVE_SLAVE as described earlier. Port A and Port B on Redbox S operate as follows:

LAN-A and LAN-B Failure Detection and Handling

Failures in LAN-A and LAN-B are detected and handled in the same way for all Redbox types described in PRP RedBox Types.

Using the example shown in PRP Redbox as DATC with P2P with the GM as a SAN in LAN C, a failure in LAN-A or LAN-B pertaining to PTP can occur due to the following reasons:

These events result in PTP Announce Receipt Timeout on Redbox S, which triggers the BMCA calculation. Refer to section 7.7.3.1 of the IEEE 1588v2 standard for details on Announce Receipt Timeout.

The BMCA, once invoked, changes the state of the PASSIVE_SLAVE port to SLAVE and SLAVE to PASSIVE_SLAVE or PASSIVE or FAULTY. The state changes are done atomically to avoid transient cases where there are two SLAVE ports or two PASSIVE_SLAVE ports.

Redbox S now synchronizes to the GM over the new SLAVE port. The change to synchronization should be quick and seamless, unless the delays experienced by PTP packets on the two LANs are very different or if there are some non-PTP devices in the LANs.

The SAN Slave in LAN D also sees this shift in the timing from Redbox S and needs to converge to the new clock. This is similar to a GM change event for this clock, but as mentioned earlier, the change is usually seamless.

Prerequisites

All releases and FPGA support basic PRP for IE 4000, IE 4010 and IE 5000 switches.

However, node table support and automatic learning of source MAC addresses in the VDAN table require the following releases and FPGA versions:

To check the FPGA version, use the show version command. For example:
 IE4000#sh ver | incl FPGA Backplane FPGA version : 0.37 IE4000# 

Guidelines and Limitations