Parallel Redundancy Protocol Overview
PRP is defined in the international standard IEC 62439-3 and provides high availability in Ethernet networks. PRP implements redundancy by using PRP-enabled nodes (IACS devices) that send duplicate Ethernet frames to two fail-independent network infrastructures, known as LAN A and LAN B.
PRP technology is well suited for variety of critical infrastructure IACS in process and heavy industries that require continuous, high availability operation. Advantages of using PRP over other network resiliency technologies include:
- No IACS data loss during a single fault in LAN A or LAN B
- Protection against extended infrastructure failures in a single LAN (e.g., maintenance work, prolonged power outages, multiple network faults)
- Recovery after multiple faults in certain situations depending on the LAN topologies
- Flexibility of allowing various network topologies, resiliency protocols, and IES platforms for each LAN
- Ease of migration from non-Ethernet redundant media technologies such as ControlNet ® Networks (not covered as part of CPwE PRP)
Important factors when selecting PRP technology are support of PRP by IACS devices, possibility of building two independent network topologies without common faults, connectivity to non-PRP parts of the plant-wide or site-wide infrastructure, and proper configuration of other network services such as multicast management and time synchronization.
The following sections provide a brief overview of the PRP operation, components, and topologies. For more details refer to:
Parallel Redundancy Protocol Components
A PRP network includes the components shown in Table 2-1 .
Table 2-1 PRP Components
|
|
|
LAN A and LAN B |
Redundant, active Ethernet networks that operate in parallel and are fault independent. |
|
Double attached node (DAN) |
An IACS device with PRP technology that connects to both LAN A and LAN B. |
1756-EN2TP ControlLogix® EtherNet/IP module 5094-AENTR Flex 5000™ EtherNet/IP module |
Single attached node (SAN) |
An IACS device without PRP technology that connects to either LAN A or LAN B. A SAN does not have PRP redundancy and typically is a non-critical device or its function is duplicated in both LANs. |
|
Redundancy box (RedBox) |
An IES with PRP technology that connects non-PRP IACS devices or non-PRP part of the network to both LAN A and LAN B. |
Stratix 5400, Stratix 5410 managed switches |
Virtual double attached node (VDAN) |
An IACS device without PRP technology that connects to both LAN A and LAN B through a RedBox. A VDAN has PRP redundancy and appears to other nodes in the network as a DAN. |
|
Infrastructure switches |
Switches in LAN A or LAN B that are not configured as a RedBox. |
Any of the Stratix managed switches |
Parallel Redundancy Protocol Operation
An IACS device with PRP technology (a DAN) has two Ethernet ports that operate in parallel and attach to independent LAN A and LAN B. During normal network operation, a DAN simultaneously sends and receives duplicate Ethernet frames through both ports.
The receiving node accepts whichever frame arrives first and discards the subsequent copy. If a failure occurs in one of the LANs, traffic continues to flow through the other LAN uninterrupted with no recovery time.
Unlike other resiliency protocols, such as Spanning Tree Protocol (STP) or DLR, PRP does not require reconfiguration in LAN A or B after the fault (e.g., unblocking the port). PRP provides redundancy by using duplicate network infrastructure rather than redundant paths in the same network.
Figure 2-1 Redundant Path versus Redundant Networks
DAN Operation
A DAN has two Ethernet ports that are attached to the upper communication layers of the IACS device through the Link Redundancy Entity (LRE). The LRE handles duplication of packets and management of redundancy (Figure 2-2). The upper layers are unaware of redundancy because the LRE provides to them the same interface as a non-redundant network adapter. The DAN uses the same MAC address and IP address to communicate on both LANs.
Figure 2-2 PRP DAN Communication Layers
When a DAN sends a frame to another DAN:
- The LRE creates two copies of the frame and sends them through LAN A and LAN B ports with a Redundancy Check Trailer (RCT) appended to each frame. The 6-byte trailer contains a sequence number, the LAN identifier, frame data size, and the PRP suffix that identifies the trailer type as PRP (Figure 2-3).
- The duplicate frames traverse the two LANs, perhaps under different network conditions and with slightly different delays, and arrive at the destination node.
- The LRE in the destination DAN forwards the first received copy of the frame to the upper layers (without the PRP trailer) and discards the second copy (if it arrives).
- PRP algorithm is designed in a way that it should never reject a legitimate frame, however in rare cases a duplicate frame can be accepted as a new one and passed to the upper layers. This could happen if the duplicate frame arrives with significant time difference. Upper layer protocols (TCP or EtherNet/IP) are able to handle occasional duplicate frames.
Figure 2-3 Ethernet Frame with RCT Appended
Note
The PRP trailer adds six bytes to an Ethernet frame. To accommodate a maximum size 1500 byte Ethernet frame with the PRP trailer attached, all LAN A/B network devices should be configured with the maximum transmission unit (MTU) size of at least 1506 bytes. This is not required for a RedBox IES.
RedBox Operation
The RedBox device acts as LRE for one or several connected VDANs or for a non-PRP bridged network segment. The RedBox keeps track of sequence numbers and handles duplicate received frames for multiple VDANs.
Two Gigabit Ethernet ports on the RedBox IES are configured as one logical interface—a PRP channel group (Figure 2-4). The PRP ports can be in access mode for single VLAN deployments, trunk mode to support multiple VLANs, or routed mode. In the channel group, the lower numbered member port is the primary port and connects to LAN A. The higher numbered port is the secondary port and connects to LAN B.
Figure 2-4 RedBox PRP Channel
- The Stratix 5400 IES supports one PRP channel. The Stratix 5410 IES supports up to two PRP channels.
- A maximum of 512 VDAN entries are supported in the PRP VDAN table. If the VDAN table is full, the switch cannot send supervisory frames for new VDANs.
- Ports in the PRP channel group cannot be configured for other resiliency protocol, e.g., DLR or Resilient Ethernet Protocol (REP).
- Once the PRP ports are added to the group, individual port settings should not be changed unless the port is removed from the group.
For more information about Stratix switch PRP functionality and configuration, refer to:
In addition to connecting VDANs, a RedBox IES in the PRP network is necessary in following situations:
- Routing is enabled in the network.
- Connectivity to a non-PRP LAN is required, e.g., a DLR segment, a plant-wide or site-wide connectivity.
- Internet Group Management Protocol (IGMP) querier role is required for multicast management.
- Boundary clock role is required for CIP Sync operation.
Recommendations for configuring RedBox IES for these use cases are described in later sections of this Design and Implementation Guide.
SAN Operation
Devices without PRP support (SAN) can be included in the PRP topology as non-redundant devices connected to either LAN A or LAN B:
- A SAN can accept and process Ethernet frames with the RCT attached. The SAN simply ignores the PRP trailer as the Ethernet padding in the frame.
- To avoid duplication of packets for SANs, the DAN or the RedBox IES keeps track of learned MAC addresses in the PRP node table, identifies the device as attached to only one LAN, then sends the frame to that LAN only without the PRP trailer.
- The SAN traffic from one of the LANs can be received and processed by the destination DAN in a normal way.
- All SANs should have unique IP addresses in the PRP network, even if they belong to different LANs.
Network Management and Supervision
Benefits of network redundancy can only be realized if the network status and performance is monitored. This can be achieved with a Network Monitoring Tool (NMT) using Simple Network Management Protocol (SNMP), EtherNet/IP diagnostic tools, and using built-in diagnostic available in PRP-capable IACS devices and RedBox IES.
PRP nodes (DAN or RedBox) verify if frames from known DANs or VDANs are received from both PRP ports and provide real-time PRP statistics and status. For more information, refer to Chapter4, “CPwE Parallel Redundancy Protocol Monitoring and Troubleshooting”
Each DAN periodically sends a PRP supervision frame that announces its presence on the network and allows other nodes to check health of the PRP network. The RedBox sends supervisory frames on behalf of connected VDANs. The supervisory frames are Layer 2 multicast Ethernet frames sent to a reserved multicast MAC address.
Note
An NMT should be connected to the PRP network via a RedBox to access IACS devices and IES in both LANs. While LAN A and LAN B are isolated on the network layer, all managed IES and IACS devices should have different IP addresses within and between each LAN for management purposes.
Parallel Redundancy Protocol Network Design Recommendations
PRP technology is implemented in IACS devices, therefore network infrastructure devices (other than the RedBoxes) do not have to be PRP capable. PRP is not dependent on any particular LAN topology and should provide a single fault tolerance with zero data loss even with non-resilient topologies in each LAN such as star, linear, or a single switch.
When designing a PRP network, follow these recommendations:
- If possible, use resilient topologies (redundant star, ring) in each LAN for additional resiliency protection. In this case, an extended outage or maintenance in one of the LANs still should allow the IACS to recover from a subsequent fault in the other LAN (with convergence time depending on the resilient LAN protocol).
- Design the architecture to avoid or minimize architecture-wide faults that impact both LANs such as power failures or damage of both redundant cabling paths.
- Use similar topologies for both LANs with similar network latency and number of hops in normal network conditions. Avoid using different types of connectivity between LAN A and LAN B, for example a high-speed wired network for LAN A and low bandwidth, high-latency cellular, satellite, or wireless technology for LAN B.
- Do not connect IES (other than RedBoxes) to both LAN A and LAN B. Direct links between LAN A and LAN B IES are not allowed.
- Do not connect any RedBox IES to each other via a Layer 2 path that bridges any of the VLANs that exist in the PRP network. Such a connection creates a bridging loop in the network. Layer 3 routed paths are allowed.
- If routing is required in the PRP network, configure a RedBox IES as the router. Do not enable routing on the LAN A or LAN B IES. For recommendations on the routing redundancy design with PRP and how to connect to the Industrial Zone network, see Connectivity to the Industrial Zone Network.
- Apply the same recommended network and security practices as for a non-PRP network, such as using managed switches with diagnostic, loop prevention, multicast management and security features, minimizing broadcast domains with VLAN segmentation, hardening the network and the IACS applications against security threats, maintaining good change control practices, and IES configuration management.
Parallel Redundancy Protocol Topology Examples
This section provides some examples of PRP architectures and topologies.
Figure 2-5 shows an example of PRP deployment with two parallel fault-isolated physical paths. This could be useful in mining or transportation applications (e.g., parallel tunnels), marine applications (two sides of a ship), and other similar use cases.
In the example below, both LANs utilize a ring topology rather than linear topology for additional resiliency. In most greenfield installations, the cost of having a return cable path is insignificant when installing a cable bundle. The benefits of using a resilient LAN topology are greater than the additional effort of configuring and monitoring of a ring protocol.
The example architecture also implements redundant programmable automation controllers (PAC) with ControlLogix ® redundancy for greater availability and protection from controller faults.
Figure 2-5 PRP Topology Example with Parallel Paths
Figure 2-6 shows an example of a PRP topology with dual rings and a single (non-redundant) PAC. This topology is common for water/wastewater, mining, oil and gas, and other industries that traditionally have used redundant dual-media topologies over large geographical areas.
Figure 2-6 PRP Topology Example with Dual Rings—Single PAC
Figure 2-7 shows an example of a PRP topology with dual rings and ControlLogix redundant PACs.
Figure 2-7 PRP Topology Example with Dual Rings—Redundant Controllers
Figure 2-8 shows an example of the star topology in a PRP network.
Note that access IES could also be connected with redundant uplinks to the aggregation IES in the LAN A or B, for example using EtherChannel technology. The cost of additional cabling and available ports should be considered.
Figure 2-8 PRP Star Topology Example
Note
The CPwE PRP architecture has been tested and validated using a dual ring topology for LAN A and LAN B with a mix of redundant and non-redundant PACs (Figure 2-6 and Figure 2-7). Other topologies (star, redundant star, linear) are supported if configuration recommendations in this Design and Implementation Guide are followed.
Unsupported Topologies
This section describes a number of invalid PRP topologies or topologies that are not recommended due to performance or availability concerns.
- LAN A and LAN B infrastructure cannot be bridged together using a direct link, a non-RedBox IES, or a two-port embedded switch device that does not support PRP (e.g., a ControlLogix 1756-EN2TR module or a 1783-ETAP EtherNet/IP DLR tap module).
Figure 2-9 Invalid Topology—Bridging LAN A and LAN B
- RedBox IES cannot be connected through a non-PRP ports via a Layer 2 path that forwards traffic from any of the PRP VLANs, including IACS data VLANs, management VLAN, or the native VLAN (Figure 2-10).
Figure 2-10 Invalid Topology—Bridging PRP VLAN via RedBox non-PRP Ports
- LAN A/B topologies should not contain two-port embedded switch devices, including 1783-ETAP modules, in the data path. Embedded switch devices cannot be configured for the larger MTU sizes to accommodate the PRP trailer in the frame. As a result, maximum size Ethernet frames may be dropped. This also applies to any IES without the option to increase the MTU size (Figure 2-11).
Figure 2-11 Unsupported Topology—Traversing Two-port Embedded Switch Devices
- It is not recommended to combine high-bandwidth low-latency LAN as the primary LAN and low-bandwidth high-latency WAN or wireless technology as the secondary LAN. One of the possible issues could be increased chance of duplicate frames arriving late and being wrongly accepted as non-duplicate (Figure 2-12).
Figure 2-12 Unsupported Topology—Using High-latency Connection as Secondary
Connectivity to the Industrial Zone Network
The CPwE PRP architecture provides guidelines for connecting a PRP-enabled Cell/Area Zone to the plant-wide or site-wide network in the Industrial Zone.
Although IACS applications may exist when a PRP network is deployed as a standalone network (e.g., an isolated I/O network), having plant-wide or site-wide connectivity to the IACS network with PRP technology allows many benefits of the converged network model. This consists of remote access for diagnostics and troubleshooting, distributed network applications using virtual server environment in the Level 3 Site Operations, and access to IACS device data and analytics as part of the Connected Enterprise™ smart manufacturing model.
When connecting a PRP topology with two redundant and isolated LANs to a non-PRP resilient topology, these rules should be followed to avoid bridging loops:
- Only RedBox IES can be used as gateways from a PRP to a non-PRP part of the network.
- Any non-PRP enabled path between RedBox IES should only include Layer 3 (routed) connections.
Figure 2-13 shows the CPwE PRP architecture that provides redundant connectivity from a resilient core layer in the Industrial Zone to a pair of distribution RedBox IES connected to the PRP topology (shown as redundant rings in this example).
Figure 2-13 Connectivity to the Industrial Zone
- Redundant RedBox IES are configured with Hot Standby Router Protocol (HSRP) as active/standby default gateways for IP subnets in the PRP topology.
- Redundant links between RedBoxes and towards the core are configured as Layer 3 EtherChannels (routed ports).
- Dynamic routing protocol is configured between the distribution RedBox HSRP pair and the core switch network.
–
Enhanced Interior Gateway Routing Protocol (EIGRP) has been validated as part of the CPwE PRP.
–
Open Shortest Path First (OSPF) routing protocol can also be used depending on the existing infra-structure and requirements. OSPF has not been validated as part of CPwE PRP.
–
Static routes between RedBox IES and the core are allowed but not recommended due to increased complexity of configuration and maintenance in large environments. CPwE PRP has not been validated with static routing.
- Cisco StackWise Virtual technology or Virtual Switching System (VSS) technology is used for the Cisco Catalyst core switch redundancy.
–
Other resiliency protocols and technologies can be used as outlined in the CPwE Resiliency DIG but have not been tested and validated as part of this CPwE PRP.
EtherChannel, HSRP, and Routing Protocol Considerations
General EtherChannel and HSRP recommendations and configuration guidelines are provided in the CPwE Resiliency Design and Implementation Guide:
For information about EIGRP design and configuration, refer to:
For information about OSPF, refer to:
An important consideration for the CPwE PRP routed design is that both Layer 3 RedBox IES carry traffic from the Industrial Zone core network to the Cell/Area Zone, e.g., from an HMI server to an HMI client or a controller (Figure 2-14).
This is due to the equal cost routing on the core switch where downstream traffic is split roughly evenly between the links to the RedBoxes. As a result, a failure of either the active or the standby HSRP RedBox IES may impact the IACS traffic from Level 3 Site Operation. Similarly, restoring a RedBox IES as the standby HSRP gateway may lead to comparatively small convergence times as the traffic is restored on the second path.
Figure 2-14 Routed Traffic Flow
The recommended and validated EtherChannel, HSRP, and EIGRP configuration for CPwE PRP is provided in Chapter3, “CPwE Parallel Redundancy Protocol Configuration” Maximum observed convergence times for different faults affecting Layer 3 traffic are listed in Table 2-2 .
Note
Convergence times for routed IACS data include the sum of multiple events including HSRP failover time, dynamic routing protocol convergence, and Layer 2 switched network convergence. Different routing configurations, protocols, and core switch platforms may provide different results.
Table 2-2 Routed Traffic Convergence Times for HSRP and EIGRP
|
Maximum Observed Convergence Time (ms)
|
|
|
Inter-VLAN to Cell/Area Zone
|
Gateway down (HSRP Active) |
2200 |
1215 |
1155 |
Gateway down (HSRP Standby) |
2180 |
0 |
0 |
Gateway restored (HSRP Standby) |
320 |
0 |
0 |
Table 2-3 provides maximum observed converged times for Layer 3 EtherChannel faults. These results are comparable to convergence results for Layer 2 EtherChannel documented in the CPwE Resiliency DIG.
The results for loss of both EtherChannel links are also included. This event can happen, for example, due to misconfiguration of the EtherChannel or physical damage to both links.
Table 2-3 Routed Traffic Convergence Times for Layer 3 EtherChannel
|
Maximum Observed Convergence Time (ms)
|
|
|
EtherChannel link loss (HSRP Active to Core) |
85 |
90 |
EtherChannel link restore (HSRP Active to Core) |
70 |
60 |
EtherChannel link loss (HSRP Standby to Core) |
90 |
0 |
EtherChannel link restore (HSRP Standby to Core) |
135 |
0 |
EtherChannel loss, both links (HSRP Active to Core) |
1735 |
2400 |
EtherChannel loss, both links (HSRP Standby to Core) |
2265 |
0 |
Note
It is important to evaluate IACS application requirements for the routed traffic and compare with the expected convergence times. For example, Requested Packet Interval (RPI) values for routed CIP Class 1 data (I/O or Produced Consumed tags) may need to be increased to allow for higher convergence.
Connectivity to Device Level Ring (DLR)
This section describes recommendations for integrating existing or new DLR networks with PRP topologies.
- The recommended resilient architecture is to connect the DLR topology to a separate distribution switch (Figure 2-15).
Figure 2-15 Connecting DLR Topology—Separate Distribution Switch
- Connecting a controller chassis to both PRP and DLR networks using separate EtherNet/IP modules is allowed, as long as other guidelines are followed (Figure 2-16). In this example, a ControlLogix Redundancy chassis is connected to the PRP topology as well as the switch DLR topology.
Figure 2-16 Connecting Controller Chassis to both PRP and DLR
- A RedBox IES can be included in the DLR topology as the DLR supervisor for one or multiple rings (Figure 2-17). DLR ports cannot be the same as the PRP channel ports.
Note
In this case the RedBox is a single point of failure for the traffic between the DLR and the PRP topologies. This architecture is not recommended for critical application data traversing the RedBox and has not been validated for performance or scalability in this CPwE PRP.
Figure 2-17 PRP to DLR Connectivity via a RedBox
- Using two RedBox IES as redundant DLR gateways is not supported due to increased convergence time for traffic traversing the gateways during certain faults, which exceeded requirements for IACS applications (Figure 2-18).
Figure 2-18 Unsupported Topology—PRP to DLR Redundant Gateways
Note
A PRP-enabled EtherNet/IP module cannot be used as part of the DLR or a linear topology. PRP modules do not implement the embedded switch technology and traffic cannot traverse from port A to port B.
Network Services Recommendations
This section provides recommendations for network services and IES features that could be required in the PRP-enabled IACS network, such as VLAN segmentation (zoning) and trunking, multicast traffic management, time synchronization, and network address translation (NAT).
VLAN Segmentation (Zoning) and Trunking
PRP technology can be deployed in networks with VLAN segmentation. Links between IES can be configured with VLAN trunking to carry traffic from DANs and VDANs that belong to multiple VLANs (Figure 2-19).
Figure 2-19 VLAN Segmentation (Zoning) in PRP Network
Follow these recommendations when using VLAN segmentation (zoning) with PRP:
- Both PRP ports on a DAN should be connected to the same VLAN in LAN A and LAN B.
- The PRP channel ports on a RedBox IES can be configured as VLAN trunk ports (more common) or access mode ports (i.e., single VLAN). Trunk mode allows having VDANs in multiple VLANs or use a separate management VLAN for the RedBox IES.
- PRP channels on the redundant Layer 3 RedBox IES in CPwE PRP are configured as VLAN trunks. The architecture has been validated with inter-VLAN IACS traffic which is routed through the RedBox.
- Links between IES in each PRP LAN are configured as trunks which follows CPwE best practices. The native VLAN should differ from any of the IACS VLANs.
Note
PRP supervisory frames are Layer 2 multicast Ethernet frames that cannot be routed between VLANs. DANs can only report diagnostic information about PRP devices in their VLANs.
Spanning Tree Protocol
Design and configuration of the Spanning Tree Protocol (STP) in PRP LAN A and LAN B should follow general recommendations in the CPwE Resiliency Design and Implementation Guide. Special considerations exist for STP operation between a RedBox IES and infrastructure IES (Figure 2-20):
- Bridge Protocol Data Unit (BPDU) frames from STP are filtered on the PRP channel ports. As a result, LAN A/B switches exclude the RedBox IES in the STP operation.
- STP is running on the RedBox IES by default and can operate in the non-PRP infrastructure connected to the RedBox (if it exists) for normal loop prevention and resiliency purposes.
- PortFast Trunk mode should be configured for the PRP channel group on all RedBox IES, including redundant Layer 3 Redboxes, and on the LAN A/B IES ports connected to the RedBox. This is necessary to minimize port recovery time during network faults.
Figure 2-20 Spanning Tree Operation in the PRP Network
Note
Other resiliency protocols such as DLR and REP, if present in the PRP LAN topologies, may have their own considerations for STP interoperability, separate from the PRP considerations. Refer to the corresponding CPwE DLR Design Guide and CPwE Resiliency Design and Implementation Guide for more information.
Multicast Management
Multicast EtherNet/IP traffic is required for I/O and consumed data in ControlLogix Redundancy and for CIP Sync communication using Precision Time Protocol (PTP). Both types of multicast data could be used in IACS applications where PRP technology is deployed.
It is critical to follow design and configuration guidelines for multicast traffic management in CPwE PRP to make sure that high availability is achieved:
- Enable Internet Group Management Protocol (IGMP) snooping on all IES to reduce the amount of unnecessary multicast traffic to end nodes. IGMP snooping is enabled by default after Express Setup on Stratix managed switches.
- Configure IGMP querier on the redundant Layer 3 RedBox IES (distribution switches in active/standby HSRP configuration). Make sure that at least two IGMP queriers are present. In order to win the querier election, switches should have the lowest IP addresses in the subnet.
- Disable IGMP querier on each IES in LAN A and LAN B.
- Configure uplink ports on the LAN IES as static multicast router (mrouter) ports, specifically the ports that could be in the path to the IGMP querier (distribution RedBox IES). This configuration enables multicast traffic flow in the topology when the IGMP querier changes (for example when the active HSRP gateway reboots).
Figure 2-21 illustrates IGMP snooping configuration in the PRP topology. For details on how to configure these settings, refer to Chapter3, “CPwE Parallel Redundancy Protocol Configuration”
Figure 2-21 Multicast Management with PRP
Note
After a LAN in a PRP network encounters a fault and is then repaired, there could be a delay in restoring multicast traffic PRP redundancy. The delay lasts until the IGMP querier reinstates the multicast traffic in the recovered LAN (typically within two minutes after the LAN is repaired). During that time, the other LAN will continue forwarding multicast traffic.
Network Address Translation
Network Address Translation (NAT) feature in IES provides benefits to OEM machine or process skid builders, such as IP address reuse and commissioning of “cookie cutter” machines without reprogramming, easier maintenance of machine configurations and controller programs, and better traffic control and additional security by limiting access only to selected devices on a machine or skid. It may also help plant or site engineers with integrating legacy stand-alone equipment into the plant-wide or site-wide network.
PRP technology is compatible with NAT since PRP operates in Layer 2 and does not rely on IP addresses (Layer 3). There are two possible implementations of NAT in a PRP topology (Figure 2-22):
- Configure NAT on the LAN A/B infrastructure IES to provide translation for DANs.
- Configure NAT on the RedBox IES to provide translation for VDANs.
Figure 2-22 Network Address Translation with PRP
Since CPwE PRP implements HSRP for router redundancy, the following translation rules need to be added to the NAT configuration:
- Gateway translation for the virtual IP address of the HSRP gateways
- Public-to-Private translation for the physical IP addresses of both active and standby HSRP gateways (RedBox IES)
Other general NAT recommendations and limitations may apply, for example topology considerations, multicast restrictions, or application restrictions. For more information on NAT with Stratix IES, refer to:
Precision Time Protocol (CIP Sync)
CIP Sync technology uses the IEEE 1588-2008 Precision Time Protocol (PTP) standard for time synchronization. CIP Sync is designed for local and plant-wide or site-wide IACS applications requiring high accuracies beyond those attainable with Network Time Protocol (NTP).
PRP networks support CIP Sync by implementing the doubly-attached clock model as specified in IEC 62439-3 standard (Figure 2-23):
- In a DAN or a RedBox with the master clock role, both ports A and B operate as CIP Sync master ports in their LAN segments.
- In a DAN or a RedBox with a slave clock role, both ports A and B are paired and function as CIP Sync slave ports.
–
One port is the active port that tunes the clock and reports its state as SLAVE.
–
The other port is passive and reports its state as PASSIVE_SLAVE. The passive port also measures path delay and maintains close synchronization to the active port. In case of a network failure on the active port, the passive port clock transitions smoothly to the active state.
- PTP traffic in the PRP network is handled differently from other traffic, specifically the Ethernet frames do not have RCT attached and bypass the duplicate/discard mechanism. PTP packets from DANs, such as Sync, Delay Request, and Delay Response are generated independently by each physical port.
Figure 2-23 PRP Doubly Attached Clock Model
Using PTP and CIP Sync in CPwE PRP requires specific configuration of the infrastructure IES and RedBox IES, as well as properly selecting the Grandmaster clock in the topology:
- Select one of the following as the Grandmaster for a PRP network:
–
A DAN (a doubly-attached clock)
–
A PAC or a time module in the PAC chassis that accesses the PRP network via a DAN module or via a RedBox IES
–
A VDAN that connects to a RedBox, e.g., a GPS time source device
–
A RedBox IES in the NTP/PTP Clock.
- For critical applications using time synchronization, consider implementing redundant Grandmasters connected as DANs, VDANs, or RedBoxes. Do not use Redundant Grandmasters connected as SANs (one in LAN A and another in LAN B).
- Configure RedBox IES in Boundary or NTP/PTP Clock mode. These modes are the only PTP modes that are supported on a RedBox IES.
–
A RedBox IES works as a boundary clock between the PRP and non-PRP segment of the network. It does not operate as a boundary clock between LAN A and LAN B.
- Configure infrastructure IES in LANA and LAN B as transparent clocks.
–
Transparent clock IES do not propagate PTP information between VLANs. Only one VLAN in a PRP topology should be enabled for time distribution.
–
Switches that do not support PTP, or in the PTP forward mode, are allowed but not recommended due to lack of delay compensation and lower time synchronization accuracy in the PRP architecture. IACS requirements for time accuracy should be considered.
–
If a switch in LAN A or LAN B is configured as a boundary clock, it may become the Grandmaster after the network fault in the local segment. This results in two Grandmaster clocks for the PRP architecture, which can create undesirable effects for doubly-attached clocks, for example causing time to drift apart.
- Select only one of the IACS VLAN for time distribution. Disable CIP Sync in all devices on all other VLANs,
–
Make sure that DANs in other VLANs have the Time Sync property disabled in the module properties.
–
Configure PTP VLAN explicitly on the PRP channel ports of RedBox IES.
These recommendations are summarized in Table 2-4 and Table 2-5 .
Table 2-4 PTP Clock Roles in the PRP Network
|
|
|
|
|
DAN |
Yes |
Yes |
Yes |
PAC |
Yes (via DAN in the controller chassis or a RedBox) |
Yes |
Yes |
VDAN |
Yes (via a RedBox) |
Yes |
Yes |
SAN |
No |
No |
Yes |
RedBox IES |
Yes (NTP/PTP mode in the Stratix 5400 and Stratix 5410) |
Yes |
Yes |
LAN A/B IES |
No |
No |
No |
Table 2-5 PTP Modes for IES in the PRP Network
|
|
|
|
|
|
RedBox IES |
Yes |
Yes |
No |
No |
LAN A/B IES |
No |
No |
Yes |
Yes |
Figure 2-24 shows an example of a CIP Sync architecture with a DAN as the master clock in the PTP-enabled VLAN. The Grandmaster clock in this example is a GPS time source (ControlLogix 1756-TIME module) connected via a DAN module in the chassis. All IACS devices are in the same PTP-enabled VLAN. PTP information is only distributed in one VLAN because of the transparent clock requirement for the infrastructure IES.
Figure 2-24 Example of CIP Sync with PRP Topology—DAN as the Grandmaster
Figure 2-25 shows an example of a CIP Sync architecture where RedBox IES (Layer 3 switches) are the primary and secondary Grandmasters in the CIP Sync architecture (NTP/PTP clock mode). Distribution switches can synchronize to an NTP source, e.g., a GPS receiver in the plant-wide or site-wide network.
In this example, PTP VLAN ID should be configured explicitly on the trunk ports (PRP channel ports) between the RedBox IES and the infrastructure switches. PTP information is only distributed in one VLAN because of the transparent clock requirement for the infrastructure IES.
Figure 2-25 Example of CIP Sync with PRP Topology—RedBox IES as the Grandmaster
For general information on designing and configuring time distribution in the plant-wide or site-wide network, refer to:
For information on how to configure PTP on Stratix managed switches, refer to:
Applying Parallel Redundancy Protocol with IACS Applications
PRP technology does not depend on the network layer or IACS application layer and therefore can be used with different types of IACS applications that require high availability. The following IACS applications and data types have been tested and validated within CPwE PRP:
- CIP Class 1 I/O and Produced Consumed tags (unicast and multicast)
- ControlLogix Redundancy (requires version 31.052 or higher)
- CIP Class 3 messaging
- CIP Safety™
- CIP Sync and PTP
- FactoryTalk® View Site Edition
- FactoryTalk® View Machine Edition
- FactoryTalk® Linx
- RSLinx® Classic
- Studio 5000 Logix Designer®
ControlLogix Redundancy with Parallel Redundancy Protocol
CPwE PRP architecture has been tested with ControlLogix Redundancy including redundant PAC chassis with EtherNet/IP modules (DAN) communicating to DAN or VDAN I/O devices, other PACs, and FactoryTalk applications. ControlLogix Redundancy included the following components:
- EtherNet/IP PRP modules for I/O and Produced Consumed data configured for IP address swapping during a chassis switchover.
- Dedicated EtherNet/IP PRP modules for HMI data that do not swap IP addresses.
- Redundant ControlLogix Controller shortcut type in FactoryTalk Linx that points to the Primary and Secondary controllers through the PRP modules without swapping IP addresses.
- ControlLogix Redundancy firmware revision 31.052 or later, FactoryTalk Linx 6.11 or later
- Infrastructure and RedBox IES configured for IGMP snooping as described in the previous sections.
PRP modules in the primary and secondary chassis can be connected to the same switch in LAN A and the same switch in LAN B or can be connected to different pair of switches in each LAN, depending on the physical layout of the architecture.
Note
Routed traffic to and from the ControlLogix Redundancy (e.g., FactoryTalk data or CIP Class 3 messages between VLANs) can be affected by Layer 3 switch faults, HSRP, and routing protocol convergence. These types of faults are not covered by the PRP zero data loss mechanism and should be considered independently in the PRP architecture design calculations.