System Management Software Configuration Guide for Cisco IE 2000U and Connected Grid Switches

Configuring Precision Time Protocol

This chapter describes Precision Time Protocol (PTP) and how to configure it on the Cisco Industrial Ethernet 2000U Series Switches, hereafter referred to as IE 2000U or switch. This chapter includes following sections:

•Information About Precision Time Protocol

•Prerequisites

•Guidelines and Limitations

•Default Settings

•Configuring PTP on the Switch

•Verifying Configuration

•Configuration Example

Information About Precision Time Protocol

Precision Time Protocol (PTP) is defined in IEEE-1588 as Precision Clock Synchronization for Networked Measurements and Control Systems, and was developed to synchronize the clocks in packet-based networks that include distributed device clocks of varying precision and stability. PTP is designed specifically for industrial, networked measurement and control systems, and is optimal for use in distributed systems because it requires minimal bandwidth and little overhead processing.

Why PTP?

Smart grid power automation applications such as peak-hour billing, virtual power generators, and outage monitoring and management, require extremely precise time accuracy and stability. Timing precision improves network monitoring accuracy and troubleshooting ability.

In addition to providing time accuracy and synchronization, the PTP message-based protocol can be implemented on packet-based networks, such as Ethernet networks. The benefits of using PTP in an Ethernet network include:

•Low cost and easy setup in existing Ethernet networks

•Very little network bandwidth is needed for PTP data packets

Ethernet Switches and Delays

In an Ethernet network, switches provide a full-duplex communication path between network devices. Switches send data packets to packet destinations using address information contained in the packets. When the switch attempts to send multiple packets simultaneously, some of the packets are buffered by the switch so that they are not lost before they are sent. When the buffer is full, the switch delays sending packets. This delay can cause device clocks on the network lose synchronization with one another.

Additional delays can occur when packets entering a switch are stored in local memory while the switch searches the MAC address table to verify packet CRC fields. This process causes variations in packet forwarding time latency, and these variations can result in asymmetrical packet delay times.

Adding PTP to a network can compensate for these latency and delay problems by correctly adjusting device clocks so that they stay synchronized with one another. PTP enables network switches to function as PTP devices, including boundary clocks and transparent clocks.

Note To learn more about PTP clock devices and their role in a PTP network, refer to the "PTP Clocks" section.

Message-Based Synchronization

To ensure clock synchronization, PTP requires an accurate measurement of the communication path delay between the time source (master) and the receiver (slave). PTP sends messages between the master and slave device to determine the delay measurement. Then PTP measures the exact message transmit time and receive times and uses these times to calculate the communication path delay. PTP then adjusts current time information contained in network data for the calculated delay, resulting in more accurate time information.

This delay measurement principle determines path delay between devices on the network, and the local clocks are adjusted for this delay using a series of messages sent between masters and slaves. The one-way delay time is calculated by averaging the path delay of the transmit and receive messages. This calculation assumes a symmetrical communication path; however, switched networks do not necessarily have symmetrical communication paths, due to the buffering process.

PTP provides a method, using transparent clocks, to measure and account for the delay in a time-interval field in network timing packets, making the switches temporarily transparent to the master and slave nodes on the network.

More Information

•To read a detailed description of synchronization messages, refer to the "PTP Event Message Sequences" section.

•To learn more about how transparent clocks calculate network delays, refer to the "Transparent Clock" section.

Figure 5-1 shows a typical 1588 PTP network that includes grandmaster clocks, switches in boundary clock mode, and Intelligent Electronic Device (IEDs) such as a digital relays or protection devices. In this diagram, Master 1 is the grandmaster clock. If Master 1 becomes unavailable, the boundary clock slaves switch to Master 2 for synchronization.

Figure 5-1 PTP Network

PTP Event Message Sequences

This section describes the PTP event message sequences that occur during synchronization.

Synchronizing with Boundary Clocks

The ordinary and boundary clocks configured for the Pdelay request-response mechanism use the following event messages to generate and communicate timing information:

•Sync

•Pdelay_Req

•Follow_Up

•Pdelay_Resp

These messages are sent in the following sequence:

1. The master sends a Sync message to the slave and notes the time (t1) at which it was sent.

2. The slave receives the Sync message and notes the time of reception (t2).

3. The master conveys to the slave the timestamp t1 by embedding the timestamp t1 in a Follow_Up message.

4. The slave sends a Pdelay_Req message to the master and notes the time (t3) at which it was sent.

5. The master receives the Pdelay_Req message and notes the time of reception (t4).

6. The master conveys to the slave the timestamp t4 by embedding it in a Pdelay_Resp message.

After this sequence, the slave possesses all four timestamps. These timestamps can be used to compute the offset of the slave clock relative to the master, and the mean propagation time of messages between the two clocks.

The offset calculation is based on the assumption that the time for the message to propagate from master to slave is the same as the time required from slave to master. This assumption is not always valid on an Ethernet network, due to asymmetrical packet delay times.

Figure 5-2 Detailed Steps—Boundary Clock Synchronization

Synchronizing with Peer-to-Peer Transparent Clocks

When the network includes multiple levels of boundary clocks in the hierarchy, with non-PTP enabled devices between them, synchronization accuracy decreases.

The round-trip time is assumed to be equal to mean_path_delay/2, however this is not always valid for Ethernet networks. To improve accuracy, the resident time of each intermediary clock is added to the correction field of SYNC or FOLLOW_UP messages. Peer-to-Peer transparent clocks also perform the peer-to-peer delay mechanism to measure the propagation delay between master clock and slave clocks.

Peer-to-peer transparent clocks measure the link delay between two clock ports implementing the peer delay mechanism. The link delay is used to correct timing information in Sync and Follow_Up messages.

Peer-to-peer transparent clocks use the following event messages:

•Pdelay_Req

•Pdelay_Resp

•Pdelay_Resp_Follow_Up

These messages are sent in the following sequence:

1. Port 1 generates timestamp t1 for a Pdelay_Req message.

2. Port 2 receives and generates timestamp t2 for this message.

3. Port 2 returns and generates timestamp t3 for a Pdelay_Resp message.

To minimize errors due to any frequency offset between the two ports, Port 2 returns the Pdelay_Resp message as quickly as possible after the receipt of the Pdelay_Req message.

4. Port 2 returns timestamps t2 and t3 in the Pdelay_Resp and Pdelay_Resp_Follow_Up messages respectively.

5. Port 1 generates timestamp t4 after receiving the Pdelay_Resp message. Port 1 then uses the four timestamps (t1, t2, t3, and t4) to calculate the mean link delay.

Figure 5-3 Detailed Steps—Peer-to-Peer Transparent Clock Synchronization

Synchronizing the Local Clock

In an ideal PTP network, the master and slave clock operate at the same frequency. However, drift can occur on the network. Drift is the frequency difference between the master and slave clock. You can compensate for drift by using the time stamp information in the device hardware and follow-up messages (intercepted by the switch) to adjust the frequency of the local clock to match the frequency of the master clock.

Best Master Clock Algorithm

The Best Master Clock (BMC) algorithm is the basis of PTP functionality. BMC specifies how each clock on the network determines the best master clock in its subdomain of all the clocks it can see, including itself. The BMC algorithm runs on the network continuously and quickly adjusts for changes in network configuration.

BMC uses the following criteria to determine the best master clock in the subdomain:

•Clock quality (for example, GPS is considered the highest quality)

•Clock accuracy of the clock's time base

•Stability of the local oscillator

•Closest clock to the grandmaster

In addition to identifying the best master clock, BMC also ensures that clock conflicts do not occur on the PTP network by ensuring that:

•Clocks do not have to negotiate with one another

•There is no misconfiguration, such as two master clocks or no master clocks, as a result of the master clock identification process

PTP Clocks

A PTP network is made up of PTP-enabled devices and devices that are not using PTP. The PTP-enabled devices typically consist of the following clock types, which are described in this section:

•Grandmaster Clock

•Ordinary Clock

•Boundary Clock

•Transparent Clock

Grandmaster Clock

Within a PTP domain, the grandmaster clock is the primary source of time for clock synchronization using PTP. The grandmaster clock usually has a very precise time source, such as a GPS or atomic clock. When the network does not require any external time reference and only needs to be synchronized internally, the grandmaster clock can free run.

Ordinary Clock

An ordinary clock is a PTP clock with a single PTP port. It functions as a node in a PTP network and can be selected by BMC as a master or slave within a subdomain. Ordinary clocks are the most common clock type on a PTP network because they are used as end nodes on a network that is connected to devices requiring synchronization. Ordinary clocks have various interface to external devices.

Boundary Clock

A boundary clock in a PTP network operates in place of a standard network switch or router. Boundary clocks have more than one PTP port, and each port provides access to a separate PTP communication path. Boundary clocks provide an interface between PTP domains. They intercept and process all PTP messages, and pass all other network traffic. The boundary clock uses the BMC algorithm to select the best clock seen by any port. The selected port is then set as a slave. The master port synchronizes the clocks connected downstream, while the slave port synchronizes with the upstream master clock.

Transparent Clock

The role of transparent clocks in a PTP network is to update the time-interval field that is part of the PTP event message. This update compensates for switch delay and has an accuracy of within one picosecond.

Peer-to-peer (P2P) transparent clocks measure the PTP event message transit time (also known as resident time) for SYNC message and PDELAY_REQUEST messages. In addition, P2P transparent clocks measure the port-to-port propagation time, that is, the link delay, between two communicating ports supporting the peer delay mechanism.

The measured transit time of a SYNC message is added to the correction field of the corresponding SYNC or the FOLLOW_UP message.

For two step clocks, these two times (message transit time and upstream link delay time) are both added to the correction field of the PTP FOLLOW_UP messages, and the correction field of the message received by the slave contains the sum of all link delays. In theory this is the total end-to-end delay (from master to slave) of the SYNC packet.

Figure 5-4 illustrates PTP clocks in a master-slave hierarchy within a PTP network.

Figure 5-4 PTP Clock Hierarchy

About the PTP Power Profile

This section describes PTP profiles and specifically the PC37.238 IEEE-1588 standard, Power Profile, which is also known as the Profile for Protection Applications.

Note The switch documentation and CLI use the terms Power Profile and power profile mode when referring to this IEEE-1588 profile and its associated configuration values.

What are PTP Profiles?

The IEEE-1588 definition of a PTP profile is the set of allowed PTP features applicable to a device. A PTP profile is usually specific to a particular type of application or environment and defines the following values:

•Best master clock algorithm options

•Configuration management options

•Path delay mechanisms (peer delay or delay request-response)

•Range and default values of all PTP configurable attributes and data set members

•Transport mechanisms that are required, permitted, or prohibited

•Node types that are required, permitted, or prohibited

•Options that are required, permitted, or prohibited

Power Profile Description

The IEEE Power Profile defines specific or allowed values for PTP networks used in power substations. The defined values include the optimum physical layer, the higher level protocol for PTP messages, and the preferred best master clock algorithm. The Power Profile values ensure consistent and reliable network time distribution within substations, between substations, and across wide geographic areas.

The switch is optimized for PTP in these ways:

•Hardware— The switch uses FPGA and PHY for the PTP function. The PHY time stamps the Fast Ethernet and Gigabit Ethernet ports.

•Software—The switch default configuration is power profile mode. In this mode, the switch uses the configuration values defined in the IEEE-1588 Power Profile standard.

Table 5-1 lists the configuration values defined by the IEEE-1588 Power Profile.

Tagging Behavior for PTP Packets

Table 5-2 describes the switch tagging behavior in power profile mode.

Power Profile Mode

Note For detailed information about the IEEE-1588 Power Profile, refer to the "About the PTP Power Profile" section.

By default, the switch PTP configuration uses the values defined by the IEEE-1588 Power Profile and the switch is in power profile mode. In this mode:

•The PTP mode of transport is Layer 2.

•The supported transparent clock mode is peer-to-peer (P2P).

Table 5-1 lists the configuration values for the switch in power profile mode.

PTP Clock Modes Supported on the Switch

PTP synchronization behavior depends on the PTP clock mode that you configure on the switch. You can configure the switch for one of the following global modes.

See the "Guidelines and Limitations" section for guidelines for configuring each of the clock modes.

Boundary Clock Mode

A switch configured for boundary clock mode participates in selecting the best master clock on the subdomain, selecting from all clocks it can see, including itself. If the switch does not detect a more accurate clock than itself, then the switch becomes the master clock. If a more accurate clock is detected, then the switch synchronizes to that clock and becomes a slave clock.

After initial synchronization, the switch and the connected devices exchange PTP timing messages to correct the changes caused by clock offsets and network delays.

P2P Transparent Clock Mode

A switch configured for peer-to-peer transparent clock mode does not synchronize its clock with the master clock. A switch in this mode does not participate in master clock selection and uses the default PTP clock mode on all ports.

Prerequisites

Review the "Information About Precision Time Protocol" section and "Guidelines and Limitations" section.

Guidelines and Limitations

PTP Mode and Profile

•The switch and the grandmaster clock must be in the same PTP domain.

•When power profile mode is enabled, the switch drops the PTP announce messages that do not include these two Type, Length, Value (TLV) message extensions: Organization_extension and Alternate_timescale.

If the grandmaster clock is not compliant with PTP and sends announce messages without these TLVs, configure the switch to process the announce message by entering the ptp allow-without-tlv command.

Refer to the "Configuring PTP Power Profile Mode on the Switch" section for a complete description of this command.

•When the switch is in power profile mode, only the peer_delay mechanism is supported.

To change to Boundary Clock Mode and the peer_delay mechanism, enter the ptp mode boundary pdelay-req command.

Packet Format

•The packet format for PTP messages can be 802.1q tagged packets or untagged packets.

•The switch does not support 802.1q QinQ Tunneling.

•In switch power profile mode:

–When the PTP interface is configured as an access port, PTP messages are sent as untagged, Layer 2 packets.

–When the PTP interface is configured as a trunk port, PTP packets are sent as 802.1q tagged Layer 2 packets over the port native VLAN.

•Slave IEDs must support tagged and untagged packets.

VLAN Configuration

•Most grandmaster clocks use the default VLAN 0. In power profile mode, the switch default VLAN is VLAN 1 and VLAN 0 is reserved. When you change the default grandmaster clock VLAN, it must be changed to a VLAN other than 0.

•When VLAN is disabled on the grandmaster clock, the PTP interface must be configured as an access port.

Clock Configuration

•All PHY PTP clocks are synchronized to the grandmaster clock. The switch system clock is not synchronized as part of PTP configuration and processes.

•When VLAN is enabled on the grandmaster clock, it must be in the same VLAN as the native VLAN of the PTP port on the switch.

•Grandmaster clocks can drop untagged PTP messages when a VLAN is configured on the grandmaster clock. To force the switch to send tagged packets to the grandmaster clock, enter the global vlan dot1q tag native command.

Clock Modes

•Boundary Clock Mode

–You can enable this mode when the switch is in Power Profile Mode (Layer 2).

–The switch must be in this mode to configure PTP on individual switch ports.

•P2P Transparent Clock Mode

–You can enable this mode only when the switch is in Power Profile Mode (Layer 2).

–When the switch is in this mode, the only PTP configuration available is PTP mode.

–The switch must be in Boundary Clock Mode to configure PTP on individual switch ports.

Default Settings

•PTP is enabled on the switch by default.

•By default, the switch uses configuration values defined in the PTP Power Profile (power profile mode is enabled).

•The switch default PTP clock mode is P2P Transparent Clock Mode.

Configuring PTP on the Switch

This section describes how to configure the switch for PTP applications.

Configuring PTP Power Profile Mode on the Switch

This section describes how to configure the switch to use the PTP Power Profile and operate in power profile mode.

•For complete information about PTP profiles and the Power Profile, refer to the "About the PTP Power Profile" section.

•For details about switch power profile mode, refer to the "Power Profile Mode" section.

BEFORE YOU BEGIN

These are some guidelines for configuring the Power Profile on the switch:

•When you enter no with PTP port configuration commands, the specified port property is set to the default value.

•To determine the value in seconds for the ptp global command interval variable, use a logarithmic scale. Below are examples of the interval variable value converted to seconds with a logarithmic scale:

DETAILED STEPS

EXAMPLE

The following example configures the switch for P2P transparent mode, specifies allow-without-tlv PTP message processing, and uses default values for all PTP interval settings:

switch(config)# ptp allow-without-tlv

The following example configures the switch for boundary clock mode using the peer delay request (pdelay-req) mechanism and uses default values for all PTP interval settings:

switch(config)# ptp mode boundary pdelay-req

Verifying Configuration

EXAMPLE

switch# show ptp parent

 PTP PARENT PROPERTIES

  Parent Clock:

  Parent Clock Identity: 0xA4:C:C3:FF:FE:BF:B4:0

  Parent Port Number: 23

  Observed Parent Offset (log variance): N/A

  Observed Parent Clock Phase Change Rate: N/A

  Grandmaster Clock:

  Grandmaster Clock Identity: 0xA4:C:C3:FF:FE:BF:2B:0

  Grandmaster Clock Quality:

        Class: 248

        Accuracy: Unknown

        Offset (log variance): N/A

        Priority1: 128

        Priority2: 128

switch# show ptp clock

 PTP CLOCK INFO

  PTP Device Type: Boundary clock

  PTP Device Profile: Power Profile

  Clock Identity: 0xA4:C:C3:FF:FE:BF:E0:80

  Clock Domain: 0

  Number of PTP ports: 26

  PTP Packet priority: 4

  Priority1: 128

  Priority2: 128

  Clock Quality:

        Class: 248

        Accuracy: Unknown

        Offset (log variance): N/A

  Offset From Master(ns): 25

  Mean Path Delay(ns): 705

  Steps Removed: 4

  Local clock time: 14:23:56 PST Apr 5 2013

switch# show ptp foreign-master-record

 PTP FOREIGN MASTER RECORDS

 Interface FastEthernet0/1

   Empty

 Interface FastEthernet0/2

   Empty

 Interface FastEthernet0/3

   Empty

 Interface FastEthernet0/4

   Empty

 Interface FastEthernet0/5

   Empty

 Interface FastEthernet0/6

   Empty

 Interface FastEthernet0/7

   Empty

 Interface FastEthernet0/8

   Empty

 Interface FastEthernet0/9

   Empty

 Interface FastEthernet0/10

   Empty

 Interface FastEthernet0/11

   Empty

 Interface FastEthernet0/12

   Empty

 Interface FastEthernet0/13

   Empty

 Interface FastEthernet0/14

   Empty

 Interface FastEthernet0/15

   Empty

 Interface FastEthernet0/16

   Empty

 Interface FastEthernet0/17

   Empty

 Interface FastEthernet0/18

   Empty

 Interface FastEthernet0/19

   Empty

 Interface FastEthernet0/20

   Empty

 Interface FastEthernet0/21

   Empty

 Interface FastEthernet0/22

   Empty

 Interface FastEthernet0/23

   Empty

 Interface FastEthernet0/24

   Foreign master port identity: clock id: 0xA4:C:C3:FF:FE:BF:B4:0

   Foreign master port identity: port num: 23

   Number of Announce messages: 4

   Message received port: 24

   Time stamps: 2718923059, 2717917723

 Interface GigabitEthernet0/1

   Empty

 Interface GigabitEthernet0/2

   Empty

switch#

switch# show ptp ?

  clock                  show ptp clock information

  foreign-master-record  show PTP foreign master records

  parent                 show PTP parent properties

  port                   show PTP port properties

  time-property          show PTP clock time property

switch# show ptp time-property

 PTP CLOCK TIME PROPERTY

  Current UTC offset valid: 0

  Current UTC offset: 35

  Leap 59: 0

  Leap 61: 0

  Time Traceable: 16

  Frequency Traceable: 32

  PTP Timescale: 1

  Time Source: Internal Osciliator

  Time Property Persistence: 300 seconds

switch# show ptp port FastEthernet 0/23

 PTP PORT DATASET: FastEthernet0/23

  Port identity: clock identity: 0xA4:C:C3:FF:FE:BF:E0:80

  Port identity: port number: 23

  PTP version: 2

  Port state: MASTER

  Delay request interval(log mean): 5

  Announce receipt time out: 3

  Peer mean path delay(ns): 507

  Announce interval(log mean): 0

  Sync interval(log mean): 0

  Delay Mechanism: Peer to Peer

  Peer delay request interval(log mean): 0

  Sync fault limit: 500000000

switch# show ptp port FastEthernet 0/24

 PTP PORT DATASET: FastEthernet0/24

  Port identity: clock identity: 0xA4:C:C3:FF:FE:BF:E0:80

  Port identity: port number: 24

  PTP version: 2

  Port state: SLAVE

  Delay request interval(log mean): 5

  Announce receipt time out: 3

  Peer mean path delay(ns): 745

  Announce interval(log mean): 0

  Sync interval(log mean): 0

  Delay Mechanism: Peer to Peer

  Peer delay request interval(log mean): 0

  Sync fault limit: 500000000

switch#

Configuration Example

The following example configures the switch for P2P transparent mode, specifies allow-without-tlv PTP message processing, and uses default values for all PTP interval settings:

switch(config)# ptp allow-without-tlv

The following example configures the switch for boundary clock mode using the peer delay request (pdelay-req) mechanism and uses default values for all PTP interval settings:

switch(config)# ptp mode boundary pdelay-req