The Precision Time Protocol (PTP) is a time synchronization protocol defined in IEEE 1588 for nodes distributed across a network. With PTP, you can synchronize distributed clocks with an accuracy of less than 1 microsecond using Ethernet networks. PTP's accuracy comes from the hardware support for PTP in the Cisco Application Centric Infrastructure (ACI) fabric spine and leaf switches. The hardware support allows the protocol to compensate accurately for message delays and variation across the network.

Note	This document uses the term "client" for what the IEEE1588-2008 standard refers to as the "slave." The exception is instances in which the word "slave" is embedded in the Cisco Application Policy Infrastructure Controller (APIC) CLI commands or GUI.

PTP is a distributed protocol that specifies how real-time PTP clocks in the system synchronize with each other. These clocks are organized into a master-client synchronization hierarchy with the grandmaster clock, which is the clock at the top of the hierarchy, determining the reference time for the entire system. Synchronization is achieved by exchanging PTP timing messages, with the members using the timing information to adjust their clocks to the time of their master in the hierarchy. PTP operates within a logical scope called a PTP domain.

The PTP process consists of two phases: establishing the master-client hierarchy and synchronizing the clocks. Within a PTP domain, each port of an ordinary or boundary clock uses the following process to determine its state:

1. Establish the master-client hierarchy using the Best Master Clock Algorithm (BMCA):

   • Examine the contents of all received Announce messages (issued by ports in the master state).

   • Compare the data sets of the foreign master (in the Announce message) and the local clock for priority, clock class, accuracy, and so on.

   • Determine its own state as either master or client.

2. Synchronize the clocks:

   • Use messages, such as Sync and Delay_Req, to synchronize the clock between the master and clients.

• Latency requires all the nodes in the fabric to be synchronized using Precision Time Protocol (PTP).

• Latency measurement and PTP are supported only on second generation or later Cisco ACI switches. The following first generation Cisco ACI switches are not supported for latency measurement and PTP:

  • N9K-C9332PQ

  • N9K-C9336PQ

  • N9K-C9372PX

  • N9K-C9372PX-E

  • N9K-C9372TX

  • N9K-C9372TX-E

  • N9K-C9396PX

  • N9K-C9396TX

  • N9K-C93120TX

  • N9K-C93128TX

  • N9K-X9736PQ

• When the Cisco ACI fabric has mixed generation of switches, the latency can be measured only for packets that go through newer generation leaf and spine switches, which support latency measurement with PTP. This implies all spine switches need to be PTP capable switches because it is difficult to predict which spine forwards which traffic flow.

• In the presence of first generation leaf switches, it is recommended to use external grandmaster and have its connectivity to all the spine switches. Otherwise, the PTP messages from the grandmaster spine may be blocked by first generation switches before they reach all the switches. Using an external grandmaster connected to all spine switches ensures that the PTP messages from the external grandmaster reach all the switches even in the presence of first generation leaf switches.

• An external grandmaster clock is not mandatory, but we recommend one for high accuracy.

• To support latency measurements across Cisco ACI pods, all pods need to synchronize to the same clock. For this purpose, we recommend that you connect an external PTP grandmaster with a primary reference time clock (PRTC) such as GPS/GNSS to IPN. Configure the IPN nodes as PTP nodes such that each pod can synchronize to the grandmaster through IPN with almost the same number of hops. If a grandmaster with a PRTC is not available, you can use one of the IPN nodes or Cisco ACI switches as the grandmaster without a PRTC based on the Best Master Clock Algorithm (BMCA). By default, Cisco ACI switches are configured with PTP priority1 of 255, except for one spine switch in each pod that is configured with PTP priority1 of a value lower than the other Cisco ACI switches by one (that is, 254). To make sure that all pods synchronize to the same clock, IPN nodes must be configured as PTP nodes or at least must not block PTP messages from one pod to another.

• PTP operates only in boundary clock mode. End-to-end transparent clock and peer-to-peer transparent clock modes are not supported.

• PTP supports transport over User Datagram Protocol (UDP). Transport over Ethernet is not supported.

• PTP supports multicast communication only; unicast mode is not supported.

• Beginning with release 4.0(1), you can change the resolution factor to 11, which then can measure up to 214 milliseconds with an accuracy of 204ns.