Troubleshoot Precision Time Protocol (PTP) on Nexus 9000
Available Languages
Contents
Introduction
This document describes how to troubleshoot the Precision Time Protocol (PTP) on Nexus 9000 switches.
Prerequisites
Cisco recommends that you know these topics:
- Basic knowledge of PTP
- Familiar with the Cisco Nexus Operating System (NX-OS)
The design and configuration of PTP (Precision Time Protocol) are not covered in this article. For such information, it is recommended that you refer to the configuration guide.
Nexus 9000 PTP configuration guide
Precision Time Protocol (PTP) for Cisco Nexus Dashboard Insights
Components Used
This document is not restricted to specific software and hardware versions.
The information in this document is based on these software and hardware versions:
- N9K Spine01: N9K-C93180YC-FX NX-OS 10.3(4a)
- N9K Spine02: N9K-C93180YC-EX NX-OS 10.3(4a)
- N9K Leaf01: N9K-C92160YC-X NX-OS 9.3.12
- N9K Host: N9K-C92160YC-X NX-OS 9.3.12
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Restrictions and Limitations
-
For PTP to function properly, you must use the latest SUP and line card FPGA versions. For information on upgrading FPGAs, access the Release Notes landing page go to the FPGA/EPLD Upgrade Release Notes (NX-OS Mode Switches) section and locate the FPGA/EPLD Upgrade Release Notes for your software version. Refer to the Installation Guidelines topic.
- On Nexus 9000 PTP operates only in boundary clock mode. End-to-end transparent clock and peer-to-peer transparent clock modes are not supported.
-
PTP is not supported on the Cisco Nexus 92348GC-X platform switch.
-
The QoS TCAM region Ingress SUP [ingress-sup] must be set to 768 or higher for PTP IPv6 transport to work.
Before troubleshooting the PTP issue, it is recommended to review the PTP section of Nexus 9000 System Management Configuration for the given platform and version.
Understand PTP
The PTP process consists of two phases: establishing the primary-secondary hierarchy and synchronizing the clocks.
Best Master Clock(BMC) Algorithm
The BMCA is used to select the time source clock on each link, and it ultimately selects the grandmaster clock for the whole PTP domain. It runs locally on each port of the ordinary and boundary clocks to compare the local data sets with the received data from Announce messages to select the best clock on the link.
- Priority1: User configurable absolute priority (lower value wins)
- Clock Class: Attribute defining clock’s traceability (not user configurable, lower value wins)
- Clock Accuracy: Defines the accuracy of a clock (not user configurable, lower value wins)
- Clock Variance: Attribute defining the precision of a clock (not user configurable)
- Priority2: User configurable
- Source Port ID: Mac address of source port
Announce messages are used to establish the synchronization hierarchy.
Synchronizing the clock
Sync, Delay_Req, Follow_Up, and Delay_Resp messages are used to calculate time.
PTP messages can be either multicast or unicast, with multicast being the default mode. PTP uses multicast destination IP address 224.0.1.129 UDP319/320 as per IEEE 1588 standards.
PTP Profiles — PTP supports default (1588), AES67, and SMPTE 2059-2 profiles. Each of these profiles has different ranges of sync and delay request intervals. For more information about these profiles, please refer to the configuration guide.
Lab Topology
Basic Configuration:
feature ptp
ptp source 192.168.1.3>>>>Define PTP packet source IP
ptp priority1 127 >>>>Define PTP priority 1
ptp priority2 127 >>>>Define PTP priority 2
interface Ethernet1/31
ptp >>>>Enable PTP in all interconnected ports.
interface Ethernet1/33
ptp
interface Ethernet1/34
ptp
Troubleshoot Steps:
Verify that PTP is Configured Correctly.
Ensure each device has a unique source IP and the PTP domain ID is the same on all devices.
N9K_Spine01# show ptp clock
PTP Device Type : boundary-clock
PTP Source IPv4 Address : 192.168.1.3>>>>PTP source IP
PTP Source IPv6 Address : 0::
Clock Identity : 00:ee:ab:ff:fe:67:3e:9d
Clock Domain: 0 >>>>PTP domain id. Must same in one PTP domain.
Slave Clock Operation : Two-step
Master Clock Operation : Two-step >>>>N9K EX/FX/FX2/FX3 only support two-step mode.
Slave-Only Clock Mode : Disabled
Number of PTP ports: 3
Priority1 : 127
Priority2 : 127
Clock Quality:
Class : 248
Accuracy : 254
Offset (log variance) : 65535
Steps removed : 1 >>>>Hops from GM
Verify PTP Hierarchy Setup Successful as Your Design.
The slave port is connected to the upstream clock device. The master port is connected to the downstream device.
N9K_Spine01# show ptp brief
PTP port status
--------------------------------------------
Port State
-------------------------------------------
Eth1/31 Slave >>>>Connected to GM
Eth1/33 Master >>>>Connected to N9K Spine 2
Eth1/34 Master >>>>Connected to N9K leaf
N9K_Spine02# show ptp brief
PTP port status
--------------------------------------------
Port State
------------------------------- ------------
Eth1/48 Passive >>>>Connected to N9K leaf. The Port should be in the passive state to avoid loop
Eth1/50 Master >>>>Connected to GM02
Eth1/53 Slave >>>>Connected to N9K Spine 1
N9K_Leaf01# show ptp brief
PTP port status
--------------------------------------------
Port State
------------------------------- ------------
Eth1/48 Master >>>>Connected to Spine02
Eth1/50 Master >>>>Connected to host
Eth1/51 Slave >>>>Connected to Spine01
GM01# show system internal ptp info announce-pkts
2024-01-02T13:36:23.242624000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [TX] If Eth1/35 (0x1a004400): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:37:e9-0x89 CORR:0 (0x0) SEQ:bda4 (48548) Int:1; TS
:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:0 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
2024-01-02T13:36:15.238816000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [TX] If Eth1/35 (0x1a004400): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:37:e9-0x89 CORR:0 (0x0) SEQ:bda0 (48544) Int:1; TS
:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:0 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
N9K_Spine01# show system internal ptp info announce-pkts
2024-01-02T13:36:20.826735000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [TX] If Eth1/33 (0x1a004000): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:3e:9d-0x81 CORR:0 (0x0) SEQ:bbfc (48124) Int:1; TS
:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:1 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
2024-01-02T13:36:17.231080000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [TX] If Eth1/34 (0x1a004200): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:3e:9d-0x85 CORR:0 (0x0) SEQ:bdac (48556) Int:1; TS
:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:1 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
2024-01-02T13:36:16.239728000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [RX] If Eth1/31 (0x1a003c00): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:37:e9-0x89 CORR:0 (0x0) SEQ:bda1 (48545) Int:1; TS
:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:0 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
N9K_Spine02# show system internal ptp info announce-pkts
2024-01-02T13:36:21.368978000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [RX] If Eth1/48 (0x1a005e00): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:4c:77:6d:ff:fe:d5:35:35-0xbd CORR:0 (0x0) SEQ:bcd7 (48343) Int:1; TS
:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
2024-01-02T13:36:19.363095000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [TX] If Eth1/50 (0x1a006200): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:c0:14:fe:ff:fe:89:9b:77-0xc5 CORR:0 (0x0) SEQ:00e8 (232) Int:1; TS:0
s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
2024-01-02T13:36:16.828573000+08:00 [M 1] [ptp] E_DEBUG [ptp_ev_hist_ann_pkt:5469] [RX] If Eth1/53 (0x1a006800): MSG: Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:3e:9d-0x81 CORR:0 (0x0) SEQ:bbfa (48122) Int:1; TS
:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:1 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9;
N9K_Leaf01# show system internal ptp info announce-pkts
2024 Jan 02 13:36:23.893622: E_PTP_ANN_PKT_EV[TX] I/f Eth1/50 (0x1a006200): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:4c:77:6d:ff:fe:d5:35:35-0xc5 CORR:0 (0x0) SEQ:1354 (4948) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2
PRIO1:1 PRIO2:1
CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9
2024 Jan 02 13:36:23.369089: E_PTP_ANN_PKT_EV[TX] I/f Eth1/48 (0x1a005e00): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:4c:77:6d:ff:fe:d5:35:35-0xbd CORR:0 (0x0) SEQ:bcd8 (48344) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2
PRIO1:1 PRIO2:1
CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9
2024 Jan 02 13:36:23.233889: E_PTP_ANN_PKT_EV[RX] I/f Eth1/51 (0x1a006400): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:3e:9d-0x85 CORR:0 (0x0) SEQ:bdaf (48559) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:1
PRIO1:1 PRIO2:1
CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9
Host# show system internal ptp info announce-pkts
2024 Jan 02 13:36:23.898218: E_PTP_ANN_PKT_EV[RX] I/f Eth1/50 (0x1a006200): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:4c:77:6d:ff:fe:d5:35:35-0xc5 CORR:0 (0x0) SEQ:1354 (4948) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2
PRIO1:1 PRIO2:1
CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9
|
V:2 |
PTP version 2 |
|
LEN:64 |
PTP message length 64 |
|
D:0 |
PTP domain 0 |
|
UC:0 |
0: PTP multicast packet,1:PTP unicast packet |
|
2S:0 |
maybe leap 59/61 |
|
UTCVAL:0 |
UTC offset valid flag. 0 means false. GM set it. |
|
PTPTS:1 |
PTP TimeScale flag. 1 means true |
|
TT:0 |
PTP TimeTrace flag. 0 means false |
|
FT:0 |
PTP FreqTrace flag .0 means false |
|
SRC: |
PTP packet source mac |
|
CORR:0 |
Correction |
|
SEQ: |
PTP sequence ID |
|
INT:1 |
Log message period. 1 mean 2s |
|
TS: |
Timestamps |
|
UTC_OFF |
UTC offset value. GM set this value. |
|
TM_SRC |
0x20 GPS, 0x40 PTP, 0x50 NTP, 0x60 Hand_set 0xa internal oscillator. GM set this value. |
|
STEP:2 |
Clock operation in two-step mode(N9K only supports two steps mode on the primary port) |
|
PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff |
GM priority, GM clock class, GM clock accuracy |
|
GM |
GM Clock Identity. Come from mac address. |
Verify Parent and Grandmaster Information in the PTP Domain
Ensure the parent clock device and grandmaster device are stable.
N9K_Spine01# show ptp parent Parent Clock: Parent Clock Identity: 00:ee:ab:ff:fe:67:37:e9 >>>>upstream clock identity. 37:e9 is GM in lab topology Parent Port Number: 137 Observed Parent Offset (log variance): N/A Observed Parent Clock Phase Change Rate: N/A Parent IP: 192.168.1.1 >>>>upstream clock source IP Grandmaster Clock: Grandmaster Clock Identity: 00:ee:ab:ff:fe:67:37:e9 >>>>GM clock identity Grandmaster Clock Quality: >>>>GM clock attributes Class: 248 Accuracy: 254 Offset (log variance): 65535 Priority1: 1 >>>>GM priority1 Priority2: 1
N9K_Spine02# show ptp parent Parent Clock: Parent Clock Identity: 00:ee:ab:ff:fe:67:3e:9d>>>upstream clock identity. 37:9d is N9K Spine01 in lab topology Parent Port Number: 129 Observed Parent Offset (log variance): N/A Observed Parent Clock Phase Change Rate: N/A Parent IP: 192.168.1.3 >>>>upstream clock source IP. 192.168.1.3 is N9K Spine01 Grandmaster Clock: Grandmaster Clock Identity: 00:ee:ab:ff:fe:67:37:e9 Grandmaster Clock Quality: Class: 248 Accuracy: 254 Offset (log variance): 65535 Priority1: 1 Priority2: 1
N9K_Leaf01# show ptp parent PTP PARENT PROPERTIES Parent Clock: Parent Clock Identity: 00:ee:ab:ff:fe:67:3e:9d Parent Port Number: 133 Observed Parent Offset (log variance): N/A Observed Parent Clock Phase Change Rate: N/A Parent IP: 192.168.1.3 Grandmaster Clock: Grandmaster Clock Identity: 00:ee:ab:ff:fe:67:37:e9
Grandmaster Clock Quality: Class: 248 Accuracy: 254 Offset (log variance): 65535 Priority1: 1 Priority2: 1
Verify PTP Correction and Bad Correction.
The Sync-SeqID must increase with each entry. The correction must be less than 10000 nanoseconds (ns).
N9K_Spine02# show system internal ptp corrections PTP past corrections ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Slave Port SUP Time Correction(ns) MeanPath Delay(ns) MasterTimestamp (sec, nsec) Slave Timestamp (sec, nsec) Sync-SeqID PTPLC ts_corr(ns)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Eth1/53 Wed Jan 3 15:29:05 2024 15928 -8 204 1704266945 14551936 1704266945 14552148 31871 0 Eth1/53 Wed Jan 3 15:29:04 2024 765051 24 204 1704266944 764413040 1704266944 764413220 31870 0 Eth1/53 Wed Jan 3 15:29:04 2024 509436 24 204 1704266944 513679576 1704266944 513679756 31869 0 Eth1/53 Wed Jan 3 15:29:04 2024 264139 0 204 1704266944 263204688 1704266944 263204892 31868 0 Eth1/53 Wed Jan 3 15:29:04 2024 13239 -8 204 1704266944 12722952 1704266944 12723164 31867 0 Eth1/53 Wed Jan 3 15:29:03 2024 762756 24 212 1704266943 761756520 1704266943 761756708 31866 0
Bad Correction Record
By default, the correction threshold is 100000 nanoseconds (100us). Corrections not within this range are recorded as bad corrections.
N9K_Spine02(config)# show system internal ptp bad-corrections PTP past corrections ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Slave Port SUP Time Correction(ns) MeanPath Delay(ns) MasterTimestamp (sec, nsec) Slave Timestamp (sec, nsec) Sync-SeqID PTPLC ts_corr(ns)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Eth1/48 Tue Jan 2 13:28:30 2024 692911 17111776 172 1704173310 705666212 1704173310 688554608 52942 0 Eth1/48 Tue Jan 2 13:28:30 2024 443146 17111808 172 1704173310 454735796 1704173310 437624160 52941 0 Eth1/48 Tue Jan 2 13:28:30 2024 188850 17111784 172 1704173310 203243428 1704173310 186131816 52940 0 Eth1/48 Tue Jan 2 13:28:29 2024 949432 51292504 172 1704173309 952419348 1704173309 901127016 52939 0
Useful Collection:
show running-config ptp
show ptp brief
show ptp counters all
show ptp clock
show system internal ptp info all
show system internal ptp info global
show ptp clock foreign-masters record
show system internal ptp corrections entries 2000
show system internal ptp bad-corrections entries 2000
show system internal ptp trouble-shooting all
show tech ptp
Common Issues:
Nexus 9000 Cannot Sync Time with Grandmaster or Upstream Boundary Clock
In most cases, these are configuration issues.
Actions to Take:
- Check if the PTP domain number is the same on all PTP-enabled devices. Ensure the unique PTP source IP is configured on all devices.
show ptp clock
TP Device Type : boundary-clock
PTP Device Encapsulation : NA
PTP Source IP Address : 192.168.1.4
Clock Identity : c0:14:fe:ff:fe:89:9b:77
Clock Domain: 0
Slave Clock Operation : Two-step
Master Clock Operation : Two-step
<snip>
Local clock time : Thu Jan 4 19:34:26 2024
PTP Clock state : Locked
- Verify that PTP is enabled on the interface. By default, it is disabled.
N9K_Spine02# show ptp brief
--------------------------------------------
Port State
------------------------------- ------------
Eth1/48 Passive >>>>Connected to N9K leaf. Port in the passive state to prevent loop
Eth1/50 Master >>>>Connected to GM02
Eth1/53 Slave >>>>Connected to N9K Spine 1
- Check PTP interface parameters. Ensure that the same PTP VLAN as the peer is being used.
N9K_Spine02# show ptp port interface e1/48
PTP Port Dataset: Eth1/48
Port identity: clock identity: c0:14:fe:ff:fe:89:9b:77
Port identity: port number: 188
PTP version: 2
Port state: Master
VLAN info: 1
Delay request interval(log mean): 0
Announce receipt time out: 3
Peer mean path delay: 0
Announce interval(log mean): 1
Sync interval(log mean): -2
Delay Mechanism: End to End
Cost: 255
Domain: 0
Unexpected Grandmaster Switchover
2024 Jan 4 19:27:05 N9K_Spine02 %PTP-2-PTP_GM_CHANGE: Grandmaster clock has changed from 00:ee:ab:ff:fe:67:37:e9 to c0:14:fe:ff:fe:a3:c4:67 for the PTP protocol
Actions to Take
- Check the PTP announce history for changes in priority or other clock attribute changes.
show system internal ptp info announce-pkts
2024 Jan 04 19:27:07.408293: E_PTP_ANN_PKT_EV[RX] I/f Eth1/48 (0x1a005e00): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:4c:77:6d:ff:fe:d5:35:35-0xbd CORR:0 (0x0) SEQ:3673 (13939) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9 2024 Jan 04 19:27:06.321569: E_PTP_ANN_PKT_EV[RX] I/f Eth1/50 (0x1a006200): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:c0:14:fe:ff:fe:a3:c4:67-0xc9 CORR:0 (0x0) SEQ:0005 (5) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:0 PRIO1:0 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:c0:14:fe:ff:fe:a3:c4:67 2024 Jan 04 19:27:05.427431: E_PTP_ANN_PKT_EV[RX] I/f Eth1/53 (0x1a006800): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:00:ee:ab:ff:fe:67:3e:9d-0x81 CORR:0 (0x0) SEQ:3593 (13715) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:1 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9 2024 Jan 04 19:27:05.407196: E_PTP_ANN_PKT_EV[RX] I/f Eth1/48 (0x1a005e00): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:4c:77:6d:ff:fe:d5:35:35-0xbd CORR:0 (0x0) SEQ:3672 (13938) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9 2024 Jan 04 19:27:04.822821: E_PTP_ANN_PKT_EV[TX] I/f Eth1/50 (0x1a006200): MSG:Announce TS:0 V:2 LEN:64 D:0 UC:0 2S:0 UTCVAL:0 PTPTS:1 TT:0 FT:0 SRC:c0:14:fe:ff:fe:89:9b:77-0xc4 CORR:0 (0x0) SEQ:0506 (1286) Int:1; TS:0 s 0 ns; UTC_OFF:37 TM_SRC:a0 STEP:2 PRIO1:1 PRIO2:1 CLASS:248 ACC:fe LOG_VAR:ffff GM:00:ee:ab:ff:fe:67:37:e9
High Bad Correction
A random bad correction can be challenging to analyze due to a lack of data. Nexus 9000 provides an auto-log feature to capture PTP logs in the backend without impacting performance.
Actions to take:
1. Identify the bad corrections.
N9K_Spine02# show system internal ptp bad-corrections entries 2000 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Slave Port SUP Time Correction(ns) MeanPath Delay(ns) MasterTimestamp (sec, nsec) Slave Timestamp (sec, nsec) Sync-SeqID PTPLC ts_corr(ns)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Eth1/48 Thu Jan 4 18:41:07 2024 140073 19167640 172 1704364867 151916416 1704364867 132748948 28473 0 Eth1/48 Thu Jan 4 18:41:06 2024 889689 19167624 172 1704364866 901509720 1704364866 882342268 28472 0 Eth1/48 Thu Jan 4 18:41:06 2024 634900 19167604 172 1704364866 651177596 1704364866 632010164 28471 0 Eth1/48 Thu Jan 4 18:41:06 2024 386534 19167636 172 1704364866 399944588 1704364866 380777124 28470 0 Eth1/48 Thu Jan 4 18:41:05 2024 732409 425695900 172 1704364866 148959160 1704364865 723263432 28469 0 Eth1/48 Thu Jan 4 18:41:05 2024 480431 425695932 172 1704364865 898386456 1704364865 472690696 28468 0 Eth1/48 Thu Jan 4 18:41:05 2024 225514 425695908 172 1704364865 647736096 1704364865 222040360 28467 0 Eth1/48 Thu Jan 4 18:41:04 2024 977564 425695924 172 1704364865 396727464 1704364864 971031712 28466 0
2. Enable PTP Auto Log
test system internal ptp auto-log correction-limit 10000 >> Set a threshold of correction to trigger if the switch starts log collection. In this case, it’s 10000ns
test system internal ptp auto-log file-max-count 5 >> Maximum Auto-log files quantity
no test system internal ptp auto-log file-rollover >> Disable auto-log rollover
test system internal ptp auto-log >> Start auto-log in backend
3. In case a PTP bad correction occurs, the PTP log is created in bootflash.
N9K_Spine02# dir bootflash:
4096 Jan 04 19:57:44 2024 ptp_autolog/
N9K_Spine02# dir ptp_autolog
1115095 Jan 04 19:27:06 2024 auto_ptp_dbg_log_1.log
1099741 Jan 04 19:57:43 2024 auto_ptp_dbg_log_2.log
53631 Jan 04 19:57:43 2024 auto_ptp_dbg_log_3.log
87478 Jan 04 19:57:44 2024 auto_ptp_dbg_log_4.log
In this file, you can locate T1-T4 to perform calculations.
19:26:56 056993 ptp_calc_mean_path_delay t1/m sec 1704367616 ns 54142980 t2/s sec 1704367616 ns 54143180 t3/ds sec 1704367616 ns 54888476 t4/dm sec 1704367616 ns 54888676 diff corr 0 ns
19:26:57 060081 ptp_calc_mean_path_delay t1/m sec 1704367617 ns 56716444 t2/s sec 1704367617 ns 56716636 t3/ds sec 1704367617 ns 57466060 t4/dm sec 1704367617 ns 57466284 diff corr 0 ns
19:26:58 062591 ptp_calc_mean_path_delay t1/m sec 1704367618 ns 59552956 t2/s sec 1704367618 ns 59553164 t3/ds sec 1704367618 ns 60248804 t4/dm sec 1704367618 ns 60249012 diff corr 0 ns
19:26:59 061974 ptp_calc_mean_path_delay t1/m sec 1704367619 ns 61891376 t2/s sec 1704367619 ns 61891632 t3/ds sec 1704367619 ns 62503968 t4/dm sec 1704367619 ns 62504128 diff corr 0 ns
PTP Port in the Master State when it is Supposed to be a Slave or Passive
A PTP port changes to the primary state if it encounters a message exchange issue on the RX (receive) path side.
Actions to take:
- Check if the PTP RX (receive) counters are increasing on the problematic
masterport.
N9K_Spine01# show ptp counters all PTP Packet Counters of Interface Eth1/31: ---------------------------------------------------------------- Packet Type TX RX ---------------- -------------------- -------------------- Announce 0 3 Sync 0 21 FollowUp 0 21 Delay Request 5 0 Delay Response 0 5
- If it is not increasing, then check the SUP redirect ACL (Access Control List) statistics.
N9K_Spine01# show system internal access-list sup-redirect-stats | in PTP|Slice
Instance: 0 [Unit: 0 Slice: 0]
3118 PTP EVENT REDIRECT 3358695
3119 ETH PTP EVENT TX TIMESTAMP 0
3120 PTP EVENT TX TIMESTAMP 5046146
3167 PTP MSG REDIRECT 3088156
3183 PTP UNICAST MSG REDIRECT 0
3184 PTP UNICAST EVENT REDIRECT 0
Instance: 1 [Unit: 0 Slice: 1]
3118 PTP EVENT REDIRECT 0
3119 ETH PTP EVENT TX TIMESTAMP 0
3120 PTP EVENT TX TIMESTAMP 0
3167 PTP MSG REDIRECT 0
3183 PTP UNICAST MSG REDIRECT 0
3184 PTP UNICAST EVENT REDIRECT 0 - Check if Control Plane Policing (CoPP) is dropping PTP messages. If there is a drop counter here, then verify your scale.
N9K_Spine01# show policy-map interface control-plane class copp-system-p-class-redirect
Service-policy input: copp-system-p-policy-strict
class-map copp-system-p-class-redirect (match-any)
match access-group name copp-system-p-acl-ptp
match access-group name copp-system-p-acl-ptp-l2
match access-group name copp-system-p-acl-ptp-uc
set cos 1
police cir 280 kbps , bc 32000 bytes
module 1 :
transmitted 875343860 bytes;
5-minute offered rate 1650 bytes/sec
conformed 1932 peak-rate bytes/sec
at Thu Jan 04 22:08:20 2024
dropped 0 bytes; >>>> Check if any counter increasing
5-min violate rate 0 byte/sec
violated 0 peak-rate byte/sec Best Practice
- Ensure that everything is in the same PTP domain.
- Sync, Announce, and Delay intervals must match on both ends of the link.
- The CLI command ensures that the leaf access port stays in the
masterstate even if the recipient is misconfigured or a grandmaster is accidentally connected under this port:
interface Ethernet1/1
ptp multicast master-only - Check the verified scalability guide for the maximum number of
masterports per model of a switch.
Related Information
Nexus 9000 PTP Configuration Guide
Precision Time Protocol (PTP) for Cisco Nexus Dashboard Insights
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
1.0 |
07-Feb-2024 |
Initial Release |
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)