Segment Routing Configuration Guide for Cisco 8000 Series Routers, IOS XR Release 7.11.x
Bias-Free Language
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
Network performance metrics such as packet loss, delay, delay variation, and bandwidth utilization is a critical measure for
traffic engineering (TE) in service provider networks. These network performance metrics provide network operators information
about the performance characteristics of their networks for performance evaluation and helps to ensure compliance with service
level agreements. The service-level agreements (SLAs) of service providers depend on the ability to measure and monitor these
network performance metrics. Network operators can use performance measurement (PM) feature to monitor the network metrics
for links as well as end-to-end TE label switched paths (LSPs).
The following table explains the functionalities supported by performance measurement feature for measuring delay for links or SR policies.
Table 1. Performance Measurement Functionalities
Functionality
Details
Profiles
You can configure different profiles for different types of delay measurements. Delay profile type interfaces is used for
link-delay measurement. Delay profile type sr-policy is used for SR policy delay measurements. Delay profile allows you to schedule probe and configure metric advertisement parameters for delay measurement.
Protocols
Protocols for delay measurement probes can be MPLS (using RFC6374 with MPLS encap) or TWAMP Light (using RFC 5357 with IP/UDP
encap).
Probe and burst scheduling
Schedule probes and configure metric advertisement parameters for delay measurement.
Metric advertisements
Advertise measured metrics periodically using configured thresholds. Also supports accelerated advertisements using configured
thresholds.
Measurement history and counters
Maintain packet delay and loss measurement history and also session counters and packet advertisement counters.
Measurement Modes
The following table compares the different hardware and timing requirements for the measurement modes supported in SR PM.
Feature Name
Release
Description
SR Performance Measurement: Loopback Measurement Mode
Release 7.5.2
Loopback measurement mode provides two-way and one-way measurements. PTP-capable hardware and hardware timestamping are required
on the Sender but are not required on the Reflector.
Liveness monitoring uses "self-addressed" PM IP packets crafted by the sender (where the destination address is the sender's
own IP address); this mode of operation is referred as "loopback mode”.
Table 2. Measurement Mode Requirements
Measurement Mode
Sender:
PTP-Capable HW and HW Timestamping
Reflector:
PTP-Capable HW and HW Timestamping
PTP Clock Synchronization between Sender and Reflector
One-way
Required
Required
Required
Two-way
Required
Required
Not Required
Loopback
Required
Not Required
Not Required
One-Way Measurement Mode
One-way measurement mode provides the most precise form of one-way delay measurement. PTP-capable hardware and hardware timestamping
are required on both Sender and Reflector, with PTP Clock Synchronization between Sender and Reflector.
Delay measurement in one-way mode is calculated as (T2 – T1).
The PM query and response for one-way delay measurement can be described in the following steps:
The local-end router sends PM query packets periodically to the remote side once the egress line card on the router applies
timestamps on packets.
The ingress line card on the remote-end router applies time-stamps on packets as soon as they are received.
The remote-end router sends the PM packets containing time-stamps back to the local-end router.
One-way delay is measured using the time-stamp values in the PM packet.
Two-Way Measurement Mode
Two-way meaurement mode provides two-way measurements. PTP-capable hardware and hardware timestamping are required on both
Sender and Reflector, but PTP clock synchronization between Sender and Reflector is not required.
Delay measurement in two-way mode is calculated as ((T4 – T1) – (T3 – T2))/2.
The PM query and response for two-way delay measurement can be described in the following steps:
The local-end router sends PM query packets periodically to the remote side once the egress line card on the router applies
timestamps on packets.
Ingress line card on the remote-end router applies time-stamps on packets as soon as they are received.
The remote-end router sends the PM packets containing time-stamps back to the local-end router. The remote-end router time-stamps
the packet just before sending it for two-way measurement.
The local-end router time-stamps the packet as soon as the packet is received for two-way measurement.
Delay is measured using the time-stamp values in the PM packet.
Loopback Measurement Mode
Loopback meaurement mode provides two-way and one-way measurements. PTP-capable hardware and hardware timestamping are required
on the Sender, but are not required on the Reflector.
Delay measurements in Loopback mode are calculated as follows:
Round-Trip Delay = (T4 – T1)
One-Way Delay = Round-Trip Delay/2
The PM query and response for Loopback delay measurement can be described in the following steps:
The local-end router sends PM probe packets periodically on the SR Policy.
The probe packets are loopback on the endpoint node (not punted), with no timestamping on endpoint node.
Round-trip Delay = T4 – T1.
Usage Guidelines and Limitations
The following usage guidelines and limitations apply:
SR PM is supported on hardware that supports Precision Time Protocol (PTP). This requirement applies to both one-way and two-way
delay measurement.
See the "Configuring Precision Time Protocol" chapter in the System Management Configuration Guide for Cisco 8000 Series Routers for Restrictions for PTP and the Timing Hardware Support Matrix.
Link Delay Measurement
Table 3. Feature History Table
Feature Name
Release Information
Feature Description
Link Delay Measurement using TWAMP Light Encoding
Release 7.3.1
The PM for link delay uses the IP/UDP packet format defined in RFC 5357 (TWAMP-Light) for probes. Two-Way Active Measurement
Protocol (TWAMP) adds two-way or round-trip measurement capabilities. TWAMP employs time stamps applied at the echo destination
(reflector) to enable greater accuracy.
The PM for link delay uses the IP/UDP packet format defined in RFC 5357 (TWAMP-Light) for probes. Two-Way Active Measurement
Protocol (TWAMP) adds two-way or round-trip measurement capabilities. TWAMP employs time stamps applied at the echo destination
(reflector) to enable greater accuracy. In the case of TWAMP Light, the Session-Reflector doesn’t necessarily know about the
session state. The Session-Reflector simply copies the Sequence Number of the received packet to the Sequence Number field
of the reflected packet. The controller receives the reflected test packets and collects two-way metrics. This architecture
allows for collection of two-way metrics.
The following figure explains the PM query and response for link delay.
The PM query and response for link delay can be described in the following steps:
The local-end router sends PM query packets periodically to the remote side once the egress line card on the router applies
timestamps on packets.
Ingress line card on the remote-end router applies time-stamps on packets as soon as they are received.
The remote-end router sends the PM packets containing time-stamps back to the local-end router. The remote-end router time-stamps
the packet just before sending it for two-way measurement.
The local-end router time-stamps the packet as soon as the packet is received for two-way measurement.
One-way delay and optionally two-way delay is measured using the time-stamp values in the PM packet.
Restrictions and Usage Guidelines for PM for Link Delay
The following restrictions and guidelines apply for the PM for link delay feature for different links.
For LSPs, remote-end line card needs to be MPLS and multicast MAC address capable.
For broadcast links, only point-to-point (P2P) links are supported. P2P configuration on IGP is required for flooding the
value.
For link bundles, the hashing function may select a member link for forwarding but the reply may come from the remote line
card on a different member link of the bundle.
For one-way delay measurement, clocks should be synchronized on two end-point nodes of the link using PTP.
Link delay measurement is supported on IPv4 unnumbered interfaces. An IPv4 unnumbered interface is identified by a node ID
(a loopback address) and the local SNMP index assigned to the interface. Note that the reply messages could be received on
any interface, since the packets are routed at the responder based on the loopback address used to identify the link.
Configuration Example: PM for Link Delay
This example shows how to configure performance-measurement functionalities for link delay as a global default profile. The
default values for the different parameters in the PM for link delay is given as follows:
probe measurement mode: The default measurement mode for probe is two-way delay measurement. If you are configuring one-way delay measurement, hardware
clocks must be synchronized between the local-end and remote-end routers using precision time protocol (PTP).
protocol: Interface delay measurement uses TWAMP-Light (RFC5357) with IP/UDP encap.
tx-interval: Interval for sending probe packet. The default value is 3000000 microseconds and the range is from 3300 to 15000000 microseconds.
computation interval: Interval for metric computation. Default is 30 seconds; range is 1 to 3600 seconds.
periodic advertisement: Periodic advertisement is enabled by default.
periodic-advertisement interval: The default value is 120 seconds and the interval range is from 30 to 3600 seconds.
periodic-advertisement threshold: The default value of periodic advertisement threshold is 10 percent.
periodic-advertisement minimum change: The default value is 1000 microseconds (usec) and the range is from 0 to 10000 microseconds.
accelerated advertisement: Accelerated advertisement is disabled by default.
accelerated-advertisement threshold: The default value is 20 percent and the range is from 0 to 100 percent.
accelerated-advertisement minimum change: The default value is 1000 microseconds and the range is from 1 to 100000 microseconds.
Configuring the UDP port for TWAMP-Light protocol is optional. By default, PM uses port 862 as the TWAMP-reserved UDP destination
port.
The UDP port is configured for each PM measurement probe type (delay, loss, protocol, authentication mode, etc.) on querier
and responder nodes. If you configure a different UDP port, the UDP port for each PM measurement probe type must match on
the querier and the responder nodes.
Note
The same UDP destination port is used for delay measurement for links and SR Policy.
This example shows how to configure the UDP destination port.
RP/0/0/CPU0:router# show performance-measurement summary detail location 0/2/CPU0
Thu Dec 12 14:09:59.162 PST
-------------------------------------------------------------------------------
0/2/CPU0
-------------------------------------------------------------------------------
Total interfaces : 1
Total SR Policies : 0
Total RSVP-TE tunnels : 0
Total Maximum PPS : 2000 pkts/sec
Total Interfaces PPS : 0 pkts/sec
Maximum Allowed Multi-hop PPS : 2000 pkts/sec
Multi Hop Requested PPS : 0 pkts/sec (0% of max allowed)
Dampened Multi Hop Requested PPS : 0% of max allowed
Inuse Burst Interval Adjustment Factor : 100% of configuration
Interface Delay-Measurement:
Total active sessions : 1
Counters:
Packets:
Total sent : 26
Total received : 26
Errors:
TX:
Reason interface down : 0
Reason no MPLS caps : 0
Reason no IP address : 0
Reason other : 0
RX:
Reason negative delay : 0
Reason delay threshold exceeded : 0
Reason missing TX timestamp : 0
Reason missing RX timestamp : 0
Reason probe full : 0
Reason probe not started : 0
Reason control code error : 0
Reason control code notif : 0
Probes:
Total started : 3
Total completed : 2
Total incomplete : 0
Total advertisements : 0
SR Policy Delay-Measurement:
Total active sessions : 0
Counters:
Packets:
Total sent : 0
Total received : 0
Errors:
TX:
Reason interface down : 0
Reason no MPLS caps : 0
Reason no IP address : 0
Reason other : 0
RX:
Reason negative delay : 0
Reason delay threshold exceeded : 0
Reason missing TX timestamp : 0
Reason missing RX timestamp : 0
Reason probe full : 0
Reason probe not started : 0
Reason control code error : 0
Reason control code notif : 0
Probes:
Total started : 0
Total completed : 0
Total incomplete : 0
Total advertisements : 0
RSVP-TE Delay-Measurement:
Total active sessions : 0
Counters:
Packets:
Total sent : 0
Total received : 0
Errors:
TX:
Reason interface down : 0
Reason no MPLS caps : 0
Reason no IP address : 0
Reason other : 0
RX:
Reason negative delay : 0
Reason delay threshold exceeded : 0
Reason missing TX timestamp : 0
Reason missing RX timestamp : 0
Reason probe full : 0
Reason probe not started : 0
Reason control code error : 0
Reason control code notif : 0
Probes:
Total started : 0
Total completed : 0
Total incomplete : 0
Total advertisements : 0
Global Delay Counters:
Total packets sent : 26
Total query packets received : 26
Total invalid session id : 0
Total missing session : 0
RP/0/0/CPU0:router# show performance-measurement interfaces detail
Thu Dec 12 14:16:09.692 PST
-------------------------------------------------------------------------------
0/0/CPU0
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
0/2/CPU0
-------------------------------------------------------------------------------
Interface Name: GigabitEthernet0/2/0/0 (ifh: 0x1004060)
Delay-Measurement : Enabled
Loss-Measurement : Disabled
Configured IPv4 Address : 10.10.10.2
Configured IPv6 Address : 10:10:10::2
Link Local IPv6 Address : fe80::3a:6fff:fec9:cd6b
Configured Next-hop Address : Unknown
Local MAC Address : 023a.6fc9.cd6b
Next-hop MAC Address : 0291.e460.6707
Primary VLAN Tag : None
Secondary VLAN Tag : None
State : Up
Delay Measurement session:
Session ID : 1
Last advertisement:
Advertised at: Dec 12 2019 14:10:43.138 (326.782 seconds ago)
Advertised reason: First advertisement
Advertised delays (uSec): avg: 839, min: 587, max: 8209, variance: 297
Next advertisement:
Threshold check scheduled in 1 more probe (roughly every 120 seconds)
Aggregated delays (uSec): avg: 751, min: 589, max: 905, variance: 112
Rolling average (uSec): 756
Current Probe:
Started at Dec 12 2019 14:15:43.154 (26.766 seconds ago)
Packets Sent: 9, received: 9
Measured delays (uSec): avg: 795, min: 631, max: 1199, variance: 164
Next probe scheduled at Dec 12 2019 14:16:13.132 (in 3.212 seconds)
Next burst packet will be sent in 0.212 seconds
Burst packet sent every 3.0 seconds
Probe samples:
Packet Rx Timestamp Measured Delay (nsec)
Dec 12 2019 14:15:43.156 689223
Dec 12 2019 14:15:46.156 876561
Dec 12 2019 14:15:49.156 913548
Dec 12 2019 14:15:52.157 1199620
Dec 12 2019 14:15:55.156 794008
Dec 12 2019 14:15:58.156 631437
Dec 12 2019 14:16:01.157 656440
Dec 12 2019 14:16:04.157 658267
Dec 12 2019 14:16:07.157 736880
You can also use the following commands for verifying the PM for link delay on the local-end router.
Command
Description
show performance-measurement history probe interfaces [interface]
Displays the PM link-delay probe history for interfaces.
show performance-measurement history aggregated interfaces [interface]
Displays the PM link-delay aggregated history for interfaces.
show performance-measurement history advertisement interfaces [interface]
Displays the PM link-delay advertisement history for interfaces.
show performance-measurement counters [interface interface] [location location-name]
Displays the PM link-delay session counters.
You can also use the following commands for verifying the PM for link-delay configuration on the remote-end router.
Command
Description
show performance-measurement responder summary [locationlocation-name]
Displays the PM for link-delay summary on the remote-end router (responder).
show performance-measurement responder interfaces [interface]
Displays PM for link-delay for interfaces on the remote-end router.
show performance-measurement responder counters [interfaceinterface] [locationlocation-name]
Displays the PM link-delay session counters on the remote-end router.
SR Performance Measurement Named Profiles
Feature
Release Information
Feature Description
SR Performance Measurement Named Profiles
Release 7.5.2
You can use this feature to create specific performance measurement delay and liveness profiles, and associate it with an
SR policy.
You can use the delay or liveness profile to be associated with an interface or network area, where the performance management
probes are enabled, and performance measurement is precise and enhanced.
You can create a named performance measurement profile for delay or liveness.
Delay Profile
This example shows how to create a named SR performance measurement delay profile.
Router(config-sr-te)# on-demand color 20
Router(config-sr-te-color)# performance-measurement delay-measurement
Router(config-sr-te-color-delay-meas)# delay-profile name profile2
Router(config-sr-te-color-delay-meas)# commit
Running Configuration
Router# show run segment-routing traffic-eng on-demand color 20
segment-routing
traffic-eng
on-demand color 20
performance-measurement
delay-measurement
delay-profile name profile2
Liveness Profile
This example shows how to create a named SR performance measurement liveness profile.
This example shows how to enable PM for SR policy liveness for a specific policy.
For the same policy, you cannot enable delay-measurement (delay-profile) and liveness-detection (liveness-profile) at the
same time. For example, if delay measurement is enabled, use the no delay-measurement command to disable it, and then enable the following command for enabling liveness detection.
Router# show run segment-routing traffic-eng policy TRST2
segment-routing
traffic-eng
policy TRST2
color 40 end-point ipv4 20.20.20.20
candidate-paths
preference 50
explicit segment-list LIST3
weight 2
!
explicit segment-list LIST4
weight 3
!
!
!
performance-measurement
liveness-detection
liveness-profile name profile3
!
Verification
Router# show performance-measurement profile named-profile delay sr-policy
-----------
0/RSP0/CPU0
-----------
SR Policy Liveness Detection Profile Name: profile1
Profile configuration:
Measurement mode : Loopback
Protocol type : TWAMP-light
Type of service:
TWAMP-light DSCP : 10
Burst interval : 60 (effective: 60) mSec
Destination sweeping mode : Disabled
Liveness detection parameters:
Multiplier : 3
Logging state change : Disabled
SR Policy Liveness Detection Profile Name: profile3
Profile configuration:
Measurement mode : Loopback
Protocol type : TWAMP-light
Type of service:
TWAMP-light DSCP : 10
Burst interval : 60 (effective: 60) mSec
Destination sweeping mode : Disabled
Liveness detection parameters:
Multiplier : 3
Logging state change : Disabled
On-Demand SR Policy
For the same policy, you cannot enable delay-measurement (delay-profile) and liveness-detection (liveness-profile) at the
same time. For example, to disable delay measurement, use the no delay-measurement command, and then enable the following command for enabling liveness detection.
Router(config-sr-te)#on-demand color 30
Router(config-sr-te-color)#performance-measurement
Router(config-sr-te-color-pm)# liveness-detection liveness-profile name profile1
Router(config-sr-te-color-delay-meas)# commit
Running Configuration
Router# show run segment-routing traffic-eng on-demand color 30
segment-routing
traffic-eng
on-demand color 30
performance-measurement
liveness-detection
liveness-profile name profile1
!
Verification
Router# show performance-measurement profile named-profile liveness
--------------------
0/RSP0/CPU0
--------------------
SR Policy Liveness Detection Profile Name: profile1
Profile configuration:
Measurement mode : Loopback
Protocol type : TWAMP-light
Type of service:
TWAMP-light DSCP : 10
Burst interval : 60 (effective: 60) mSec
Destination sweeping mode : Disabled
Liveness detection parameters:
Multiplier : 3
Logging state change : Disabled
Delay Normalization
Performance measurement (PM) measures various link characteristics like packet loss and delay. Such characteristics can be
used by IS-IS as a metric for Flexible Algorithm computation. Low latency routing using dynamic delay measurement is one of
the primary use cases for Flexible Algorithm technology.
Delay is measured in microseconds. If delay values are taken as measured and used as link metrics during the IS-IS topology
computation, some valid ECMP paths might be unused because of the negligible difference in the link delay.
The Delay Normalization feature computes a normalized delay value and uses the normalized value instead. This value is advertised
and used as a metric during the Flexible Algorithm computation.
The normalization is performed when the delay is received from the delay measurement component. When the next value is received,
it is normalized and compared to the previous saved normalized value. If the values are different, then the LSP generation
is triggered.
The following formula is used to calculate the normalized value:
Dm – measured Delay
Int – configured normalized Interval
Off – configured normalized Offset (must be less than the normalized interval Int)
Dn – normalized Delay
a = Dm / Int (rounded down)
b = a * Int + Off
If the measured delay (Dm) is less than or equal to b, then the normalized delay (Dn) is equal to b. Otherwise, Dn is b + Int.
Example
The following example shows a low-latency service. The intent is to avoid high-latency links (1-6, 5-2). Links 1-2 and 5-6
are both low-latency links. The measured latency is not equal, but the difference is insignificant.
We can normalize the measured latency before it is advertised and used by IS-IS. Consider a scenario with the following:
Interval = 10
Offset = 3
The measured delays will be normalized as follows:
Dm = 29
a = 29 / 10 = 2 (2.9, rounded down to 2)
b = 2 * 10 + 3 = 23
In this case, Dm (29) is greater than b (23); so Dn is equal to b+I (23 + 10) = 33
Dm = 31
a = 31 / 10 = 3 (3.1, rounded down to 3)
b = 3 * 10 + 3 = 33
In this case, Dm (31) is less than b (33); so Dn is b = 33
The link delay between 1-2 and 5-6 is normalized to 33.
Configuration
Delay normalization is disabled by default. To enable and configure delay normalization, use the delay normalize intervalinterval [offsetoffset] command.
interval – The value of the normalize interval in microseconds.
offset – The value of the normalized offset in microseconds. This valude must be smaller than the value of normalized interval.
This feature allows you to define thresholds above the measured delay that is considered “anomalous” or unusual. When this
threshold is exceeded, an anomaly (A) bit/flag is set along with link delay attribute that is sent to clients.
Customers might experience performance degradation issues, such as increased latency or packet loss on a link. Degraded links
might be difficult to troubleshoot and can affect applications, especially in cases where traffic is sent over multiple ECMP
paths where one of those paths is degraded.
The Anomaly Detection feature allows you to define a delay anomaly threshold to identify unacceptable link delays. Nodes monitor
link performance using link delay monitoring probes. The measured value is compared against the delay anomaly threshold values.
When the upper bound threshold is exceeded, the link is declared “abnormal”, and performance measurement sets an anomaly bit
(A-bit). When IGP receives the A-bit, IGP can automatically increase the IGP metric of the link by a user-defined amount to
make this link undesirable or unusable. When the link recovers (lower bound threshold), PM resets the A-bit.
Usage Guidelines and Limitations
This feature is not active when narrow metrics are configured because the performance measurement advertisement requires the
“wide” metric type length values.
Configuration Example
The following example shows how to configure the upper and lower anomoly thresholds. The range for upper_bound and lower_bound is from 1 to 200,000 microseconds. The lower_bound value must be less than the upper_bound value.
IP Endpoint Delay Measurement and Liveness Monitoring
Table 5. Feature History Table
Feature Name
Release Information
Feature Description
IP Endpoint Delay Measurement and Liveness Monitoring
Release 7.4.1
This feature measures the end-to-end delay and monitors liveness of a specified IP endpoint node, including VRF-aware (awareness
of multiple customers belonging to different VRFs).
This feature is supported on IPv4, IPv6, and MPLS data planes.
The Segment Routing Performance Measurement for IP Endpoint feature dynamically measures the end-to-end delay and liveness
towards a specified IP endpoint. IP endpoints can be located in default or non-default VRFs.
The following are benefits of this feature:
Performance values (delay metrics and liveness states) are computed using TWAMP light, which is a commonly supported protocol.
Performance values, including Histograms, are sent out using Streaming Telemetry, which is a push-based data collection technique,
rather than a manual data collection technique.
The feature is applicable to multiple data plane types (e.g. IPv4, IPv6, and MPLS).
Configuring IP Endpoint Performance Measurement
Usage Guidelines and Limitations
Observe the following usage guidelines and limitations:
The endpoint of a probe is specified with an IP address. IPv4 and IPv6 endpoint addresses are supported.
The endpoint of a probe can be any IP address reachable by the sender. For example, a local interface or a remote node or
host located within an operator's network or reachable through a VRF.
The endpoint's IP address can be located in the global routing table or under a user-specified VRF routing table.
VRF-awareness allows operators to deploy probes in the following scenarios:
Managed Customer Equipment (CE) scenarios:
PE to CE probes
CE to CE probes
Unmanaged Customer Equipment (CE) scenarios:
PE to PE probes
PE to PE (source from PE-CE interface) probes
SRv6 locator prefix and VRF SRv6 locator/function (uDT4/uDT6) as IPv6 endpoint of a probe is not supported.
The endpoint's IP address can be reached through an IP path, MPLS LSP, or IP tunnel (GRE).
When the endpoint is reachable using an MPLS LSP (for example, SR, LDP, RSVP-TE, SR Policy), the forwarding stage imposes
the corresponding MPLS transport labels.
When the endpoint is reachable via a GRE tunnel, the forwarding stage imposes the corresponding GRE header.
PM probe over GREv4 is supported.
When the endpoint is reachable via a VRF in an MPLS network, the forwarding stage imposes the corresponding MPLS service labels.
In the forward path, sender node uses the configured VRF for the endpoint address. In the return path, reflector node derives
the VRF based on which incoming VRF label the probe packet is received with.
Configuring Probe Endpoint Source and Destination Addresses
Use the performance-measurement endpoint {ipv4 | ipv6} ip_addr [vrfWORD] source-address {ipv4 | ipv6} ip_addr command to configure a probe endpoint source and destination addresses on a sender node.
Router(config)# performance-measurement endpoint ipv4 10.10.10.100 vrf green source-address ipv4 1.1.1.1
Configuring Probe Description
Use the performance-measurement endpoint {ipv4 | ipv6} ip_addr [vrfWORD] descriptionLINE command to configure a probe description.
Example:
Router(config)# performance-measurement endpoint ipv4 1.1.1.5 description Probe to 1.1.1.5
Router(config)# performance-measurement endpoint ipv4 10.10.10.100 vrf green description Probe to 10.10.10.100
Configuring Probe Segment-lists
Observe the following usage guidelines and limitations:
You can specify a custom labeled path via one or more user-configured segment-list. User-configured segment-list represents
the forwarding path from sender to reflector when probe configured in delay-measurement mode.
User-configured segment-list can also represent the reverse path (reflector to sender) when probe configured in livenes-detection
mode.
Up to 128 segment-lists can be configured under a probe.
An additional PM session is created for each segment-list.
Examples of the custom segment-list include:
Probe in delay-measurement mode with segment-list that includes Flex-Algo prefix SID of the end-point
Probe in liveness-detection mode with segment-list that includes both Flex-Algo prefix SID of the end-point and the sender
Probe in delay-measurement mode with segment-list that includes SID-list with labels to reach the end-point or the sender
(forward direction)
Probe in liveness-detection mode with segment-list that includes SID-list with labels to reach the end-point and then back
to sender (forward and reverse directions, respectively)
Probe in delay-measurement mode with segment-list that includes BSID associated with SR policy to reach the end-point
Endpoint segment list configuration not supported under non-default VRF.
Segment-lists are configured under segment-routing traffic-eng segment-list submode. See SR-TE Policy with Explicit Path for details about configuring segment lists.
Use the performance-measurement endpoint {ipv4 | ipv6} ip_addr [vrfWORD] segment-list nameWORD command to configure probe segment-lists.
Example:
Router(config)# performance-measurement endpoint ipv4 1.1.1.5 segment-list name SIDLIST1
Router(config)# performance-measurement endpoint ipv4 10.10.10.100 vrf green segment-list name SIDLIST1
Enabling Delay Measurement
Observe the following usage guidelines and limitations:
Probe dynamically measures end-to-end performance delay values of an IP endpoint.
Probe can be configured in either liveness-monitoring or delay-measurement modes; not both concurrently.
Use the performance-measurement endpoint {ipv4 | ipv6} ip_addr [vrfWORD] delay-measurement command to enable delay measurement.
Router(config)# performance-measurement endpoint ipv4 10.10.10.100 vrf green delay-measurement
Assigning a Delay-Profile to the Probe
Use the performance-measurement endpoint {ipv4 | ipv6} ip_addr [vrfWORD] delay-measurement delay-profile nameWORD command to assign a delay-profile associated with the probe.
Example:
Router(config)# performance-measurement endpoint ipv4 1.1.1.5 delay-measurement delay-profile name PROFILE1
Router(config)# performance-measurement endpoint ipv4 10.10.10.100 vrf green delay-measurement delay-profile name PROFILE1
Enabling Liveness Detection
Observe the following usage guidelines and limitations:
Probe dynamically monitors liveness of an IP endpoint.
Liveness monitoring uses "self-addressed" PM IP packets crafted by the sender (where IP DA = sender's own IP address); this
mode of operation is referred as "loopback mode".
Liveness monitoring applies to endpoints reachable through MPLS LSP or IP tunnel.
Liveness monitoring does not apply to endpoints reachable through IP path since sender's self-addressed PM IP packets would
not be able to reach the intended endpoint destination.
Probe can be configured in either liveness-monitoring or delay-measurement modes; not both concurrently.
Use the performance-measurement endpoint {ipv4 | ipv6} ip_addr [vrfWORD] liveness-detection command to enable liveness detection.
Router(config)# performance-measurement endpoint ipv4 10.10.10.100 vrf green liveness-detection
Assigning Liveness-Profile to the Probe
Use the performance-measurement endpoint {ipv4 | ipv6} ip_addr [vrfWORD] liveness-detection liveness-profile nameWORD command to configure the liveness-profile associated with the probe.
Example:
Router(config)# performance-measurement endpoint ipv4 1.1.1.5 liveness-detection liveness-profile name PROFILE3
Router(config)# performance-measurement endpoint ipv4 10.10.10.100 vrf green liveness-detection liveness-profile name PROFILE3
Collecting IP Endpoint Probe Statistics
Statistics associated with the probe (computed delay metrics and liveness state) are available via Histogram and Streaming
Telemetry.
Model Driven Telemetry (MDT) is supported for the following data:
Summary, endpoint, session, and counter show command bags
History buffers data
Model Driven Telemetry (MDT) and Event Driven Telemetry (EDT) are supported for the following data:
Delay metrics computed in the last probe computation-interval (event: probe-completed)
Delay metrics computed in the last aggregation-interval; i.e. end of the periodic advertisement-interval (event: advertisement-interval
expired)
Delay metrics last notified (event: notification-triggered)
The following use-cases show different ways to deploy delay measurement and liveness detection for IP endpoints.
Use-Case 1: Delay Measurement Probe Toward an IP Endpoint Reachable in the Global Routing Table
The following figure illustrates a delay measurement probe toward an IP endpoint reachable in the global routing table. The
network interconnecting the sender and the reflector provides plain IP connectivity.
RouterA# show performance-measurement endpoint ipv4 1.1.1.5
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/RSP0/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Endpoint name: IPv4-1.1.1.5-vrf-default
Source address : 1.1.1.1
VRF name : default
Delay-measurement : Enabled
Description : Not set
Profile Keys:
Profile name : default
Profile type : Endpoint Delay Measurement
Segment-list : None
Delay Measurement session:
Session ID : 33554433
Last advertisement:
No advertisements have occured
Next advertisement:
Threshold check scheduled in 4 more probes (roughly every 120 seconds)
No probes completed
Current computation:
Started at: Jul 19 2021 16:28:06.723 (17.788 seconds ago)
Packets Sent: 6, received: 0
Measured delays (uSec): avg: 0, min: 0, max: 0, variance: 0
Next probe scheduled at: Jul 19 2021 16:28:36.718 (in 12.207 seconds)
Next burst packet will be sent in 0.207 seconds
Burst packet sent every 3.0 seconds
Use-Case 2: Delay Measurement Probe Toward an IP Endpoint Reachable in a User-Specified VRF
The following figure illustrates a delay measurement probe toward an IP endpoint reachable in a user-specified L3VPN's VRF
routing table. The L3VPN ingress PE (Router A) acts as the sender. The reflector is located in a CE device behind the L3VPN
egress PE (Router E). The network interconnecting the L3VPN PEs provides MPLS connectivity with Segment Routing.
RouterA# show performance-measurement endpoint vrf green
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/RSP0/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Endpoint name: IPv4-10.10.10.100-vrf-green
Source address : 1.1.1.1
VRF name : green
Delay-measurement : Enabled
Description : Not set
Profile Keys:
Profile name : default
Profile type : Endpoint Delay Measurement
Segment-list : None
Delay Measurement session:
Session ID : 33554434
Last advertisement:
No advertisements have occured
Next advertisement:
Advertisement not scheduled as the probe is not running
Current computation:
Not running: Unable to resolve (non-existing) vrf
Use Case 3: Delay Measurement Probe Toward an IP Endpoint Using Custom Labeled Paths
The following figure illustrates a delay measurement probe toward an IP endpoint learned by the IGP. The network interconnecting
the sender and reflector provides MPLS connectivity with Segment Routing.
The IP endpoint is advertised with multiple SR algorithms (Algo 0 and Flex Algo 128). The probe is configured with two custom-labeled
paths in order to monitor the LSP for each algorithm separately.
Note
The probe response messages are not shown in the above figure.
RouterA# show performance-measurement endpoint ipv4 1.1.1.5
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/RSP0/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Endpoint name: IPv4-1.1.1.5-vrf-default
Source address : 1.1.1.1
VRF name : default
Delay-measurement : Enabled
Description : Not set
Profile Keys:
Profile name : default
Profile type : Endpoint Delay Measurement
Segment-list : None
Delay Measurement session:
Session ID : 33554433
Last advertisement:
No advertisements have occured
Next advertisement:
Threshold check scheduled in 4 more probes (roughly every 120 seconds)
No probes completed
Current computation:
Started at: Jul 19 2021 16:31:53.827 (15.844 seconds ago)
Packets Sent: 6, received: 0
Measured delays (uSec): avg: 0, min: 0, max: 0, variance: 0
Next probe scheduled at: Jul 19 2021 16:32:22.957 (in 13.286 seconds)
Next burst packet will be sent in 1.286 seconds
Burst packet sent every 3.0 seconds
Segment-list : SIDLIST1-Algo0
Delay Measurement session:
Session ID : 33554435
Last advertisement:
No advertisements have occured
Next advertisement:
Threshold check scheduled in 4 more probes (roughly every 120 seconds)
No probes completed
Current computation:
Started at: Jul 19 2021 16:31:53.827 (15.844 seconds ago)
Packets Sent: 4, received: 0
Measured delays (uSec): avg: 0, min: 0, max: 0, variance: 0
Next probe scheduled at: Jul 19 2021 16:32:22.957 (in 13.286 seconds)
Next burst packet will be sent in 2.940 seconds
Burst packet sent every 3.0 seconds
Segment-list : SIDLIST2-FlexAlgo128
Delay Measurement session:
Session ID : 33554436
Last advertisement:
No advertisements have occured
Next advertisement:
Threshold check scheduled in 4 more probes (roughly every 120 seconds)
No probes completed
Current computation:
Started at: Jul 19 2021 16:31:53.827 (15.844 seconds ago)
Packets Sent: 4, received: 0
Measured delays (uSec): avg: 0, min: 0, max: 0, variance: 0
Next probe scheduled at: Jul 19 2021 16:32:22.957 (in 13.286 seconds)
Next burst packet will be sent in 2.940 seconds
Burst packet sent every 3.0 seconds
Use-Case 4: Liveness Detection Probe Toward an IP Endpoint
IP endpoint liveness detection leverages the loopback measurement-mode. The following workflow describes the sequence of events.
The sender creates and transmits the PM probe packets.
The IP destination address (DA) on the probe packets is set to the loopback value of the sender itself.
The transmit timestamp (T1) is added to the payload.
The probe packet is encapsulated with the label corresponding to the endpoint.
The network delivers the PM probe packets following the LSP toward the endpoint.
The end-point receives the PM probe packets.
Packets are forwarded back to the sender based on the forwarding entry associated with the IP DA of the PM probe packet. If
an LSP exists, the probe packet is encapsulated with the label of the sender.
The sender node receives the PM probe packets.
The received timestamp (T4) stored.
If the sender node doesn't receive the specified number of probe packets (based on the configured multiplier), the sender
node declares the PM session as down.
The following figure illustrates a liveness detection probe toward an IP endpoint learned by the IGP. The network interconnecting
the sender and reflector provides MPLS connectivity with Segment Routing.
The liveness detection multiplier is set to 5 to specify the number of consecutive missed probe packets before the PM session
is declared as down.
RouterA# show performance-measurement endpoint ipv4 1.1.1.5
--------------------------------------------------------------------------------
0/RSP0/CPU0
--------------------------------------------------------------------------------
Endpoint name: IPv4-1.1.1.5-vrf-default
Source address : 1.1.1.1
VRF name : default
Liveness Detection : Enabled
Profile Keys:
Profile name : default
Profile type : Endpoint Liveness Detection
Segment-list : None
Session State: Down
Missed count: 0
SR Policy End-to-End Delay Measurement
Feature Name
Release
Description
SR Policy End-to-End Delay Measurement
Release 7.5.2
This feature allows you to monitor the end-to-end delay experienced by the traffic sent over an SR policy to ensure that the
delay does not exceed the requested “upper-bound” and violate SLAs.
The PM for SR Policy uses the MPLS packet format defined in RFC 6374 or IP/UDP packet format defined in RFC 5357 (TWAMP-Light) for probes. The MPLS packet format requires the remote-side line card to be MPLS-capable.
The extended TE link delay metric (minimum-delay value) can be used to compute paths for SR policies as an optimization metric
or as an accumulated delay bound.
There is a need to monitor the end-to-end delay experienced by the traffic sent over an SR policy to ensure that the delay
does not exceed the requested “upper-bound” and violate SLAs. You can verify the end-to-end delay values before activating
the candidate-path or the segment lists of the SR policy in forwarding table, or to deactivate the active candidate-path or
the segment lists of the SR policy in forwarding table.
Note
The end-to-end delay value of an SR policy will be different than the path computation result (for example, the sum of TE
link delay metrics) due to several factors, such as queuing delay within the routers.
The PM query and response for end-to-end SR Policy delay can be described in the following steps:
The local-end router sends PM query packets periodically to the remote side once the egress line card on the router applies
timestamps on packets.
The ingress line card on the remote-end router applies time-stamps on packets as soon as they are received.
The remote-end router sends the PM packets containing time-stamps back to the local-end router.
One-way delay is measured using the time-stamp values in the PM packet.
Restrictions and Usage Guidelines for PM for SR Policy Delay
Hardware clocks must be synchronized between the querier and the responder nodes of the link using PTP for one-way delay measurement.
Enable One-Way Delay Mode
This example shows how to enable one-way delay mode.
When one-way delay mode is enabled, an IP/UDP TLV (defined in RFC 7876) is added in the query packet to receive the PM reply
via IP/UDP. Only two time-stamps (T1 and T2) defined in the RFC 6374 packets are used. Hardware clocks must be synchronized
between querier and responder nodes (using PTP).
This example shows how to configure performance-measurement parameters for SR policy delay as a global default profile. The
default values for the different parameters in the PM for SR policy delay is given as follows:
probe: The default mode for probe is one-way delay measurement.
tx-interval: Interval for sending probe packet. The default value is 3000000 microseconds and the range is from 3300 to 15000000 microseconds.
computation interval: Interval for metric computation. Default is 30 seconds; range is 1 to 3600 seconds.
protocol:
twamp-light: SR Policy delay measurement using RFC 5357 with IP/UDP encap. This is the default protocol.
pm-mpls: SR Policy delay measurement using RFC6374 with MPLS encap.
tos: Type of Service
dscpvalue: The default value is 0 and the range is from 0 to 63.
traffic-classvalue: The default value is 0 and the range is from 0 to 7.
advertisement threshold-check: The advertisement threshold-check has three types:
Configuring the UDP port for TWAMP-Light protocol is optional. By default, PM uses port 862 as the TWAMP-reserved UDP destination
port.
The UDP port is configured for each PM measurement probe type (delay, loss, protocol, authentication mode, etc.) on querier
and responder nodes. If you configure a different UDP port, the UDP port for each PM measurement probe type must match on
the querier and the responder nodes.
Note
The same UDP destination port is used for delay measurement for links and SR Policy.
This example shows how to configure the UDP destination port.
This example shows how to configure SR Policy ECMP IP-hashing mode.
The destination IPv4 address 127.x.x.x – 127.y.y.y is used in the Probe messages to take advantages of 3-tuple IP hashing
(source-address, destination-address, and local router ID) for ECMP paths of SR-MPLS Policy.
Note
The destination IPv4 address must be 127/8 range (loopback), otherwise it will be rejected.
One PM session is always created for the actual endpoint address of the SR Policy.
You can specify the number of IP addresses to sweep. The range is from 0 (default, no sweeping) to 128.
Platforms may have a limitation for large label stack size to not check IP address for hashing.
SR Policy Liveness Monitoring on Segment Routing over IPv6 (SRv6)
Release 7.11.1
In segment routing over IPv6 (SRv6), you can now verify end-to-end traffic forwarding over an SR policy candidate path by
periodically sending probe messages. Performance monitoring on an SRv6 network enables you to track and monitor traffic flows
at a granular level.
Earlier releases supported SR policy liveness monitoring over an SR policy candidate path on MPLS.
SR Policy Liveness Monitoring
Release 7.5.2
This feature allows you to verify end-to-end traffic forwarding over an SR Policy candidate path by periodically sending performance
monitoring packets.
SR Policy liveness monitoring allows you to verify end-to-end traffic forwarding over an SR Policy candidate path by periodically
sending probe messages. The head-end router sends PM packets to the SR policy's endpoint router, which sends them back to
the head-end without any control-plane dependency on the endpoint router.
The following are benefits to using SR-PM liveness monitoring:
Allows both liveness monitoring and delay measurement using a single-set of PM packets as opposed to running separate monitoring
sessions for each purpose. This improves the overall scale by reducing the number of PM sessions required.
Eliminates network and device complexity by reducing the number of monitoring protocols on the network (for example, no need
for Bidirectional Failure Detection [BFD]). It also simplifies the network and device operations by not requiring any signaling
to bootstrap the performance monitoring session.
Improves interoperability with third-party nodes because signaling protocols aren't required. In addition, it leverages the
commonly supported TWAMP protocol for packet encoding.
Improves liveness detection time because PM packets aren't punted on remote nodes
Provides a common solution that applies to data-planes besides MPLS, including IPv4, IPv6, and SRv6.
The workflow associated with liveness detection over SR policy is described in the following sequence.
Consider an SR policy programmed at head-end node router 1 towards end-point node router 5. This SR policy is enabled for
liveness detection using the loopback measurement-mode.
A: The head-end node creates and transmits the PM probe packets.
The IP destination address (DA) on the probe packets is set to the loopback value of the head-end node itself.
A transmit (Tx) timestamp is added to the payload.
Optionally, the head-end node may also insert extra encapsulation (labels) to enforce the reverse path at the endpoint node.
Finally, the packet is injected into the data-plane using the same encapsulation (label stack) of that of the SR policy being
monitored.
B: The network delivers the PM probe packets as it would user packet for the SR policy.
C: The end-point node receives the PM probe packets.
Packets are switched back based on the forwarding entry associated with the IP DA of the packet. This would typically translate
to the end-point node pushing the prefix SID label associated with the head-end node.
If the head-end node inserted label(s) for the reverse path, then the packets are switched back at the end-point node based
on the forwarding entry associated with the top-most reverse path label.
D: Headend node receives the PM probe packets.
A received (Rx) timestamp stored.
If the head-end node receives the PM probe packets, the head-end node assume that the SR policy active candidate path is up
and working.
If the head-end node doesn't receive the specified number of consecutive probe packets (based on configured multiplier), the
head-end node assumes the candidate path is down and a configured action is trigerred.
Usage Guidelines and Limitations
The following usage guidelines and limitations apply:
Liveness-detection and delay-measurement aren't supported together
When liveness-profile isn't configured, SR Policies use the default values for the liveness-detection profile parameters.
Configuring SR Policy Liveness Monitoring
Configuring SR Policy liveness monitoring involves the following steps:
Configuring a performance measurement liveness profile to customize generic probe parameters
Enabling liveness monitoring under SR Policy by associating a liveness profile, and customizing SR policy-specific probe parameters
Liveness monitoring parameters are configured under sub-mode. The following parameters are configurable:
liveness-profile {sr-policy default | namename}
probe: Configure the probe parameters.
measurement-mode: Liveness detection must use loopback mode (see Measurement Mode topic within this guide).
tx-interval: Interval for sending probe packet. The default value is 3000000 microseconds and the range is from 3300 to 15000000 microseconds.
tos dscpvalue: The default value is 48 and the range is from 0 to 63. You can modify the DSCP value of the probe packets, and use this
value to priortize the probe packets from headend to tailend.
sweepdestination ipv4 127.x.x.xrange range: Configure SR Policy ECMP IP-hashing mode. Specifiy the number of IP addresses to sweep. The range is from 0 (default, no
sweeping) to 128. The option is applicable to IPv4 packets.
Note
The destination IPv4 headendaddress 127.x.x.x – 127.y.y.y is used in the Probe messages to take advantages of 3-tuple IP hashing
(source-address, destination-address, and local router ID) for ECMP paths of SR-MPLS Policy.
The destination IPv4 address must be 127/8 range (loopback), otherwise it will be rejected.
Note
One PM session is always created for the actual endpoint address of the SR Policy.
liveness-detection: Configure the liveness-detection parameters:
multiplier: Number of consecutive missed probe packets before the PM session is declared as down. The range is from 2 to 10, and the
default is 3.
Note
The detection-interval is equal to (tx-interval * multiplier).
Enabling Liveness Monitoring under SR Policy
Enable liveness monitoring under SR Policy, associate a liveness-profile, and configure SR Policy-specific probe parameters
under the segment-routing traffic-eng policy performance-measurement sub-mode. The following parameters are configurable:
liveness-detection: Enables end-to-end SR Policy Liveness Detection for all segment-lists of the active and standby candidate-path that are
in the forwarding table.
liveness-profile namename: Specifies the profile name for named profiles.
invalidation-action {down | none}:
Down (default): When the PM liveness session goes down, the candidate path is immediately operationally brought down.
None: When the PM liveness session goes down, no action is taken. If logging is enabled, the failure is logged but the SR Policy
operational state is not modified.
logging session-state-change: Enables Syslog messages when the session state changes.
reverse-path label {BSID-value | NODE-SID-value | ADJACENCY-SID-value}: Specifies the MPLS label to be used for the reverse path for the reply. If you configured liveness detection with ECMP
hashing, you must specify the reverse path. The default reverse path uses IP Reply.
BSID-value: The Binding SID (BSID) label for the reverse SR Policy. (This is practical for manual SR policies with a manual BSID.)
NODE-SID-value: The Node SID is a segment type that represents the ECMP-aware shortest path to reach a particular IP prefix from any IGP
topology location
ADJACENCY-SID-value: The absolute SID label of the (local) Sender Node to be used for the reverse path for the reply.
Configuration Examples
Configure a Default SR-Policy PM Liveness-Profile
The following example shows a default sr-policy liveness-profile:
segment-routing
traffic-eng
policy FOO
performance-measurement
liveness-detection
liveness-profile name sample-profile
invalidation-action none
!
!
!
!
!
end
Enable Liveness Monitoring under SR Policy with Optional Parameters
The following example shows how to enable liveness monitoring under SR Policy, associate a liveness-profile, and configure
reverse path label and session logging:
segment-routing
traffic-eng
policy BAA
performance-measurement
liveness-detection
logging
session-state-change
!
liveness-profile name sample-profile
invalidation-action down
!
reverse-path
label 16001
!
!
!
!
!
end
Configure Segment Lists to Activate Candidate Paths in SRv6 for PM Liveness
Table 7. Feature History Table
Feature Name
Release Information
Feature Description
Configure Segment Lists to Activate Candidate Paths in SRv6 for PM Liveness
Release 7.11.1
You can now enable a candidate path to be up by configuring the minimum number of active segment lists associated with the
candidate path. The head-end router determines that a candidate path is up based on the minimum number of active segment lists
configured.
In earlier releases, the head-end router identified a candidate path as up only when all the segment lists associated with
the path were active.
The state of the segment lists in a candidate path determines whether a candidate path is up or down. You can now configure
the minimum number of active segment lists associated with a candidate path. The head-end router identifies a candidate path
as up when one or more segment lists are active.
Note
If the configured minimum number of active segment lists is greater than the number of available segment lists in a candidate
path, the head-end router determines the candidate path as up only when all the segment lists are active.
In earlier releases, the router identified a candidate path as up only when all the segment lists associated with the path
were active.
Configure the minimum number of segment lists in SRv6
Perform this task to activate three segment lists to have the PM liveness session up:
The following example shows three active segment-lists to have the PM liveness session up:
Router#show performance-measurement sr-policy liveness color 103 detail verbose private
Mon Oct 30 15:10:51.863 EDT
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/1/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
SR Policy name: srte_c_103_ep_3::1
Color : 103
SRv6 Encap Source Address : 1::1
Endpoint : 3::1
Handle : 0x00000000
Policy to be deleted : False
Number of candidate-paths : 1
Candidate-Path:
Instance : 5
Preference : 300
Protocol-origin : Configured
Discriminator : 300
Profile Keys:
Profile name : default
Profile type : SR Policy Liveness Detection
Candidate path to be deleted: False
Source address : 1::1
Local label : Not set
Fast notification for session down: Disabled
No fast notifications have been sent
Number of segment-lists : 3
Liveness Detection: Enabled
Minumum SL Up Required: 1
Session State: Up
Last State Change Timestamp: Oct 30 2023 15:10:16.322
Missed count: 0
Segment-List : sl-1041
fccc:cc00:1:fe10:: (Local Adjacency SID)
fccc:cc00:2:fe41::/64
Format: f3216
Segment List ID: 0
Reverse path segment-List: Not configured
Segment-list to be deleted: False
Number of atomic paths : 1
Liveness Detection: EnabledSession State: Up
Last State Change Timestamp: Oct 30 2023 15:10:16.322
Missed count: 0
Atomic path:
Flow Label : 0
Session ID : 4198
Trace ID : 738913600
Atomic path to be deleted: False
NPU Offloaded session : False
Timestamping Enabled : True
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 30 2023 15:10:16.322
Missed count: 0
Responder IP : 1::1
Number of Hops : 3
Segment-List : sl-1042
fccc:cc00:1:fe10:: (Local Adjacency SID)
fccc:cc00:2:fe42::/64
Format: f3216
Segment List ID: 0
Reverse path segment-List: Not configured
Segment-list to be deleted: False
Number of atomic paths : 1
Liveness Detection: EnabledSession State: Up
Last State Change Timestamp: Oct 30 2023 15:10:16.322
Missed count: 0
Atomic path:
Flow Label : 0
Session ID : 4199
Trace ID : 954039677
Atomic path to be deleted: False
NPU Offloaded session : False
Timestamping Enabled : True
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 30 2023 15:10:16.322
Missed count: 0
Responder IP : 1::1
Number of Hops : 3
Segment-List : sl-1043
fccc:cc00:1:fe10:: (Local Adjacency SID)
fccc:cc00:2:fe43::/64
Format: f3216
Segment List ID: 0
Reverse path segment-List: Not configured
Segment-list to be deleted: False
Number of atomic paths : 1
Liveness Detection: Enabled Session State: Up
Last State Change Timestamp: Oct 30 2023 15:10:16.322
Missed count: 0
Atomic path:
Flow Label : 0
Session ID : 4200
Trace ID : 1119107116
Atomic path to be deleted: False
NPU Offloaded session : False
Timestamping Enabled : True
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 30 2023 15:10:16.322
Missed count: 0
Responder IP : 1::1
Number of Hops : 3
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/RSP0/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Configure Flow Labels in SRv6 Header for PM Liveness
Table 8. Feature History Table
Feature Name
Release Information
Feature Description
Configure Flow Labels in SRv6 Header for PM Liveness
Release 7.11.1
You can now monitor the activeness of multiple paths for a given segment list using flow labels in the SRv6 header.
In earlier releases, the SRv6 header didn't include flow labels.
To monitor the activeness of multiple paths for a given a segment list, you can configure the SRv6 header to include flow
labels as the packet travels in the network. When there are multiple paths, different traffic flows may use different paths.
A flow label is a flow identifier and you can use different flow labels to monitor different ECMP paths. It's only used for
IPv6 probe packets. Flow labels are 20-bit fields in the SRv6 header.
Configure flow labels in the SRv6 header
Perform the following task in the global configuration mode to configure flow labels in the SRv6 header:
Router#configure
Router(config)#performance-measurement
Router(config-perf-meas)#liveness-profile name name1
Router(config-pm-ld-profile)#probe flow-label from 0 to 1000000 increment 10
Running Configuration
performance-measurement
liveness-profile name name1
probe
flow-label from 0 to 1000000 increment 10
!
!
!
Verification
The following example shows an SR-policy configured with flow labels:
Router#show performance-measurement sr-policy liveness color 1001 detail verbose private
Mon Oct 30 15:25:55.241 EDT
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/1/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
SR Policy name: srte_c_1001_ep_3::1
Color : 1001
SRv6 Encap Source Address : 1::1
Endpoint : 3::1
Handle : 0x00000000
Policy to be deleted : False
Number of candidate-paths : 1
Candidate-Path:
Instance : 3
Preference : 300
Protocol-origin : Configured
Discriminator : 300
Profile Keys:
Profile name : profile-scale
Profile type : Generic Liveness Detection
Candidate path to be deleted: False
Source address : 1::1
Local label : Not set
Fast notification for session down: Disabled
No fast notifications have been sent
Number of segment-lists : 2
Liveness Detection: Enabled
Minumum SL Up Required: 2
Session State: Up
Last State Change Timestamp: Oct 26 2023 15:31:43.478
Missed count: 0
Segment-List : sl-1041
fccc:cc00:1:fe10:: (Local Adjacency SID)
fccc:cc00:2:fe41::/64
Format: f3216
Segment List ID: 0
Reverse path segment-List: Not configured
Segment-list to be deleted: False
Number of atomic paths : 2
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 26 2023 15:31:43.478
Missed count: 0
Atomic path:
Flow Label : 0
Session ID : 4178
Trace ID : 280178832
Atomic path to be deleted: False
NPU Offloaded session : False
Timestamping Enabled : True
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 26 2023 15:31:43.478
Missed count: 0
Responder IP : 1::1
Number of Hops : 3
Atomic path:
Flow Label : 10
Session ID : 4179
Trace ID : 1866227171
Atomic path to be deleted: False
NPU Offloaded session : False
Timestamping Enabled : True
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 26 2023 15:31:43.478
Missed count: 0
Responder IP : 1::1
Number of Hops : 3
Segment-List : sl-scale
fccc:cc00:1:fe10:: (Local Adjacency SID)
fccc:cc00:2:fed1::/64
Format: f3216
Segment List ID: 0
Reverse path segment-List: Not configured
Segment-list to be deleted: False
Number of atomic paths : 2
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 26 2023 15:31:43.478
Missed count: 0
Atomic path:
Flow Label : 0
Session ID : 4180
Trace ID : 2609815826
Atomic path to be deleted: False
NPU Offloaded session : False
Timestamping Enabled : True
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 26 2023 15:31:43.478
Missed count: 0
Responder IP : 1::1
Number of Hops : 3
Atomic path:
Flow Label : 10
Session ID : 4181
Trace ID : 170501506
Atomic path to be deleted: False
NPU Offloaded session : False
Timestamping Enabled : True
Liveness Detection: Enabled
Session State: Up
Last State Change Timestamp: Oct 26 2023 15:31:43.478
Missed count: 0
Responder IP : 1::1
Number of Hops : 3
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/RSP0/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Path Tracing in SRv6 Network
Table 9. Feature History Table
Feature Name
Release
Description
Path Tracing Midpoint Node
Release 7.8.1
Path Tracing (PT) provides a log or record of the packet path as a sequence of interface IDs along with its time stamp. In
Path Tracing, a node can behave as a source, midpoint, or sink node.
The Path Tracing Midpoint feature is implemented in this release which measures the hop-by-hop delay, traces the path in the
network and collects egress interface load information and interface Id, and stores them in the Midpoint Compressed Data (MCD)
section of Hop-by-Hop Path Tracing (HbH-PT) header.
This feature provides visibility to the Path Tracing Midpoint node that handles IPv6 transit in Path Tracing and full characterization
of the packet delivery path. It provides real time information and the current status of the network.
Operators do not know the actual path that the packets take within their network. This makes operations, such as troubleshooting
routing problems, or verifying Equal-Cost Multipath (ECMP), a complex problem. Also, operators want to characterize the network
in terms of delay and load on a per-hop basis.
Knowledge of the Path Tracing Midpoint helps the operators to troubleshoot the routing problems faster.
This feature allows the operators to:
Detect the actual path the packet takes between any two nodes in network (A and Z).
Measure the end-to-end delay from A to Z.
Measure the per-hop delay at each node on the path from A to Z.
Detects the load on each router that forwards the packet from A to Z
Path Tracing (PT) provides a log or record of the packet path as a sequence of interface IDs along with its time-stamp. In
addition, it provides a record of end-to-end delay, per-hop delay, and load on each egress interface along the packet delivery
path.
In Path Tracing, a node can behave as a source, midpoint, or a sink node.
The source node generates and injects probe packets toward a destination node to trace the time-stamp and interface ID along
the path of the probe packet. The Interface ID value of 0 means that Path Tracing (PT) is disabled on the interface.
Path Tracing (PT) Midpoint: It is a transit node that performs IPv6 routing. In addition, it records the PT information (MCD)
in the HbH-PT.
Note
There is no support for Path Tracing Midpoint on transit nodes that perform SRH operations or SRv6 endpoint operations.
Midpoint Compressed Data (MCD): The PT Midpoint along the packet delivery path from the Source to Sink node, stores its PT
information into the HbH-PT header. This PT information is called Midpoint Compressed Data (MCD).
Hop-by-Hop Path Tracing (HbH-PT): In IPv6 The HbH PT Options header is used to carry optional information that is examined
and processed by every node along a packet's delivery path. It contains a stack of MCDs.
PT-Aware Midpoint: A midpoint node that performs plain IPv6 routing or SR Endpoint processing and in addition stores the Path
Tracing information in HbH-PT.
PT-Unaware Midpoint: A midpoint node that performs plain IPv6 routing or SR Endpoint processing and is not capable of performing
Path Tracing.
PT-Legacy Midpoint: A midpoint node that performs plain IPv6 routing or SR Endpoint processing and is not capable of recording
Path Tracing information in the HBH-PT. However, it is capable of exporting Path Tracing information directly to the collector,
using the node telemetry system.
PT Source: A Source node is the one that starts a PT session and generates PT probes.
PT Sink: A node that receives the PT probes sent from the Source node containing the information recorded by every PT Midpoint
along the path and forwards them to the collector after recording its Path Tracing information.
RC: Regional collector that receives PT probes, parses, and stores them in Timeseries DB
The destination or sink node that receives the PT probes generated by the PT source node, stores PT related info into PT-TLV
and forwards them to a Regional Collector (RC). This Regional Collector (RC) parses and stores them in the TimeSeries Database.
It uses the information in the Hop-by-Hop Path Tracing (HbH-PT) to construct the packet delivery path and the timestamps at
each node.
Limitations and Guidelines
This section lists the limitations of this feature.
PT Source and Sink nodes are not supported yet. The system can still work as PT midpoint for other devices acting as Source
or Sink in the PT network path.
No support for interface load calculation and recording on IPv6 Path Tracing MidPoint Node. MCD contains interface load value
of 0.
SRv6 Segment Endpoint Midpoint PT (Update DA from SRH.SL and PT MCD update) at midpoint node is not supported. SRv6 endpoint
function will not execute properly.
IPv6 and SRv6 Path Tracing Midpoint Node are supported. SRv6 PT midpoint support Micro-SID (uSID) Shift and Forward action
with MCD update.
Path tracing on Bundled Interfaces and subinterfaces is supported by configuring path-tracing interface-id on physical ports.
PT unaware IPv6 and SRv6 midpoint forwards transparently without PT update or may punt the packet locally and the control-plane
drops the packet.
PT unaware SRv6 Segment Endpoint Midpoint Node will not execute SRv6 endpoint function. PT packet is forwarded transparently
without PT update or punted locally and the control-plane drops the packet.
Configuration Steps
Note
These configurations must be done on the Source, Midpoint and Sink routers as shown in the following configuration examples.
Configuration example of Source node:
Configure the endpoint with the probe profile name on the source node:
It is good to check the target interface configuration and performance-measurement configuration for that interface.
Verify using the show commands listed below to check if the PT configuration is applied to the interface properly.
Source Node Verification
Router# sh run performance-measurement
performance-measurement
probe-profile name foo
tx-interval 6000
flow-label from 100 to 300 increment 10
!
!
Router# sh performance-measurement profile named-profile
Endpoint Probe Measurement Profile Name: foo
Profile configuration:
Measurement mode : One-way
Protocol type : TWAMP-light
Type of service:
TWAMP-light DSCP : 48
TX interval : 6000000 (effective: 6000000) uSec
Destination sweeping mode : Disabled
Liveness detection parameters:
Multiplier : 3
Logging state change : Disabled
Hop Limit : 255
Flow Label Count : 21
Flow Labels: 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240,
250, 260, 270, 280, 290, 300
Packet Size Count : 0
Traffic Class Count : 0
Router#sh run performance-measurement
performance-measurement
endpoint ipv6 bbbb::
path-assurance
session-id 11
!
!
!
source-address ipv6 aaaa::
!
Router# sh performance-measurement endpoint
Endpoint name: IPv6-bbbb::-vrf-default
Source address : Unknown
VRF name : default
Probe Measurement : Enabled
Profile Keys:
Profile name : default
Profile type : Endpoint Probe MeasurementRun this show command to verify the probe sessions:
Router# show performance-measurement probe-sessions
Transport type : Endpoint
Measurement type : Probe
Endpoint name : IPv6-bbbb:bbbb:2::-vrf-default
endpoint : bbbb:bbbb:2::
source : bbbb:bbbb:1::
vrf : default
Segment-list :
Path Tracing session:
Session ID : 10
Profile Keys:
Profile name : pt1
Profile type : Probe
Current status:
Packet sent every 0.30000 seconds (value stretched for rate-limiting)
Next packet will be sent in 0.20 seconds
Transport type : Endpoint
Measurement type : Probe
Endpoint name : IPv6-bbbb:bbbb:2::-vrf-default
endpoint : bbbb:bbbb:2::
source : bbbb:bbbb:1::
vrf : default
Segment-list :
Path Tracing session:
Session ID : 11
Profile Keys:
Profile name : pt2 (Profile not found)
Profile type : N/A
Current status:
Not running: Profile is not configured
Transport type : Endpoint
Measurement type : Probe
Endpoint name : IPv6-bbbb:bbbb:2::-vrf-default
endpoint : bbbb:bbbb:2::
source : bbbb:bbbb:1::
vrf : default
Segment-list :
Path Assurance session:
Session ID : 20
Profile Keys:
Profile name : pa1
Profile type : Probe
Current status:
Packet sent every 0.30000 seconds (value stretched for rate-limiting)
Next packet will be sent in 0.24 secondsRun this show command to view the summary of all the probe sessions:
Router# show performance-measurement summary
Measurement Information:
Total interfaces with PM sessions : 0
Total SR Policies with PM sessions : 0
Total Endpoints with PM sessions : 1
Total RSVP-TE tunnels with PM sessions : 0
Global Counters:
Total packets sent : 0
Total query packets received : 0
Total invalid session id : 0
Total missing session : 0
Probe sessions:
Total sessions : 3
Path-tracing sessions:
Total running sessions : 1
Total running error sessions : 0
Path-assurance sessions:
Total running PA sessions : 1
Total running error PA sessions : 0
Counters:
Path-tracing packets:
Total sent : 3063
Total sent errors : 0
Path-assurance packets:
Total sent : 470
Total sent errors : 0
Router# show cef interface fourHundredGigE 0/0/0/1
FourHundredGigE0/0/0/1 is up if_handle 0x0f000208 if_type IFT_FOURHUNDREDGE(0xcd)
idb info 0x94dfbf88 flags 0x30001 ext 0x0
Vrf Local Info (0x0)
Interface last modified, create
Reference count 1 Next-Hop Count 0
PT (path tracing) is enabled: id:0xC8 load_in:0x0 load_out:0x0 tts:0x3
Protocol Reference count 0
Protocol ipv4 not configured or enabled on this card
Primary IPV4 local address NOT PRESENT
This is an example of Show CLI with Interface ID:
Router# show run performance-measurement
performance-measurement
probe-profile name foo
tx-interval 6000
flow-label from 100 to 300 increment 10
!
!
Router# sh performance-measurement profile named-profile
Endpoint Probe Measurement Profile Name: foo
Profile configuration:
Measurement mode : One-way
Protocol type : TWAMP-light
Type of service:
TWAMP-light DSCP : 48
TX interval : 6000000 (effective: 6000000) uSec
Destination sweeping mode : Disabled
Liveness detection parameters:
Multiplier : 3
Logging state change : Disabled
Hop Limit : 255
Flow Label Count : 21
Flow Labels: 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240,
250, 260, 270, 280, 290, 300
Packet Size Count : 0
Traffic Class Count : 0
Router# show cef interface GigabitEthernet 0/2/0/0
GigabitEthernet0/2/0/0 is up if_handle 0x01000020 if_type IFT_GETHERNET(0xf)
idb info 0x619f16f0 flags 0x30101 ext 0x627ef180 flags 0x30050
Vrf Local Info (0x626510f0)
Interface last modified Mar 4, 2022 13:34:43, modify
Reference count 1 Next-Hop Count 3
PT (path tracing) is enabled: id:0x40 load_in:0x0 load_out:0x0 tts:0x1
Forwarding is enabled
ICMP redirects are never sent
ICMP unreachables are enabled
Protocol MTU 1500, TableId 0xe0000000(0x61ccf768)
Protocol Reference count 4
Primary IPV4 local address 10.10.10.1
Router# show performance-measurement interfaces
Interface Name: GigabitEthernet0/2/0/0 (ifh: 0x1000020)
Delay-Measurement : Disabled
Loss-Measurement : Disabled
Path-Tracing : Enabled
Configured IPv4 Address : 10.10.10.1
Configured IPv6 Address : 10:10:10::1
Link Local IPv6 Address : fe80::91:e4ff:fe60:6707
Configured Next-hop Address : Unknown
Local MAC Address : 0291.e460.6707
Next-hop MAC Address : 023a.6fc9.cd6b
In-use Source Address : 10.10.10.1
In-use Destination Address : 10.10.10.2
Primary VLAN Tag : None
Secondary VLAN Tag : None
State : Up
Path-Tracing:
Interface ID : 64
Load IN : 0
Load OUT : 0
Load Interval : 60
Last FIB Update:
Updated at: Mar 04 2022 13:34:43.112 (0.392 seconds ago)
Update reason: Path tracing config
Update status: Done
This is an example of Show CLI without InterfaceID, which means PT is disabled on the target interface. So, you can configure
timestamp template:
Router# show cef interface GigabitEthernet 0/2/0/0
GigabitEthernet0/2/0/0 is up if_handle 0x01000020 if_type IFT_GETHERNET(0xf)
idb info 0x619f16f0 flags 0x30101 ext 0x627ef180 flags 0x30050
Vrf Local Info (0x626510f0)
Interface last modified Mar 4, 2022 13:49:37, modify
Reference count 1 Next-Hop Count 3
Forwarding is enabled
ICMP redirects are never sent
ICMP unreachables are enabled
Protocol MTU 1500, TableId 0xe0000000(0x61ccf768)
Protocol Reference count 4
Primary IPV4 local address 10.10.10.1
Router# sh performance-measurement interfaces
Interface Name: GigabitEthernet0/2/0/0 (ifh: 0x1000020)
Delay-Measurement : Disabled
Loss-Measurement : Disabled
Path-Tracing : Enabled
Configured IPv4 Address : 10.10.10.1
Configured IPv6 Address : 10:10:10::1
Link Local IPv6 Address : fe80::91:e4ff:fe60:6707
Configured Next-hop Address : Unknown
Local MAC Address : 0291.e460.6707
Next-hop MAC Address : 023a.6fc9.cd6b
In-use Source Address : 10.10.10.1
In-use Destination Address : 10.10.10.2
Primary VLAN Tag : None
Secondary VLAN Tag : None
State : Up
Path-Tracing:
Interface ID : 0
Timestamp Template : 3
Load IN : 0
Load OUT : 0
Load Interval : 60
Last FIB Update:
Updated at: Mar 04 2022 13:49:37.492 (176.418 seconds ago)
Update reason: Path tracing config
Update status: Done
Two-Way Active Measurement Protocol Light Source Address Filtering
Table 10. Feature History Table
Feature Name
Release Information
Feature Description
Two-Way Active Measurement Protocol Light Source Address Filtering
Release 7.11.1
You can now restrict unauthorized users from sending packets to the network and prevent compromising the network security
and reliability. For a destination UDP port, you can configure the list of IP addresses that can send Two-Way Active Measurement
Protocol (TWAMP)-light packets to responder or querier nodes.
In earlier releases, the responder or querier node accepted TWAMP-light packets from all IP addresses.
Earlier, the responder node scanned all IP addresses of a querier on the destination UDP port. In other words, the responder
node accepted packets from any IP address. See Measurement Modes for more information about querier and responder nodes.
Note
The responder is also called the reflector, and the querier is also called the sender.
With this configuration, you can specify the source IP addresses on both the responder and querier nodes. The responder or
querier nodes accept packets only from the IP addresses configured in the TWAMP-light protocol, and reject the packets from
an IP address that isn’t included in the configured list.
All the configured addresses are available for use on all interfaces in the Local Packet Transport Services (LPTS). The configured
address filter applies to both default and nondefault VRFs. The TWAMP delay measurement sessions use the configured addresses.
Usage Guidelines and Limitations
The following usage guidelines and limitations apply:
When you configure the prefix entries on the responder or querier nodes, the PM adds the responder or querier node source
IP address to the LPTS. For each prefix, a new LPTS entry is added or created.
For TWAMP liveness sessions, the PM automatically adds the source IP addresses to the LPTS for if you have configured the
prefix entries on the responder or querier nodes.
As the maximum number of LPTS hardware entries are limited, ensure that enough LPTS entries are allocated for the IP addresses
on a line card. You can scale the LPTS configuration to maximum LPTS entries for the PM flow-type. For more details on configuring
the LTPS entries for PM flow-type, refer to the IP Addresses and Services Configuration Guide.
Note
The PM UDP port accepts all incoming IPv4 or IPv6 packets when there are no IPv4 or IPv6 prefix entries configured.
Configure IP address on querier and responder nodes
The length of the IPv4 and IPv6 prefixes must be less than 32 and 128 respectively.
The length or mask of the source IP address must be:
For IPv4: 0–31
For IPv6: 0–127
Configure the IP address on a responder
Perform this task to configure the IP address of a querier on a responder node for delay measurement.
The following example shows output from the IP address of a querier, which is configured on a responder node for delay measurement.
Router#show performance-measurement allowed-querier summary
Wed Oct 11 10:41:43.268 UTC
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/RP0/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Allowed-querier IPv4 prefix : 1
10.10.10.1/32
Allowed-querier IPv6 prefix : 0
RX UDP port status:
TWAMP-Light Default Unauthenticated responder port : 862
Opened IPv4 port : 862
IPv4 Port Update Time : Oct 11 2023 10:37:48.118
Opened IPv6 port : 862
IPv6 Port Update Time : Oct 11 2023 10:37:47.778
Configure the source IP address on a querier
Perform this task to configure the IP address of a responder on a querier node for delay measurement.
The following example shows output from the IP address of a responder, which is configured on a querier node for delay measurement.
Router#show performance-measurement allowed-responder summary
Wed Mar 29 19:38:06.381 UTC
----------------------------------------------------------------------------------------------------------------------------------------------------------------
0/RP1/CPU0
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Allowed-responder IPv4 prefix : 2
10.10.10.1/32 [Auto]
3.3.3.3/32
Allowed-responder IPv6 prefix : 1
fc00:0:1::1/128 [Auto] [Pending Add]
Querier CPU UDP port status:
TWAMP-Light Default Unauthenticated querier port : N/A
Opened IPv4 port : 27643
IPv4 Port Update Time : Mar 29 2023 18:43:49.080
Opened IPv6 port : 28274
IPv6 Port Update Time : Mar 24 2023 20:58:46.150