Router Hardware Management Guide for Cisco 8000 Series Routers, Cisco IOS XR Releases

PDF

Router Hardware Management Guide for Cisco 8000 Series Routers, Cisco IOS XR Releases

Fabric link management for uncorrectable errors

Want to summarize with AI?

Log in

Describes how the router uses Forward Error Correction (FEC) to detect uncorrectable errors on fabric links, retune noisy links, and permanently shuts down a link after repeated failures.



Summary

The key components involved in the fabric link monitoring process are:

  • Router: A network device that monitors fabric links by processing FEC reports, performing retuning, logging system messages, and managing link states based on failure frequency.

  • Fabric link: A physical connection that undergoes quality monitoring, retuning, and snapshot collection to ensure stable data plane operation, with a permanent shutdown occurring after the sixth failure within one hour.

  • MAC port: A network interface that stops and reactivates between the second and fifth link failure to facilitate recovery.

Workflow

For every FEC report, the router performs this process to monitor the fabric links:

  1. If the fabric link is faulty before the link is up, the router retunes and checks again for the FEC improvement.
  2. If the link quality does not improve after retuning, the router displays the syslog message after tuning for 100 times and does not bring the link to the data plane.
  3. Post link up, when the fabric link becomes noisy, the router collects a snapshot and retunes after the first failure.
  4. From second failure to fifth failure, the MAC port stops and re-activates. Retune is done as part of this process.
  5. If the fabric link fails for the sixth time within an hour, the router permanently shuts down the link.


Procedure

Verify the FEC link information using show controllers npu link-info command.

Example:

Router#show controllers npu link-info rx 254 255 fsm instance 0 location 0/2/CPU0 detail
Sat Jan 13 00:39:49.448 UTC
 
Node ID: 0/2/CPU0
Link ID: 0/2/0/254    255      Oper State: DOWN   0/FC1/2/158
Event                      State                 Timestamp
+--------------------------------------------------------------------------------+
LINK_UP_INTR               UP                    Sat Jan 13 00:18:44 2018
BAD_FEC_BELOW_THR          PORT_ACTIVATE_DELAYED Sat Jan 13 00:19:16 2018
LINK_MON                   FAB_PORT_CREATED      Sat Jan 13 00:19:17 2018
LINK_MON                   ACTIVATED             Sat Jan 13 00:19:19 2018
LINK_UP_INTR               MAC_UP                Sat Jan 13 00:19:24 2018
LINK_UP_INTR               PEER_DISCOVERY        Sat Jan 13 00:19:24 2018
LINK_UP_INTR               PEER_DETECTED         Sat Jan 13 00:19:24 2018
LINK_UP_INTR               TOPOLOGY_CHECK        Sat Jan 13 00:19:24 2018
LINK_UP_INTR               SYNC_WAIT             Sat Jan 13 00:19:24 2018
LINK_UP_INTR               KEEPALIVE_START       Sat Jan 13 00:19:24 2018
LINK_UP_INTR               CHECK_REACH           Sat Jan 13 00:19:24 2018
LINK_UP_INTR               UP                    Sat Jan 13 00:19:24 2018
BAD_FEC                    UP                    Sat Jan 13 00:20:16 2018
DIS_PERM_SHUT              MAC_UP                Sat Jan 13 00:20:16 2018
DIS_PERM_SHUT              STOPPED               Sat Jan 13 00:20:16 2018
+--------------------------------------------------------------------------------+

This table describes the significant fields shown in this example.

Table 2. show controllers npu link-info field descriptions

Field

Description

BAD_FEC_BELOW_THR

There are FEC failures, but the number of failures has not exceeded the predefined threshold (in this case, 5 per hour). The router retunes and checks for FEC improvement.

BAD_FEC

This part of the log entry indicates that FEC detected failures, and the number of these failures surpassed a predefined threshold. As a result, the decision was made to permanently shut down the affected interface or port as a protective measure.

DIS_PERM_SHUT

The link or port has been intentionally disabled and is in a shutdown state after FEC fails for the threshold limit (After fifth failure).


Procedure

1.

Enter the global configuration mode.

Example:

2.

Disable the fabric FEC monitor.

Example:


Router(config)# hw-module fabric-fec-monitor disable
3.

Save the changes.

Example:

Router(config)# commit