Understanding MDS Congestion Counters and Commands - reference document

Counter Name

Description

Commands

Additional Info

F16_RCM_RCP0_RBBZ_CH0^5,96S,2410 

F16_RCM_RCP0_RBBZ_CH1^5,96S,2410 

F16_RCM_RCP0_RBBZ_CH2^5,96S,2410 

F16_RCM_RCP0_RBBZ_CH3^5,96S,2410 

VIP_RCM_RBBZ_CH0_CNT  ^{50i, 48S}

F32_RCM_P8_P15_RCP_RBBZ_CH0^{6,32T,48T,96T,20i}

F32_RCM_P8_P15_RCP_RBBZ_CH1^{6,32T,48T,96T,20i}

F32_RCM_P8_P15_RCP_RBBZ_CH2^{6,32T,48T,96T,20i}

F32_RCM_P8_P15_RCP_RBBZ_CH3^{6,32T,48T,96T,20i}

F64_RCM_P0_P7_RCP_RBBZ_CH0^{7,24V,48V,96V}

F64_RCM_P0_P7_RCP_RBBZ_CH1^{7,24V,48V,96V}

F64_RCM_P0_P7_RCP_RBBZ_CH2^{7,24V,48V,96V}

F64_RCM_P0_P7_RCP_RBBZ_CH3^{7,24V,48V,96V}

Total count of transitions to zero for Rx B2B credits on ch0; these transitions typically indicate that the switch is applying back pressure to the attached device because of perceived congestion, and this perceived congestion can be the result of a lack of Tx B2B credits being returned on an interface over which this device is communicating

There is no indication of time at zero for this counter. It could stay at zero for just an instant or for an extended duration of time.

Also shown in the output of show interface counters:

xxxx receive B2B credit transitions from zero

or

xxxx  Receive B2B credit transitions to zero

show interface counters “from” was changed to “to” via this Cisco bug ID

CSCug35184 show interface counters - transitions of rx BB credit to zero state

For F16, F32, F64, CH0 – CHx indicates the virtual channel or virtual link. If the link is in R_RDY mode then only VL0(CH0) will be used. If the link is in ER_RDY mode then they will increment according to which VL it is occurring on. Specifically:

CH0 – VL0 – This is the Control virtual link

CH1 – VL1 – This is the High priority virtual link (currently not used)

CH2 – VL2 – This is the Slow device virtual link

CH3 – VL3 – This is the Normal virtual link

Pxx_Pxx indicates the port group

Sup:

show hardware internal statistics <module x | module-all>  device <fcmac | all>  all <port x> ^5,6,note1

None^48s,50i

show hardware internal statistics device  <fcmac | all>  all <port x> ^FS

Linecard:

slot x show hardware internal statistics device fcmac|all^{5,6,7,2410,48s,50i,96S,32T,48T,96T,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

F16_TMM_PORT_TBBZ_CH0^5,96S,2410

F16_TMM_PORT_TBBZ_CH1^5,96S,2410

F16_TMM_PORT_TBBZ_CH2^5,96S,2410

F16_TMM_PORT_TBBZ_CH3^5,96S,2410

VIP_TMM_TBBZ_CH0_CNT^{50i, 48S}

VIP_TMM_TBBZ_CH1_CNT^{50i, 48S}

F32_TMM_PORT_TBBZ_CH0^{6,32T,48T,96T,20i}

F32_TMM_PORT_TBBZ_CH1^{6,32T,48T,96T,20i}

F32_TMM_PORT_TBBZ_CH2^{6,32T,48T,96T,20i}

F32_TMM_PORT_TBBZ_CH3^{6,32T,48T,96T,20i}

F64_TMM_PORT_TBBZ_CH0^{7,24V,48V,96V}

F64_TMM_PORT_TBBZ_CH1^{7,24V,48V,96V}

F64_TMM_PORT_TBBZ_CH2^{7,24V,48V,96V}

F64_TMM_PORT_TBBZ_CH3^{7,24V,48V,96V}

Total count of transitions to zero for Tx B2B credits on ch0 or ch1; these transitions are typically the result of the attached device's withholding of R_Rdy primitive from the switch due to congestion in that device.

There is no indication of time at zero for this counter. It could stay at zero for just an instant or for an extended duration of time.

Also shown in the output of show interface counters:

xxxx transmit B2B credit transitions from zero

or

xxxx  Transmit B2B credit transitions to zero

show interface counters “from” was changed to “to” via this Cisco bug ID

CSCug35184     show interface counters - transitions of rx BB credit to zero state

For F16, F32 and F64, CH0 – CH3 indicates the virtual channel or virtual link. If the link is in R_RDY mode then only VL0(CH0) will be used. If the link is in ER_RDY mode then they will increment according to which VL it is occurring on. Specifically:

CH0 – VL0 – This is the Control virtual link

CH1 – VL1 – This is the High priority virtual link (currently not used)

CH2 – VL2 – This is the Slow device virtual link

CH3 – VL3 – This is the Normal virtual link

Sup:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^5,6,7,note1

None^48s,50i

show hardware internal statistics device  <fcmac | all>  all <port x> ^FS

Linecard:

slot x show hardware internal statistics device fcmac|all^{5,6,7,2410,48s,50i,96S,32T,48T,96T,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

F16_TMM_PORT_TWAIT_CNT^5,2410,96S 

VIP_TMM_TXWAIT_CH0_CNT^{50i, 48S}

VIP_TMM_TXWAIT_CH1_CNT^{50i, 48S}

F32_TMM_PORT_TWAIT^{6,32T,48T,96T,20i}

F64_TMM_PORT_TWAIT^{7,24V,48V,96V}

Packet is available to send, but no credit is available on link in R_RDY mode.

Gen5(F16): increments every clock cycle. Two clock cycles depending speed group:

1)     Port in 2/4/8/16G mode:

cycle = 2.353 nanoseconds 425Mhz)

2)     Port in 10G mode:

Cycle = 3.14ns 320Mhz

9250i/9148s: Increments every clock cycle (cycle = 2ns 500MHz)

F32: increments every clock cycle 850Mhz (1.175 ns)

F64: increments every clock cycle 1.1 Ghz (0.909ns )

F16/F32/F64: To calculate actual time:

Seconds = Twait  / clock_rate

See Table 3 SNMP variable  fcIfTxWaitCount

Sup:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note1}

None^48s,50i

show hardware internal statistics device  <fcmac | all>  all <port x> ^,96S

Linecard:

slot x show hardware internal statistics device fcmac|all all <port xx>^5,6,48s,50i

Note1: If module is not specified the default is the module number of the active supervisor. 

F32_ENI_PORT_CWAIT_CH0^{6,32T,48T,96T,20i} 

F32_ENI_PORT_CWAIT_CH1^{6,32T,48T,96T,20i}  

F32_ENI_PORT_CWAIT_CH2^{6,32T,48T,96T,20i} 

F32_ENI_PORT_CWAIT_CH3^{6,32T,48T,96T,20i}

F64_ENI_PORT_CWAIT_CH0^{7,24V,48V,96V}

F64_ENI_PORT_CWAIT_CH1^{7,24V,48V,96V}

F64_ENI_PORT_CWAIT_CH2^{7,24V,48V,96V}

F64_ENI_PORT_CWAIT_CH3^{7,24V,48V,96V}

F64_ENI_PORT_CWAIT_CH4^{7,24V,48V,96V}

F64_ENI_PORT_CWAIT_CH5^{7,24V,48V,96V}

F64_ENI_PORT_CWAIT_CH6^{7,24V,48V,96V}

F64_ENI_PORT_CWAIT_CH7^{7,24V,48V,96V}

Packet is available to send, but no credit is available on VL in link in ER_RDY mode.

Count of time in 1.33ns that a specific VL in an ER_RDY link has zero Tx credits and there is a frame to send on that VL.  

For  F32 CH0 – CH3 indicates the virtual channel or virtual link.  If the link is in R_RDY mode then the F32_TMM_PORT_TWAIT counter will be used. Increments every clock cycle 750 Mhz (1.333 ns) for ENI.

For F64 CH0 – CH7 indicates the virtual channel or virtual link. F64 has 8 VLs but only 4 are in use in ER_RDY mode.  If the link is in R_RDY mode then the F64_TMM_PORT_TWAIT counter will be used.  Increments every clock cycle 1.1GHz (0.909ns) for ENI.

If the link is in ER_RDY mode then the VLs represent this:

CH0 – VL0 – This is the Control virtual link

CH1 – VL1 – This is the High priority virtual link (currently not used)

CH2 – VL2 – This is the Slow device virtual link

CH3 – VL3 – This is the Normal virtual link

Sup:

show hardware internal statistics <module x | module-all>  device <fcmac | all>  all <port x> ^6,note1

Linecard:

slot x show hardware internal statistics device fcmac|all all <port xx>⁶

Note1: If module is not specified the default is the module number of the active supervisor. 

FCP_SW_CNTR_RX_WT_AVG_B2B_ZERO^{5,2410,48S,50i,96S, note1}

F32_MAC_KLM_CNTR_RX_WT_AVG_B2B_ZERO^{6,32T,48T,96T,20i}

F64_CMON_RX_WT_100MS_CH0_CNT1^{7,24V,48V,96V}

F64_CMON_RX_WT_100MS_CH3_TMR1_HIT^{7,24V,48V,96V}

Count of the number of times an interface was at zero Rx B2B credits for 100 ms; this status typically indicates that the switch is withholding R_Rdy primitive to the device attached on that interface due to congestion in the path to devices with which it is communicating

CH0 is channel zero when the link is in R_RDY mode

CH3 is the ‘normal’ VL when the link is in ER_RDY mode

Incremented by the software creditmon process prior to F32. Starting with F32 (including F64) incremented by HW timers.

OBFL:

Show logging onboard error-stats^{2,3,4,56,,48,48s,50i}

Sup Hardware internal errors:

show hardware internal errors all|module x^{5,2410,6,7,note2}

None^{48s,50i,note4}

show hardware internal errors^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note2}

None^48S,50i

show hardware internal statistics  device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard Hardware internal statistics:

slot x show hardware internal errors^{5,2410,6,7,48s,50i}

slot x show hardware internal statistics device fcmac all <port x>^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: Gen5 and 9250i do not increment 6.2(1), 6.2(5) and 6.2(7).

Cisco bug ID CSCui27981     FCP_SW_CNTR_RX_WT_AVG_B2B_ZERO not incrementing on DS-X9448-768K9

Integrated  in: 6.2(9)

Note2: If module is not specified the default is the module number of the active supervisor. 

Note3: Will be added to ‘show interface counters detailed’. See  Cisco bug ID CSCvy61847

Add five congestion related counters to show interface counters detailed

FCP_SW_CNTR_TX_WT_AVG_B2B_ZERO^{5,2410,50i,48S,96S}

F32_MAC_KLM_CNTR_TX_WT_AVG_B2B_ZERO^{6,32T,48T,96T,20i}

F64_CMON_TX_WT_100MS_CH0_TMR1_HIT^{7,24V,48V,96V}

F64_CMON_TX_WT_100MS_CH3_TMR1_HIT^{7,24V,48V,96V}

Count of the number of times that an interface was at zero Tx B2B credits for 100ms. This status typically indicates congestion at the device attached on that interface.

CH0 is channel zero when the link is in R_RDY mode

CH3 is the ‘normal’ VL when the link is in ER_RDY mode

NX-OS 6.2(1) through 6.2(7) on the 9710 and NX-OS 6.2(5) through 6.2(7) on the 9250i this was incremented based on F16_FCP_INTR_TMM_P_CWAIT_FORCE_TIMEOUT_L_H and 

VIPER_FCP_INTR_TMM_CWAIT_AVG_LIVE_CH1_OVER_THRESHOLD_RAISING interrupts.

Consequently, this only occurred once when the HW interrupt occurred and not each 100ms interval like in prior instances.

MDS 9700, 9250i and 9148S:  In NX-OS 6.2(9) these are once again incremented by the software creditmon process. They will once again increment each 100ms interval where the port remains at 0 Tx credits. Starting with F32 (including F64) incremented by HW timers.

OBFL:

Show logging onboard error-stats^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal errors:

show hardware internal errors <all|module x>^{5,2410,6,7,Note1}

None^48S,50i

show hardware internal errors^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{4,5,2410,6,7,note1}

None^48,48s,50i

Linecard Hardware internal errors:

slot x show hardware internal errors^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

Linecard Hardware internal statistics:

slot x show hardware internal statistics device fcmac all port x^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2: Will be added to ‘show interface counters detailed’. See  Cisco bug ID CSCvy61847

Add five congestion related counters to show interface counters detailed

F16_TMM_PORT_STUCK_FORCE_TIMEOUT_L_H_CNT^{5,96S,note1,note3}

F32_TMM_PORT_STUCK_FORCE_TIMEOUT_L_H^{6,32T,48T,96T,20i,Note4}

F64_TMM_PORT_STUCK_FORCE_TIMEOUT_L_H^{7,24V,48V,96V}

Count of times port was at zero Tx credits for the stuck port timeout value(no-credit-drop threshold).

NX-OS 6.2(1) through 6.2(7) on the 9710  this was used for  credit loss recovery so was set to 1s(F port)/1.5s(E port).

Needs to be configured via:

 System timeout no-credit-drop <ms> mode f

Or in 8.1(1) and later:

  System timeout no-credit-drop <ms> logical-type edge

This counter will increment even if “system timeout no-credit-drop” is not configured since it defaults to 500ms. If no-credit-drop is not configured then no action is taken and it simply indicates the port was at zero Tx credits for 500ms. ^Note3

This is similar to the viper counter:

VIP_TMM_STK_PRT_TO_TRANSITION_CHx_CNT

Sup Hardware internal errors:

show hardware internal errors <all|module x>^,5,6

show hardware internal errors^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,Note2}

show hardware internal statistics  device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard:

Slot x show hardware internal statistics device all^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

slot x show hardware internal errors^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> counter-stats^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> error-stats^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V,Note 4}

Note1:  May falsely increment during port flap:

Cisco bug ID CSCus70632     F16_TMM_PORT_STUCK_FORCE_TIMEOUT_L_H_CNT increments during port flap

Integrated in NX-OS 6.2(13)

Note2: If module is not specified the default is the module number of the active supervisor. 

Note3:   Cisco bug ID CSCut27271 Stuck port threshold not reset to default when removing no-credt-drop

Integrated  in: NX-OS 6.2(13)

Note 4: Added to OBFL error-stats in NX-OS 9.3(2) via bug  CSCwa76582

Note5: Will be added to ‘show interface counters detailed’. See bug:   Cisco bug ID CSCvy61847

Add five congestion related counters to show interface counters detailed

F16_TMM_PORT_STUCK_FORCE_TIMEOUT_H_L_CNT^5,96S,note1

F32_TMM_PORT_STUCK_FORCE_TIMEOUT_H_L^{6,32T,48T,96T,20i,Note3}

F64_TMM_PORT_STUCK_FORCE_TIMEOUT_H_L^{7,24V,48V,96V}

Count of times port received a credit after being stuck(hit no-credit-drop threshold)

Sup Hardware internal errors:

show hardware internal errors <all|module x>^5,2410,6,7

show hardware internal errors^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note2}

show hardware internal statistics  device  <fcmac | all>  all <port x> ^96S

Linecard:

Slot x show hardware internal statistics device all^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

slot x show hardware internal errors^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> counter-stats^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> error-stats^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V,Note3}

Note1:  Cisco bug ID  CSCut27271 Stuck port threshold not reset to default when removing no-credit-drop

Integrated  in: NX-OS 6.2(13)

Note2: If module is not specified the default is the module number of the active supervisor. 

Note3: Added to OBFL error-stats in NX-OS 9.3(2) via  Cisco bug ID CSCwa76582

F16_TMM_PORT_CWAIT_FORCE_TIMEOUT_L_H_CNT^5,2410,96S

F32_TMM_CH0_TMR1_HIT^{6,32T,48T,96T,20i,Note2}

F64_TMM_PORT_CWAIT_FORCE_TIMEOUT_L_H^{7,24V,48V,96V}

Count of times port was at zero Tx credits for the slow port timeout value.

NX-OS 6.2(1) through 6.2(7) on the 9710 and NX-OS 6.2(5) through 6.2(7) on the 9250i this was used to increment the  FCP_SW_CNTR_TX_WT_AVG_B2B_ZERO

counter.

Consequently, this only occurred once when the HW interrupt occurred and not each 100ms interval like in prior instances.

In NX-OS 6.2(9) this is used for the slowport-monitor feature.

Needs to be configured via:

 System timeout slowport-monitor <ms> mode e|f

Slowport-monitor events can be displayed via:

 show process creditmon slowport-monitor-events

  show logging onboard slowport-monitor-events

This is similar to the viper counter:

VIP_TMM_SLO_PRT_TO_TRANSITION_CHx_CNT

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,Note1}

show hardware internal statistics  device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard:

Slot 1 show hardware internal statistics device all|fcmac ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> counter-stats^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2: In NX-OS 8.1(1) slowport-monitor only works in ER_RDY mode.

Note3: Will be added to ‘show interface counters detailed’. See  Cisco bug ID CSCvy61847

Add five congestion related counters to show interface counters detailed

F16_TMM_PORT_CWAIT_FORCE_TIMEOUT_H_L_CNT^5,2410,96S

None^{6,32T,48T,96T,20i}

F64_TMM_PORT_CWAIT_FORCE_TIMEOUT_H_L^{7,24V,48V,96V}

Count of times a credit was received after the slow port timeout threshold had been triggered.

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,Note1}

show hardware internal statistics  device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard:

Slot 1 show hardware internal statistics device all|fcmac^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

VIP_TMM_STK_PRT_TO_TRANSITION_CH0_CNT^{48S,50i,note2}

VIP_TMM_STK_PRT_TO_TRANSITION_CH1_CNT^{48S,50i,note2}

Count of times port was at zero Tx credits for the stuck port timeout value.

-          channel 0 (high priority queue)

-          channel 1 (low priority queue)

NX-OS 6.2(5) through 6.2(7) on the 9250i this was used for  credit loss recovery so was set to 1s(F port)/1.5s(E port).

In NX-OS 6.2(9) the software creditmon process once again detects credit loss recovery and the stuck force timout is used for “system timeout no-credit-drop”. Defaults to 500ms with no action taken (no packets are dropped then it is reached).

Needs to be configured via:

 System timeout no-credit-drop <ms> mode e|f

This counter will increment even if “system timeout no-credit-drop” is not configured since it defaults to 500ms. If no-credit-drop is not configured then no action is taken and it simply indicates the port was at zero Tx credits for 500ms. ^Note3

This is similar to the Gen5 counter F16_TMM_PORT_STUCK_FORCE_TIMEOUT_L_H_CNT

Sup:

None

Linecard:

slot 1 show hardware internal statistics device all|fcmac

slot 1 show hardware internal errors

Note2:   Cisco bug ID CSCut27271

Stuck port threshold not reset to default when removing no-credit-drop

Integrated  in: NX-OS 6.2(13)

VIP_TMM_SLO_PRT_TO_TRANSITION_CH0_CNT^48S,50i

VIP_TMM_SLO_PRT_TO_TRANSITION_CH1_CNT^48S,50i

Count of times port was at zero Tx credits for the slow port timeout value.

-          channel 0 (high priority queue)

-          channel 1 (low priority queue)

NX-OS 6.2(5) through 6.2(7) on the 9250i this was used to increment the  FCP_SW_CNTR_TX_WT_AVG_B2B_ZERO counter.

Consequently, this only occurred once when the HW interrupt occurred and not each 100ms interval like in prior instances.

In NX-OS 6.2(9) this is used for the slowport-monitor feature.

Needs to be configured via:

 System timeout slowport-monitor <ms> mode e|f

Slowport-monitor events can be displayed via:

 show process creditmon slowport-monitor-events

  show logging onboard slowport-monitor-events

This is similar to the Gen5 counter;

F16_FCP_INTR_TMM_P_CWAIT_FORCE_TIMEOUT_L_H

Sup:

None

Linecard:

 show hardware internal statistics device all|fcmac

F64_RCM_RWAIT_CH0_CNT2^{7,24V,48V,96V}

F64_RCM_RWAIT_CH1_CNT2^{7,24V,48V,96V}

F64_RCM_RWAIT_CH2_CNT2^{7,24V,48V,96V}

F64_RCM_RWAIT_CH3_CNT2^{7,24V,48V,96V}

F64_RCM_RWAIT_CH4_CNT2^{7,24V,48V,96V}

F64_RCM_RWAIT_CH5_CNT2^{7,24V,48V,96V}

F64_RCM_RWAIT_CH6_CNT2^{7,24V,48V,96V}

F64_RCM_RWAIT_CH7_CNT2^{7,24V,48V,96V}

RxWait counters (2.5us intervals) as measured at Receive Credit Manager. As compared to TxWait, which uses the much more granular TWAIT, it will miss intervals of zero Rx credits less than 2.5us. What this means that RxWait will always be less than the TxWait counter on the other side of the link.

Sup:

None

Linecard:

slot x show hardware internal statistics device fcmac|all^{7,24V,48T,96V}

Counter Name

Description

Commands

Additional Info

F16_TMM_PORT_FRM_DROP_CNT^5,2410,96S

FCP_CNTR_TMM_NORMAL_DROP⁴⁸

VIP_TMM_NORMAL_DROP_CNT^{50i, 48S, note1}

VIP_TMM_TOTAL_DROP_ CNT^{50i, 48S, note1}

F32_TMM_PORT_FRAME_DROP^{6,32T,48T,96T,20i}

F64_TMM_PORT_FRAME_DROP^{7,24V,48V,96V}

Number of frames dropped in tolb_path or np path by the Transmit Memory Manager(TMM); these drops include all types of packet drops: timeout, offline, abort drops,  dummy frame drops  at egress, and so on.

These counters are the aggregate counters for all the underlying counters.

For example for F32 this includes:

F32_TMM_PORT_ABORT

F32_TMM_PORT_LB_DROP

F32_TMM_PORT_TIMEOUT_DROP

F32_TMM_PORT_OFFLINE

F32_TMM_PORT_ABORT_DROP

F32_TMM_PORT_TO_DROP_CLASSF

F32_TMM_PORT_TO_DROP_CLASS2

F32_TMM_PORT_TO_DROP_CLASS3

F32_TMM_PORT_TO_DROP_CLASSO

F32_TMM_PORT_TO_CLASSF

For F64 the list is similar.

Sup Hardware internal errors

show hardware internal errors <module x> ^{5,2410,6,7, note2}

None^48s,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7, note2}

None^48s,50i

show hardware internal statistics  device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard Hardware internal errors

show hardware internal errors^{5,2410,,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

Show logging onboard <module x> error-stats ^{5,2410,,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

Note 1: It is not normal for drops to occur so this counter's name is misleading.   Cisco bug ID CSCus60322  renamed VIP_TMM_NORMAL_DROP_CNT to VIP_TMM_TOTAL_DROP_CNT since the drops included in this counter are not necessarily normal.

Cisco bug ID CSCus60322 Add VIP_TMM_TO_CNT and VIP_TMM_TO_DROP_CNT to packet-flow dropped

Integrated in NX-OS 6.2(13)

Note2: On a MDS 9700 if module is not specified the default is the module number of the active supervisor. 

F16_TMM_TOLB_TIMEOUT_DROP_CNT^5,2410,96S 

VIP_TMM_TO_DROP_CNT^{50i,48S,note1,note3}

F32_TMM_PORT_TIMEOUT_DROP^{6,32T,48T,96T,20i}

F64_TMM_PORT_TIMEOUT_DROP^{7,24V,48V,96V}

Total number of Timeout drops which includes frames timed out due to congestion drop, HW stuck force timeout(no-credit-drop), HW slow port force timeout(we don’t ever so this).

Congestion drop threshold is set via the  system timeout congestion-drop command. It is on at 500ms by default on all “modes” (port types):

system timeout congestion-drop xxx mode e|f

or

system timeout congestion-drop  xxx logical-type edge|core

Sup Hardware internal errors

show hardware internal errors <module x> ^{5,2410,6,7, note2}

None^48s,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all> all <port x> ^{5,2410,6,7, note2}

None^48s,50i

show hardware internal statistics  device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard Hardware internal errors

show hardware internal errors ^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

show hardware internal statistics device  <fcmac | all>  all <port x> ^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

Show logging onboard <module x> error-stats ^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: This includes VIP_TMM_TO_CNT and VIP_TMM_STUCK_PORT_TO_CNT .

Note2: If module is not specified the default is the module number of the active supervisor. 

Note3: This is the value that is output in the show interface counters “timeout discards” and fcIfTimeoutDiscards starting in  NX-OS 8.3(1). Previously it was VIP_TMM_TO_CNT which was incorrect. See:

Cisco bug ID CSCva11058 : MDS 9148S 9250i timeout discards and fcIfTimeoutDiscards not including all timeout drops

Integrated in 8.3(1)

and

Cisco bug ID CSCvi82068 MDS 9148S 9250i fcIfTimeoutDiscards including all timeout drops 2 times

Integrated in 8.3(1)

F16_TMM_TOLB_LB_DROP_CNT^5,2410,96S

F32_TMM_PORT_LB_DROP^{6,32T,48T,96T}

F64_TMM_PORT_LB_DROP^{7,24V,48V,96V}

Loopback test frames that are dropped due to congestion. These are AT=LB frames. They are aggregated into Fxx_TMM_PORT_FRAME_DROP

Sup Hardware internal statistics

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7, note2}

show hardware internal statistics device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

show hardware internal statistics device  <fcmac | all>  all <port x> ^{5,2410,6,7,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL

None

These are classified as misc (not error) so they are not included in ‘show hardware internal errors’ or ‘show logging onboard error-stats’ or ‘show logging onboard counter-stats’.

F16_TMM_PORT_STUCK_FORCE_TIMEOUT_CNT^5,2410,96S

VIP_TMM_STUCK_PORT_TO_CNT ^48S,50i

F32_TMM_PORT_STUCK_FORCE_TIMEOUT ^{6,32T,48T,96T,20i}

F64_TMM_PORT_STUCK_FORCE_TIMEOUT ^{7,24V,48V,96V}

Total number of frames force timeout dropped by Stuck port processing(no-credit-drop)

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,,6,7,Note1}

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

None^48S,50i

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,48S,50i32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2: Will be added to ‘show interface counters detailed’. See  Cisco bug ID CSCvy61847

Add five congestion related counters to show interface counters detailed

VIP_TMM_TO_CNT ^{48S,50i,note1}

Total number of timeout drops due to congestion drop only.

See VIP_TMM_STUCK_PORT_TO_CNT for the

total  timeout drops due to no-credit-drop

See VIP_TMM_TO_DROP_CNT for the number of timeout drops from both congestion-drop and no-credit-drop

Sup hardware internal errors

None ^48S,50i

Sup hardware internal statistics:

None ^48S,50i

Linecard hardware internal errors:

show hardware internal errors^48S,50i

Linecard hardware internal statistics:

show hardware internal statistics device all|fcmac^48S,50i

OBFL:

show logging onboard error-stats ^48S,50i

show logging onboard counter-stats ^48S,50i

Note1: This is the value that is output in the show interface counters “timeout discards” and fcIfTimeoutDiscards up to NX-OS 8.3(1). This is incorrect. See  Cisco bug ID

CSCva11058 : MDS 9148S 9250i timeout discards and fcIfTimeoutDiscards not including all timeout drops

Integrated in 8.3(1)

and

Cisco bug ID  CSCvi82068 MDS 9148S 9250i fcIfTimeoutDiscards including all timeout drops 2 times

Integrated in 8.3(1)

F16_TMM_PORT_CWAIT_FORCE_TIMEOUT_CNT⁵

VIP_TMM_SLOW_PORT_TO_CNT ^48S,50i

Total timeout packets dropped due to slow-port-monitor processing. This doesn’t increment since the slow-port-monitor feature doesn’t include a packet drop function

None.

Not implemented.

VIP_RHP_CNAV_DRP_CNT^48S,50i

VIP_RHP_CNAV_DROP_CNT^{48S,50i,Note2}

F16_RHP_CNAV_DRP_CNT ^5,2410,96S

F16_RHP_CNAV_DROP_CNT ^{5,2410,96S,Note2}

F32_RHP_CIP_CNAV_DRP_CNT ^{6,32T,48T,96T,20i}

F32_RHP_CNAV_DROP_CNT^{6,32T,48T,96T,20i,Note2}

F64_RHP_CIP_CNAV_DRP_CNT ^{7,24V,48V,96V}

F64_RHP_CNAV_DROP_CNT ^{7,24V,48V,96V,Note2}

Frame received but Credit Not Available in Ingress. These frame(s) will be dropped as CNAV_DROPS. These are ingress frame drops.

Sup hardware internal errors

show hardware internal errors <module x> ^{5,2410,,6,7,Note1}

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

None^48S,50i

Sup hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors | all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

None^48S,50i

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Linecard hardware internal statistics:

show hardware internal statistics device <fcmac | all>  errors | all <port x> ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,48S,50i,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2:  Cisco bug ID CSCwi57997 added these counters to OBFL and also renamed them ‘DRP’ to ‘DROP’

CSCwi57997

Add VIP_RHP_CNAV_DRP_CNT, F32_RHP_CIP_CNAV_DRP_CNT and F64_RHP_CIP_CNAV_DRP_CNT to OBFL

Integrated in 9.4(2)

This Nexus bug was transmitting without a Tx credit causing the CNAV_DROPs:

Cisco bug ID CSCwe31997 Nexus C93360YC-FX2 FC interface randomly

IP_FCMAC_CNT_STATS_ERRORS_RX_BAD_WORDS_FROM_DECODER ^{48S,50i,5,2410,96S}

F32_MAC_KLM_CNTR_RX_BAD_WORDS_FROM_DECODER^{6 ,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_RX_BAD_WORDS_FROM_DECODER ^{7,24V,48V,96V}

Invalid, non-decodeable,  transmission word(ITW) received. These can be primitive signals, primitive sequences or parts of actual data frames.

Sup hardware internal errors

show hardware internal errors <module x> ^{5,2410,,6,7,Note1}

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

None^48S,50i

Sup hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors | all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

None^48S,50i

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Linecard hardware internal statistics:

show hardware internal statistics device <fcmac | all>  errors | all <port x> ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

None^{48S,50i,Note2}

F32_MAC_KLM_CNTR_RX_FEC_UNCORRECTED_BLOCKS^{6 ,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_RX_FEC_UNCORRECTED_BLOCKS^{7,24V,48V,96V}

Number of Forward Error Correction(FEC) blocks that could not be corrected due to bit errors. The block is then passed up to the FCMAC layer. This can result in Invalid Transmission Words(ITWs), lost FC primitives (like R_RDY) and/or CRCs if the FEC block was part of an actual data frame.

FEC Is optional on 16G speeds but mandatory on 32G and higher speeds.

FEC_Uncorrected errors were not being recorded in OBFL – Note3

In some cases these seem to increment in increments of about 33,000 at times - Note4

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2: Viper ASIC in 9148S and 9250i does not support FEC.

Note3:

Cisco bug ID CSCvv93541

F32_MAC_KLM_CNTR_RX_FEC_UNCORRECTED_BLOCKS not being recorded in OBFL error-stats counter-stats

Integrated in 8.4(2c) and 9.2(1)

Note4:

Cisco bug ID CSCvw52616

FEC Corrected and uncorrected errors incrementing on 32G modules

Integrated in: Open

IP_FCMAC_CNT_STATS_ERRORS_RX_CRC ^{5,2410,96S,48S,50i}

IP_FCMAC_ERRORS_RX_CRC^{6,7,32T,48T,96T,20i,24V,48V,96Vj}

Frames received with a bad CRC at the FCMAC layer.

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

None^,48S,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

F16_RHP_NP_RCRC ^5,2410,96S

F32_RHP_NP_RCRC ^{6,32T,48T,96T,20i}

F64_RHP_NP_RCRC ^{7,24V,48V,96V}

Total normal packets received with CRC error or mac err that did not have the error EOF ordered set

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

IP_FCMAC_CNT_STATS_ERRORS_RX_EOFA ^5,2410,96S

IP_FCMAC_ERRORS_RX_EOFA ^{6,32T,48T,96T,20i,6,32T,48T,96T,20i}

Frames with EOF Abort received at the FC MAC from the external device. This indicates the MDS actually received frames with EOFa on them. Compare that to the Fxx_RHP_NP_EOFA counter which is at the higher RHP layer. 

These errors typically mean a cut-through device, like a UCS Fabric Interconnect, is detecting Ethernet CRCs and is forwarding them up to the MDS FC interfaces as ‘stomped CRC’ errors. Because of this, when the MDS detects CRCs and also EOFA these do not indicate physical layer issues on the FC links. They indicate physical layer Ethernet errors within the UCS domain.

Sup Hardware internal errors:

show hardware internal errors <module x> ^{6,7,Note1,Note2}

show hardware internal errors ^{32T,48T,96T,20i,24V,48V,96V,Note2}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,note1}

show hardware internal statistics device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{6,7,32T,48T,96T,20i,24V,48V,96V,Note2}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

Not in OBFL

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2:  IP_FCMAC_CNT_STATS_ERRORS_RX_EOFA is classified as MISC and not error so for F16 based platforms this error is not in any ‘show hardware internal errors’.

Note3:

Cisco bug ID CSCww26937

Replace Fxx_RHP_NP_EOFA with IP_FCMAC_ERRORS_RX_EOFA in OBFL error-stats

F16_RHP_NP_EOFA ^5,2410,96S

F32_RHP_NP_EOFA ^{6,32T,48T,96T,20i}

F64_RHP_NP_EOFA ^{6,32T,48T,96T,20i}

Frame with EOF Abort received at the Receive Header Processor (RHP) block in the Fxx ASIC. Normally, when a frame is received at the FCMAC it will have an EOFn (Normal). If the calculated CRC does not match the CRC in the frame then the frame is corrupted and will be dropped. To facilitate this the FCMAC replaces the EOFn with an EOFa (Abort). If a frame with a bad CRC is received that also has EOFa on it, for example from a cut-through switch that has ‘stomped’ a frame, then  IP_FCMAC_ERRORS_RX_EOFA will increment *and* the  Fxx_RHP_NP_EOFA counter will also increment.

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

None^,48S,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

IP_FCMAC_CNT_STATS_ERRORS_RX_MAX_FRAME_TRUNCATE^{5,2410,96S,48S,50i}

IP_FCMAC_ERRORS_RX_MAX_FRAME_TRUNCATE CRC^{6,7,32T,48T,96T,20i,24V,48V,96Vj}

FCMAC received a frame that was larger than the 2142 (I think) maximum size. This occurs when an EOF was not detected. The frame will be terminated with an EOFabort so it is dropped and the RHP clock.

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

None^,48S,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

IP_FCMAC_CNT_STATS_ERRORS_RX_MIN_FRAME_PAD^{5,2410,96S,48S,50i}

IP_FCMAC_ERRORS_RX_MIN_FRAME_PAD ^{6,7,32T,48T,96T,20i,24V,48V,96Vj}

FCMAC received a frame that was ended before the minimum number of bytes (60 I think). The frame will be terminated with an EOF abort so it is dropped and the RHP clock.

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

None^,48S,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,48S,50i,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

F16_TMM_TOLB_OFFLINE_CNT ^5,2410,96S

VIP_TMM_OFFLINE_DROP_CNT ^48S,50i

F32_TMM_PORT_OFFLINE ^{6,32T,48T,96T,20i}

F64_TMM_PORT_OFFLINE_DROP ^{7,24V,48V,96V}

These are packets that arrive at an interface to be transmitted but before they can be sent the interface goes down. This occurs when ports go down without warning. Normally it is just be one or two at a time.

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

None^,48S,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,96S,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

Counter Name

Description

Commands

Additional Info

FCP_SW_CNTR_CREDIT_LOSS^{5,2410,96S,48S,50i}

F32_MAC_KLM_CNTR_CREDIT_LOSS^{6,32T,48T,96T,20i}

F64_CMON_CREDIT_LOSS_CH0_TMR2_HIT^{7,24V,48V,96V}

F64_CMON_CREDIT_LOSS_CH3_TMR2_HIT^{7,24V,48V,96V}

Count of the number of times that creditmon credit loss recovery has been invoked on a port

This is set to 1 full second on F ports and 1.5 seconds on E ports.

CH0 is channel zero when the link is in R_RDY mode

CH3 is the ‘normal’ VL when the link is in ER_RDY mode

Sup Hardware internal errors:

show hardware internal errors <module x> ^{5,2410,6,7,Note1}

None^,48S,50i

show hardware internal errors ^{96S,32T,48T,96T,20i,24V,48V,96V}

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{,5,2410,6,7,note1}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{96S,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

FCP_SW_CNTR_FORCE_TIMEOUT_ON^{5,50i,48S,note2}

Count of the number of times the "system timeout no-credit-drop threshold" has been reached by this port; when a port is at zero Tx B2B credits for the time specified, the port starts to drop packets at line rate

Note 1: For the 9700 and 9250i these counters will only increment prior to the introduction of the HW slow drain feature in 6.2(9). Since the 9148S was first supported in NX-OS 6.2(9) it will never have it increment. Reference the

F16_TMM_PORT_STUCK_FORCE_TIMEOUT_L_H_CNT⁵  and VIP_TMM_STK_PRT_TO_TRANSITION_CH0_CNT^48S,50i counters which indicate the same thing (but are not in OBFL).

Sup Hardware internal errors:

show hardware internal errors <module x> ^5,Note1

None^,48S,50i

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^5,note1

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^5,50i,48S

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^5,50i,48S

OBFL:

show logging onboard <module x> error-stats ^5,50i,48S

show logging onboard <module x> counter-stats ^5,50i,48S

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2: After NX-OS 6.2(9) this counter is not incrementing on MDS 9700, 9250i, 9148S:

Cisco bug ID CSCus93140     no-credit-drop SW counters not incrementing on MDS 9700, 9250i, 9148S

Integrated in: None. Bug is Closed.

FCP_SW_CNTR_FORCE_TIMEOUT_OFF^{5,50i,96S,48s,note 2}

Count of the number of times that the port has recovered from the system timeout no-credit-drop condition; this status typically means that R_Rdy primitive has been returned or possibly that an LR and LRR even has occurred.

Note 1: For the 9700 and 9250i these counters will only increment prior to the introduction of the HW slow drain feature in 6.2(9). Since the 9148S was first supported in NX-OS 6.2(9) they will never increment. They are re-added into hardware internal statistics and logging onboard error-stats via bug  CSCus93140 :

CSCus93140 no-credit-drop SW counters not incrementing on MDS 9700, 9250i, 9148S

F16_TMM_PORT_STUCK_FORCE_TIMEOUT_H_L_CNT⁵  indicates the same thing.

Sup Hardware internal errors:

show hardware internal errors <module x> ^5,Note1

None^,48S,50i

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^5,note1

Linecard  Hardware Internal Errors:

slot x show hardware internal errors ^5,50i,48S

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> errors | all <port x> ^5,50i,48S

OBFL:

show logging onboard <module x> error-stats ^5,50i,48S

show logging onboard <module x> counter-stats ^5,50i,48S

Note1: If module is not specified the default is the module number of the active supervisor. 

Note2: After NX-OS 6.2(9) this counter is not incrementing on MDS 9700, 9250i, 9148S:

Cisco bug ID CSCus93140     no-credit-drop SW counters not incrementing on MDS 9700, 9250i, 9148S

Integrated in: None. Bug is Closed.

FCP_SW_CNTR_LINK_RESET_OUT^{5,2410,96S,50i,48S, note 1}

F32_MAC_KLM_CNTR_LINK_RESET_OUT^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_LINK_RESET_OUT^{7,24V,48V,96V}

Count of times a Link Credit Reset(LR) was transmitted from the interface. This occurs during credit loss recovery.

Also shown in the output of “show interface counters detail” as:

“Tx Link Reset(LR) while link is active”

Note: Link Resets that are transmitted are only counted when the link is active.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

Integrated in: None. Bug is Closed not fixed.

FCP_SW_CNTR_LINK_RESET_IN^{5,2410,96S,50i,48S,Note 1}

F32_MAC_KLM_CNTR_LINK_RESET_IN^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_LINK_RESET_IN^{7,24V,48V,96V}

Count of times a Link Credit Reset(LR) was received on the interface.

Note 1: These are not incremented.

Also shown in the output of “show interface counters detail” as:

“Rx Link Reset(LR) while link is active”

Note: Link Resets that are received are only counted when the link is active.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

Integrated in: None. Bug is Closed not fixed.

FCP_SW_CNTR_LRR_OUT^{5,48,50i,48S,Note1}

F32_MAC_KLM_CNTR_LRR_OUT^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_LRR_OUT^{7,24V,48V,96V}

Count of times a Link Credit Reset Response(LRR) was transmitted from the interface.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

Integrated in: None. Bug is Closed not fixed.

FCP_SW_CNTR_LRR_IN^{5,48,50i,48S,Note1}

F32_MAC_KLM_CNTR_LRR_IN^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_LRR_IN^7,24V8V,96V

Count of times a Link Credit Reset Response(LRR) was received on the interface.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

Integrated in: None. Bug is Closed not fixed.

FCP_SW_CNTR_OLS_OUT^{5,48,60i,48S,Note1}

F32_MAC_KLM_CNTR_OLS_OUT^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_OLS_OUT^{7,24V,48V,96V}

Count of times an Off Line Sequence(OLS) was transmitted from the interface.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

Integrated in: None. Bug is Closed not fixed.

FCP_SW_CNTR_OLS_IN^{5,48,50i,48S,Note1}

F32_MAC_KLM_CNTR_OLS_IN^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_OLS_IN^{7,24V,48V,96V}

Count of times an Off Line Sequence(OLS) was received on the interface.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

Integrated in: None. Bug is Closed not fixed.

FCP_SW_CNTR_NOS_OUT^{5,48,50i,48S,Note1}

F32_MAC_KLM_CNTR_NOS_OUT^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_NOS_OUT^{7,24V,48V,96V}

Count of times an Not Operational Sequence(NOS) was transmitted from the interface.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

FCP_SW_CNTR_NOS_IN^{5,48,50i,48S,Note1}

F32_MAC_KLM_CNTR_NOS_IN^{6,32T,48T,96T,20i}

F64_MAC_KLM_CNTR_NOS_IN^{7,24V,48V,96V}

Count of times an Not Operational Sequence(NOS) was received on the interface.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics:

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^,6,7,note1

None ^{5,2410,96S,48S,50i}

show hardware internal statistics  device  <fcmac | all>  errors|all <port x> ^{32T,48T,96T,20i,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard  Hardware Internal Statistics:

slot x show hardware internal statistics device <fcmac | all> all <port x> ^{6,7,32T,48T,96T,20i,24V,48V,96V}

None ^{5,2410,96S,48S,50i}

OBFL:

show logging onboard <module x> error-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

show logging onboard <module x> counter-stats ^{5,2410,6,7,32T,48T,96T,20i,24V,48V,96V}

Note1: These are not incremented.

Cisco bug ID CSCus99138 Port software counters not incrementing

Counter Name

Description

Commands

Additional Info

F16_TMM_CH_CNT_CH0^5,2410,96S

F16_TMM_CH_CNT_CH1^5,2410,96S

F16_TMM_CH_CNT_CH2^5,2410,96S

F16_TMM_CH_CNT_CH3^5,2410,96S

F32_TMM_PORT_CH0_TX_NP^{6,32T,48T,96T,20i}

F32_TMM_PORT_CH1_TX_NP^{6,32T,48T,96T,20i}

F32_TMM_PORT_CH2_TX_NP^{6,32T,48T,96T,20i}

F32_TMM_PORT_CH3_TX_NP^{6,32T,48T,96T,20i}

F64_TMM_PORT_CH0_TX_NP^{7,24V,48V,96V}

F64_TMM_PORT_CH1_TX_NP^{7,24V,48V,96V}

F64_TMM_PORT_CH2_TX_NP^{7,24V,48V,96V}

F64_TMM_PORT_CH3_TX_NP^{7,24V,48V,96V}

Total transmitted frame counts on CHx (or VLx).

For F16 and F32 CH0 – CH3 indicates the virtual channel or virtual link. If the link is in R_RDY mode then only VL0(CH0) will be used. If the link is in ER_RDY mode then they will increment according to which VL it is occurring on. Specifically:

CH0 – VL0 – This is the Control virtual link

CH1 – VL1 – This is the High priority virtual link (currently not used)

CH2 – VL2 – This is the Slow device virtual link

CH3 – VL3 – This is the Normal virtual link

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,Note1}

show hardware internal statistics device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard hardware internal statistics

show hardware internal statistics device fcmac all <port x>^{5,2410,96S,6,32T,48T,96T,20i,7,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

F16_RHP_NP_TOTRG_CH0^5,96S

F16_RHP_NP_TOTRG_CH1^5,96S

F16_RHP_NP_TOTRG_CH2^5,96S

F16_RHP_NP_TOTRG_CH3^5,96S

F32_RHP_NP_TOTRG_CH0⁶

F32_RHP_NP_TOTRG_CH1⁶ F32_RHP_NP_TOTRG_CH2⁶ F32_RHP_NP_TOTRG_CH3⁶

F64_RHP_NP_TOTRG_CH0^{7,24V,48V,96V}

F64_RHP_NP_TOTRG_CH1^{7,24V,48V,96V}

F64_RHP_NP_TOTRG_CH2^{7,24V,48V,96V}

F64_RHP_NP_TOTRG_CH3^{7,24V,48V,96V}

Total received frame counts on CHx (or VLx).

CH0 – CH3 indicates the virtual channel or virtual link. If the link is in R_RDY mode then only VL0(CH0) will be used. If the link is in ER_RDY mode then they will increment according to which VL it is occurring on. Specifically:

CH0 – VL0 – This is the Control virtual link

CH1 – VL1 – This is the High priority virtual link (currently not used)

CH2 – VL2 – This is the Slow device virtual link

CH3 – VL3 – This is the Normal virtual link

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,Note1}

show hardware internal statistics device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard hardware internal statistics

show hardware internal statistics device fcmac all <port x>^{5,2410,96S,6,32T,48T,96T,20i,7,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

VIP_RHP_FRM_CNT^48S,50i

Total received frame counts

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,Note1}

show hardware internal statistics device  <fcmac | all>  all <port x> ^{96S,32T,48T,96T,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard hardware internal statistics

show hardware internal statistics device fcmac all <port x>^{5,2410,96S,6,32T,48T,96T,20i,7,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

VIP_TMM_CH0_CNT^48S,50i

Total transmitted frame counts

Sup Hardware internal errors:

None

Sup Hardware internal statistics

show hardware internal statistics device  <fcmac | all>  all <port x> ^48S,50i

Linecard  Hardware Internal Errors:

None (not an error)

Linecard hardware internal statistics

show hardware internal statistics device fcmac all <port x>^{5,2410,96S,6,32T,48T,96T,20i,7,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

IP_FCMAC_CNT_STATS_CREDIT_IG_XG_MUX_SEND^{48S,50i, 5,2410,96S}

IP_FCMAC_CREDIT_IG_XG_MUX_SEND_RRDY_REQ^{6,32T,48T,96T,20i,7,24V,48V,96V}

Count of RRDYs sent out by the port to the peer. Ideally equal to the TOTAL_RX frame count.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,Note1}

show hardware internal statistics device  <fcmac | all>  all <port x> ^{96S,48S,50i,32T,48T,96T,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard hardware internal statistics

show hardware internal statistics device fcmac all <port x>^{5,2410,96S,48S,50i,6,32T,48T,96T,20i,7,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

IP_FCMAC_CNT_STATS_CREDIT_EG_DEC_RRDY^{48S,50i, 5,2410,96S}

IP_FCMAC_CREDIT_EG_DEC_RRDY^{6,32T,48T,96T,20i,7,24V,48V,96V}

Count of RRDYs received by the port from the peer. Ideally be equal to the TOTAL_TX frame count.

Sup Hardware internal errors:

None (not an error)

Sup Hardware internal statistics

show hardware internal statistics <module x | module-all>  device  <fcmac | all>  all <port x> ^{5,2410,6,7,Note1}

show hardware internal statistics device  <fcmac | all>  all <port x> ^{96S,48S,50i,32T,48T,96T,24V,48V,96V}

Linecard  Hardware Internal Errors:

None (not an error)

Linecard hardware internal statistics

show hardware internal statistics device fcmac all <port x>^{5,2410,96S,48S,50i,6,32T,48T,96T,20i,7,24V,48V,96V}

Note1: If module is not specified the default is the module number of the active supervisor. 

Counter Name

Description

Commands

Additional Info

fcIfTxWaitCount^5,note1

fcIfTxWaitCount^{50i,48S,note2}

fcIfTxWaitCount^{2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.15

The number of times the FC-port waited due to lack of transmit credits and there were packets queued for transmit. This is in units of 2.5us.

To calculate seconds user: txwait * 2.5 /1000000

There is no OID for the Rx direction of this.

Not generated by port-monitor

Based on the counters:

F16_TMM_PORT_TWAIT_CNT^5,2410,96S 

VIP_TMM_TXWAIT_CH0_CNT^{50i, 48S}

VIP_TMM_TXWAIT_CH1_CNT^{50i, 48S}

F32_TMM_PORT_TWAIT RRDY^{6,32T,48T,96T,20i}

F64_TMM_PORT_TWAIT^{7,24V,48V,96V}

Displayed via:

Show interface fcx/y counters detailed | i wait

Example:

F241-15-09-9710-1# show interface fc10/1 counters detailed

…

fc10/1

…

Congestion Stats:

  Tx Timeout discards:                                                       0

  Tx Credit loss:                                                            0

  TxWait 2.5us due to lack of transmit credits:                              0

  Percentage TxWait for last 1s/1m/1h/72h:                         0%/0%/0%/0%

  RxWait 2.5us due to lack of receive credits:                              25

Note1: Prior to 6.2(11a) this counter was inaccurate. See Cisco bug ID

CSCus15233 fcIfTxWaitCount incorrect on DS-X9232-256K9 and DS-X9248-768K9

Fixed in 6.2(11a)

Note2: Prior to 6.2(11a) this counter was inaccurate. See bug:

CSCus15745 fcIfTxWaitCount incorrect for MDS 9250i and 9148S

Fixed in 6.2(11a)

fcIfRxWaitCount^{6,24V,48V,96V,Note1}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.48

The amount of time in 2.5us (microsecond) intervals that the interface was at 0 Rx B2B credits remaining. This indicates the switchport was preventing further input frames for a time.

Unlike TxWait, which uses the much more granular than 2.5us, underlying TWAIT counter, RxWait will only increment when the duration of time is 2.5us or longer. Small durations of 0 Rx B2B credit remaining will not be counted. This means that TxWait on the adjacent side will always be a better measure if it is available.

Not currently used by port-monitor

Only available on 64G capable interfaces.

Displayed via:

Show interface fcx/y counters detailed | i wait

Example:

F241-15-09-9710-1# show interface fc10/1 counters detailed

…

fc10/1

…

Congestion Stats:

  …

  RxWait 2.5us due to lack of receive credits:                              25

  Percentage RxWait for last 1s/1m/1h/72h:                         0%/0%/0%/0%

Note1: Cisco bug ID CSCwe21119

Create fcIfRxWaitCount SNMP variable

Integrated in 9.4(5a)

fcIfCreditLoss^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.37

The number of link resets that have occurred due to unavailable
credits from the peer side of the link.

Used by port-monitor counter credit-loss-reco

fcIfLinkResetOuts^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.10

The number of link reset protocol errors issued by
the FC-Port to the attached FC-Port.

Used by port-monitor counter lr-tx

fcIfLinkResetIns^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.9

The number of link reset protocol errors received by the FC-Port from the attached FC-port

Used by port-monitor counter lr-rx

fcIfTimeOutDiscards^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.35

The number of packets that are dropped due to time-out at the FC-port or due to the FC-port going offline.

Used by port-monitor counter timeout-discards

fcIfOutDiscards^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.36

The total number of packets that are discarded in the egress side of the FC-port. This is an aggregate counter comprised of several types of drops.

Used by port-monitor counter tx-discards

fcIfTxWtAvgBBCreditTransitionToZero^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.38

Count of the number of times that an interface was at zero Tx B2B credits for 100 ms. This status typically indicates congestion at the device attached on that interface.

Used by port-monitor counter tx-credit-not-available

See counter xxx_CNTR_TX_WT_AVG_B2B_ZERO

for more info:

Note: There is no OID in the Rx direction.

See Cisco bug ID

CSCus93323 Portmonitor  fcIfTxWtAvgBBCreditTransitionToZero truncates hcAlarmOwner

fcIfBBCreditTransistionFromZero^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.28

Increments if the transmit B2B credit is zero

There is no indication of time at zero for this counter. It could stay at zero for just an instant or for an extended duration of time.

Also shown in the output of show interface counters:

xxxx Transmit B2B credit transitions to zero

Not currently used by port-monitor

Based off of the TBBZ hardware statistic.

fcHCIfBBCreditTransistionFromZero^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.40

Increments if the transmit B2B credit is zero

There is no indication of time at zero for this counter. It could stay at zero for just an instant or for an extended duration of time.

Also shown in the output of show interface counters:

xxxx Transmit B2B credit transitions to zero

Not currently used by port-monitor

Based off of the TBBZ hardware statistic.

fcIfBBCreditTransistionToZero^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.39

Increments if the receive B2B credit is zero

There is no indication of time at zero for this counter. It could stay at zero for just an instant or for an extended duration of time.

Also shown in the output of show interface counters:

xxxx Receive B2B credit transitions to zero

Not currently used by port-monitor

Based off of the RBBZ hardware statistic.

fcHCIfBBCreditTransistionToZero^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

OID 1.3.6.1.4.1.9.9.289.1.2.1.1.41

Increments if the receive B2B credit is zero

There is no indication of time at zero for this counter. It could stay at zero for just an instant or for an extended duration of time.

Also shown in the output of show interface counters:

xxxx Receive B2B credit transitions to zero

Not currently used by port-monitor

Based off of the RBBZ hardware statistic.

Counter Name

Description

Commands

Additional Info

F16_FCP_INTR_TMM_P_CWAIT_FORCE_TIMEOUT_L_H^5,2410

Slowport condition detected count  (Low to High transition: That is, credit wait (cwait) > threshold)

NX-OS 6.2(1) through 6.2(7) -  Count of times port was at zero Tx credits for 100ms. Only increments on the initial 100ms interval. . In these “pre-slowport-monitor” releases this counter was used to trigger the FCP_SW_CNTR_TX_WT_AVG_B2B_ZERO counter increment.

NX-OS 6.2(9) and later – Will not occur.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

F16_FCP_INTR_TMM_P_CWAIT_FORCE_TIMEOUT_H_L^5,2410

Slowport condition exited count (High to Low transition: ie creditwait (cwait) < threshold)

NX-OS 6.2(1) through 6.2(7) - Count of times port received a credit after being at zero Tx credits for 100ms or longer. In these “pre-slowport-monitor” releases this counter was used to re-arm the FCP_SW_CNTR_TX_WT_AVG_B2B_ZERO counter increment.

NX-OS 6.2(9) and later – Will not occur.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

F16_FCP_INTR_TMM_P_STUCK_FORCE_TIMEOUT_L_H^5,2410

Stuck port condition detected count  (Low to High transition.

Configured via:

“system timeout slowport-monitor xxx mode|f”

Defaults to 500ms with no action.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

F16_FCP_INTR_TMM_P_STUCK_FORCE_TIMEOUT_H_L^5,2410

Count of times stuck port condition exited.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

VIPER_FCP_INTR_TMM_CWAIT_AVG_LIVE_CH0_OVER_THRESHOLD_RAISING^48s,50i

VIPER_FCP_INTR_TMM_CWAIT_AVG_LIVE_CH1_OVER_THRESHOLD_RAISING^48s,50i

Slowport condition detected count  (Low to High transition: That is. credit wait (cwait) > threshold)

NX-OS 6.2(5) through 6.2(7) -  Count of times port was at zero Tx credits for 100ms. Only increments on the initial 100ms interval. . In these “pre-slowport-monitor” releases this counter was used to trigger the FCP_SW_CNTR_TX_WT_AVG_B2B_ZERO counter increment.

NX-OS 6.2(9) and later – Will not occur.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

Note: VIPER does not have a High to Low interrupt like F16.

VIPER_FCP_INTR_TMM_CWAIT_AVG_LIVE_CH0_OVER_THRESHOLD_FALLING^48s,50i

VIPER_FCP_INTR_TMM_CWAIT_AVG_LIVE_CH1_OVER_THRESHOLD_FALLING^48s,50i

Slowport condition detected count exited.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

Note: These are displayed in OBFL with the  VIPER_FCP_INTR_ prefix but without the prefix in other places.

VIPER_FCP_INTR_TMM_STUCK_PORT_TIMEOUT_CH0_RAISING^48s,50i

VIPER_FCP_INTR_TMM_STUCK_PORT_TIMEOUT_CH1_RAISING^48s,50i

Count of times port was at zero Tx credits for the stuck port timeout value ”no-credit-drop” (default value 500ms).

NX-OS 6.2(5) through 6.2(7) -  Count of times port was at zero Tx credits for 1s(F port) or 1.5s(E port). In these “pre-slowport-monitor” releases this interrupt was used to trigger the  FCP_SW_CNTR_CREDIT_LOSS counter increment.

NX-OS 6.2(9) and later – Should not occur.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

VIPER_FCP_INTR_TMM_STUCK_PORT_TIMEOUT_CH0_FALLING^48s,50i VIPER_FCP_INTR_TMM_STUCK_PORT_TIMEOUT_CH1_FALLING^48s,50i

Count of times stuck port condition exited.

OBFL:

show logging onboard interrupt-stats

Linecard:

Slot x show hardware internal fcmac port x  interrupt-counts

IP_FCMAC_INTR_PRIM_RX_SEQ_NOS^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

Not Operational Sequence received on the interface.

NOS is a primitive sequence that is transmitted continuously until a OLS is received.

OBFL:

show logging onboard interrupt-stats

Linecard:

show hardware internal fc-mac port 1 interrupt-counts^2,3,48

Slot x show hardware internal fcmac port x interrupt-counts^4,5,48s,50i

show interface fcx/y counters details

show interface detailed-counters

xxx non-operational sequences received

IP_FCMAC_INTR_PRIM_RX_SEQ_OLS^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

Off Line Sequence received on the interface.

OLS is a sequence that is transmitted continuously until a LR is received.

OBFL:

show logging onboard interrupt-stats

Linecard:

show hardware internal fc-mac port 1 interrupt-counts^2,3,48

Slot x show hardware internal fcmac port x interrupt-counts^4,5,48s,50i

show interface fcx/y counters details

show interface detailed-counters

xxx Offline Sequence errors received

IP_FCMAC_INTR_PRIM_RX_SEQ_LR^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

Link Reset received on the interface. LR is sent under two conditions normally:

1)      Link bringup – NPS/OLS/LR/LRR

2)      Credit Loss Recovery – LR is sent to bring each side up to its full complement of B2B credits. This doesn’t bounce or flap the link but just restore the B2B credits.

OBFL:

show logging onboard interrupt-stats

Linecard:

show hardware internal fc-mac port 1 interrupt-counts^2,3,48

Slot x show hardware internal fcmac port x interrupt-counts^4,5,48s,50i

show interface fcx/y counters details

show interface detailed-counters

xxx link reset protocol errors received

IP_FCMAC_INTR_PRIM_RX_SEQ_LRR^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

Link Reset received on the interface. This is sent in response to a Link Reset.

OBFL:

show logging onboard interrupt-stats

Linecard:

show hardware internal fc-mac port 1 interrupt-counts^2,3,48

Slot x show hardware internal fcmac port x interrupt-counts^{4,5,6,48s,50i}

show interface fcx/y counters details

show interface detailed-counters

xxx link reset responses received

IP_FCMAC_INTR_ERR_BB_SCR_INCREMENT ^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

Credit recovery action due to R_RDY loss

This interrupt count captures how many times the port detected R_RDY(s) loss between BB_SCr checkpoints and corrected the credit accounting by internally incrementing the TX B2B credit counter. It does not indicate how many credits were actually lost.

Defined in FC-FS-3

17.4.9 BB_Credit Recovery

Note: BBSCN is a long standing feature on E ports that is enabled by default. To disable:

Interface fcx/y

no switchport fcbbscn

NX-OS 8.2(1) added support for F ports.

NX-OS 8.4(1) added support for NP ports.

On NX-OS 8.4(1) if enabled the interface shows this:

B2B State Change: Admin(on), Oper(up), Negotiated Value(14)

Does not function when interface is in ER_RDY mode. But it still displays as on.

OBFL:

show logging onboard interrupt-stats

Linecard:

show hardware internal fc-mac port 1 interrupt-counts^2,3,48

Slot x show hardware internal fcmac port x interrupt-counts^{4,5,6,48s,50i}

Displayed via:

Show interface fcx/y counters detailed | i wait

Example:

F241-15-09-9710-1# show interface fc10/1 counters detailed

fc10/1

…

Link Stats:

…

  BB_SCs credit resend actions:                            0

  BB_SCr Tx credit increment actions:                 0

NX-OS 8.2(1) added BBSCN support for F ports

NX-OS 8.4(1) added BBSCN support for NP ports and Buffer-to-Buffer State Change SOF (BB_SCs) and Buffer-to-Buffer State Change R_RDY (BB_SCr) counter information in the show interface counters detailed command.

MDS9710-1# show interface fc1/13 counters detailed | i i BB

      0 BB_SCs credit resend actions, 0 BB_SCr Tx credit increment actions

IP_FCMAC_INTR_ERR_BB_SCS_RESEND^{5,2410,6,96S,32T,48T,96T,20i,7,24V,48V,96V}

Credit recovery action due to Frame Loss (SOF corruption)

This interrupt count captures how many times the port detected frame(s) loss between BB_SCs checkpoints and corrected the peer ports credit accounting by sending extra R_RDY(s) to the peer that is equivalent the number of frame(s) lost. It does not indicate the actual number of frames lost.

Defined in FC-FS-3

17.4.9 BB_Credit Recovery

Note: BBSCN is a long standing feature on E ports that is enabled by default. To disable:

Interface fcx/y

no switchport fcbbscn

NX-OS 8.2(1) added support for F ports.

NX-OS 8.4(1) added support for NP ports.

On NX-OS 8.4(1) if enabled the interface shows this:

B2B State Change: Admin(on), Oper(up), Negotiated Value(14)

Does not function when interface is in ER_RDY mode. But it still displays as on.

OBFL:

show logging onboard interrupt-stats

Linecard:

show hardware internal fc-mac port 1 interrupt-counts^2,3,48

Slot x show hardware internal fcmac port x interrupt-counts^{4,5,6,48s,50i}

Displayed via:

Show interface fcx/y counters detailed | i wait

Example:

F241-15-09-9710-1# show interface fc10/1 counters detailed

fc10/1

…

Link Stats:

…

  BB_SCs credit resend actions:                             0

  BB_SCr Tx credit increment actions:                  0

NX-OS 8.2(1) added BBSCN support for F ports

NX-OS 8.4(1) added BBSCN support for NP ports and Buffer-to-Buffer State Change SOF (BB_SCs) and Buffer-to-Buffer State Change R_RDY (BB_SCr) counter information in the show interface counters detailed command.

MDS9710-1# show interface fc1/13 counters detailed | i i BB

      0 BB_SCs credit resend actions, 0 BB_SCr Tx credit increment actions

F32_FCP_INTR_RCM_PORTxx_CH0_SLOW_STUCK1 ^{6,32T,48T,96T}

In R_RDY mode:

Increments fcIfRxWtAvgBBCreditTransistiontoZero

This is the 100ms ingress congestion indicator

In ER_RDY mode:

Increments channel 0 (VL0) Rx transitions to zero

Port number is the internal port number in the F64 ASIC as seen in  `show hardware internal fcmac drv-info`.

OBFL:

`show logging onboard interrupt-stats`

Linecard:

`show hardware internal fcmac inst x interrupt-counts`

PORTxx is not the front panel port number. It is the internal port number as seen in ‘show hardware internal fcmac port x port-info”

F64_FCP_INTR_TMM4_Px_CH0_TMR2_THR

Increments fcIfCreditLoss

In R_RDY mode:

Indicates the credit-loss-recovery timer (either 1 second(F) or 1.5 seconds(E)) has expired. Credit-loss-recovery is initiated.

In ER_RDY mode:

Indicates the credit-loss-recovery timer (either 1 second or 1.5 seconds) has expired in channel 0 (VL0).

Port number is the internal port number in the F64 ASIC as seen in  `show hardware internal fcmac drv-info`.

OBFL:

`show logging onboard interrupt-stats`

Linecard:

`show hardware internal fcmac inst x interrupt-counts`