The implementation of Link Aggregation (LAG) changes the behavior of "show port npu counters" and "show port utilization table" commands. The port commands are important for troubleshooting port and throughput related issues and so it is important to be able to properly interpret their output, especially since it is unintuitive when compared to non-LAG ports. The bottom line is that port npu counters for LAG on an individual port basis are not available and are reported for the entire LAG group only up to at least StarOS v18 which is the time of this writing. This could change in future releases.
Due to design/architectural limitations, reporting of port npu counters is limited to the conglomeration of all the ports in a LAG group and not at the individual port level. This does not apply to port datalink counters which continue to report as expected.
Because the implementation of LAG requires all the ports in the LAG to be active, "show port utilization table" reports utilization for all the LAG ports whether they are distributing (active) or agreed (standby) for both ASR 5000/5500. Sidenote: Normally agreed ports show no traffic, but there have been instances where the Rx and/or Tx direction of agreed ports are also carrying traffic (not the subject of this article but just pointing it out).
Meanwhile for non-LAG ports, there is a difference between what is reported for ASR 5000 vs. ASR 5500. ASR 5000 does not report utilization for standby ports, while ASR 5500 does report utilization for standby ports (even though those ports are operationally down)
Consistent with what has just been mentioned, "show port table" for LAG reports all the ports as operationally up, compared to non-LAG where only the active port of a port pair is operationally up.
For "show port npu counters", ALL LAG ports are listed, but the following is true:
- ASR 5000:
- the counters under the primary (configured) port are a TOTAL count across all the currently ACTIVE ports
- the counters for ALL other ports (including the primary port's pair) are not relevant and should not be used
- ASR 5500:
- the counters under the primary port and its standby are a TOTAL count across all the currently ACTIVE ports (they will both report a similar but slightly different value - use either one)
- the counters for ALL other ports are 0s
For NON-LAG ports, only counters for active ports are reported. Standby ports are not even listed in the output at the NPU level (and never have been).
The output here is to support the previous explanations. It is based on hardware configurations as follows:
ASR 5000: LAG Ports 19/20, 23/26, 27/28, and non-LAG ports 21/37
ASR 5500: LAG Ports 5/ 10, 11, 15, 16; 6/ 10, 11, 15, 16, and non-LAG ports 5/28 & 6/28, 5/29 & 6/29
Reminder: Focus of this article are the counters for LAG ports.
******** show port utilization *******
Wednesday May 28 12:28:04 UTC 2014
------ Average Port Utilization (in mbps) ------
Port Type Current 5min 15min
Rx Tx Rx Tx Rx Tx
----- ------------------------ ------- ------- ------- ------- ------- -------
19/1 10G Ethernet 514 572 503 534 490 517
20/1 10G Ethernet 0 0 0 0 0 0
21/1 1000 Ethernet 0 0 0 0 0 0
23/1 10G Ethernet 460 529 448 516 431 510
26/1 10G Ethernet 0 0 0 0 0 0
27/1 10G Ethernet 674 532 634 519 619 499
28/1 10G Ethernet 0 0 0 0 0 0
******** show port table all *******
Wednesday May 28 12:28:03 UTC 2014
Port Role Type Admin Oper Link State Pair Redundant
----- ---- ------------------------ -------- ---- ---- ------- ----- ---------
19/1 Srvc 10G Ethernet Enabled - Up - None LA+ 19/1
Untagged Enabled Up - Active - -
Tagged VLAN 2423 Enabled Up - Active - -
Tagged VLAN 2424 Enabled Up - Active - -
Tagged VLAN 2401 Enabled Up - Active - -
Tagged VLAN 2009 Enabled Up - Active - -
Tagged VLAN 2010 Enabled Up - Active - -
Tagged VLAN 2007 Enabled Up - Active - -
Tagged VLAN 2498 Enabled Up - Active - -
Tagged VLAN 2499 Enabled Up - Active - -
20/1 Srvc 10G Ethernet Enabled Up Up Active None LA~ 19/1
21/1 Srvc 1000 Ethernet Enabled - Up - 37/1 L2 Link
Untagged Enabled Down - Active - -
Tagged VLAN 30 Enabled Up - Active - -
23/1 Srvc 10G Ethernet Enabled Up Up Active None LA+ 19/1
26/1 Srvc 10G Ethernet Enabled Up Up Active None LA~ 19/1
27/1 Srvc 10G Ethernet Enabled Up Up Active None LA+ 19/1
28/1 Srvc 10G Ethernet Enabled Up Up Active None LA~ 19/1
37/1 Srvc 1000 Ethernet Enabled - Up - 21/1 L2 Link
Untagged Enabled Down - Standby - -
Tagged VLAN 30 Enabled Down - Standby - -
******** show port npu counters *******
Counters for port 19/1
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
-------------------- ------------- --------------- ------------- ---------------
Unicast 74783944546254086740066587874 69151428800023783215178712378
Counters for port 20/1
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 23/1
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 26/1
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 27/1
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 28/1
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
while for NON-LAG, only the active ports are listed and those values ARE relevant:
Counters for port 21/1 Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
[local]PGW> show port utilization table
Sunday June 01 03:57:59 UTC 2014
------ Average Port Utilization (in mbps) ------
Port Type Current 5min 15min
Rx Tx Rx Tx Rx Tx
----- ------------------------ ------- ------- ------- ------- ------- -------
5/10 10G Ethernet 1919 1973 1982 2066 2025 2094
5/11 10G Ethernet 1911 1751 1976 1828 2023 1883
5/15 10G Ethernet 1910 2064 1975 2064 2004 2130
5/16 10G Ethernet 1933 1943 1987 2012 2014 2019
5/28 10G Ethernet 9 69 9 70 9 71
5/29 10G Ethernet 0 0 0 0 0 0
6/10 10G Ethernet 0 0 0 0 0 0
6/11 10G Ethernet 0 0 0 0 0 0
6/15 10G Ethernet 0 0 0 0 0 0
6/16 10G Ethernet 0 0 0 0 0 0
6/28 10G Ethernet 0 0 0 0 0 0
6/29 10G Ethernet 1 0 1 10 1 11
[local]PGW> show port table all
Sunday June 01 03:58:48 UTC 2014
Port Role Type Admin Oper Link State Pair Redundant
----- ---- ------------------------ -------- ---- ---- ------- ----- ---------
5/10 Srvc 10G Ethernet Enabled - Up - 6/10 LA+ 5/10
Untagged Enabled Up - Active - -
Tagged VLAN 2011 Enabled Up - Active - -
Tagged VLAN 2405 Enabled Up - Active - -
Tagged VLAN 2015 Enabled Up - Active - -
Tagged VLAN 2427 Enabled Up - Active - -
Tagged VLAN 2407 Enabled Up - Active - -
Tagged VLAN 2455 Enabled Up - Active - -
5/11 Srvc 10G Ethernet Enabled Up Up Active 6/11 LA+ 5/10
5/15 Srvc 10G Ethernet Enabled Up Up Active 6/15 LA+ 5/10
5/16 Srvc 10G Ethernet Enabled Up Up Active 6/16 LA+ 5/10
5/28 Srvc 10G Ethernet Enabled - Up - 6/28 L2 Link
Untagged Enabled Up - Active - -
Tagged VLAN 2400 Enabled Up - Active - -
5/29 Srvc 10G Ethernet Enabled - Up - 6/29 L2 Link
Untagged Enabled Down - Standby - -
Tagged VLAN 31 Enabled Down - Standby - -
6/10 Srvc 10G Ethernet Enabled - Up - 5/10 LA~ 5/10
Untagged Enabled Up - Active - -
Tagged VLAN 2011 Enabled Up - Active - -
Tagged VLAN 2405 Enabled Up - Active - -
Tagged VLAN 2015 Enabled Up - Active - -
Tagged VLAN 2427 Enabled Up - Active - -
Tagged VLAN 2407 Enabled Up - Active - -
Tagged VLAN 2455 Enabled Up - Active - -
6/11 Srvc 10G Ethernet Enabled Up Up Active 5/11 LA~ 5/10
6/15 Srvc 10G Ethernet Enabled Up Up Active 5/15 LA~ 5/10
6/16 Srvc 10G Ethernet Enabled Up Up Active 5/16 LA~ 5/10
6/28 Srvc 10G Ethernet Enabled - Up - 5/28 L2 Link
Untagged Enabled Down - Standby - -
Tagged VLAN 2400 Enabled Down - Standby - -
6/29 Srvc 10G Ethernet Enabled - Up - 5/29 L2 Link
Untagged Enabled Up - Active - -
Tagged VLAN 31 Enabled Up - Active - -
[local]PGW> show port npu counters
Counters for port 5/10
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
-------------------- ------------- --------------- ------------- ---------------
Unicast 936150697918 636869996072149 9369282682521055230987905964
Counters for port 5/11
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
-------------------- ------------- --------------- ------------- ---------------
Unicast 0 0 0 0
Counters for port 5/15
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 5/16
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 6/10
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
-------------------- ------------- --------------- ------------- ---------------
Unicast 936156167721 636873912574349 9369336716261055237102737046
Counters for port 6/11
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 6/15
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Counters for port 6/16
Counter Rx Frames Rx Bytes Tx Frames Tx Bytes
Again, only active ports are listed by this command:
Counters for port 5/28 Counter Rx Frames Rx Bytes Tx Frames Tx Bytes Counters for port 6/29 Counter Rx Frames Rx Bytes Tx Frames Tx Bytes