Troubleshooting WS-X6348 Module Port Connectivity for Catalyst 6500/6000 Using CatOS

Available Languages

Download Options

  • PDF     (31.1 KB)
    View with Adobe Reader on a variety of devices
Updated:April 24, 2007
Document ID:29422

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Introduction

This document discusses detailed troubleshooting for the WS-X6348 module on the Catalyst 6500/6000 that runs the CatOS and the command outputs to collect before you contact Cisco Technical Support.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these software and hardware versions:

  • Catalyst 6500 with Supervisor II with Multilayer Switch Feature Card 2 (MSFC2)

  • WS-X6348 module

  • CatOS version 6.3.9

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to the Cisco Technical Tips Conventions for more information on document conventions.

Background Information

WS-X6348 Module Architecture

Each WS-X6348 card is controlled by a single Application-Specific Integrated Circuit (ASIC) that connects the module to both the 32 GB data bus backplane of the switch and to a set of four other ASICs which controls groups of 12 10/100 ports.

An understanding of this architecture is important as it can help to troubleshoot port problems. For example, if a group of 12 10/100 ports fails the online diagnostics, this typically indicates that one of the ASICs previously mentioned failed. See step 13 in order to learn more about the show test <module#> .

Known Issues

  1. Cisco bug ID CSCdu03935 (registered customers only) : 6348-RJ-45 Coil Pinnacle Header Checksum Error

    You can see this error message:

    %SYS-5-SYS_LCPERR5:Module 9: Coil Pinnacle Header Checksum Error - Port #37

    If you only see this message and no other Coil related messages in the syslogs or in the output of the show logging buff 1023 command, and the transmit is stuck on one port, not a group of 12 ports, complete these steps in order to fix the problem:

    1. Disable and enable the ports.

    2. Issue the reset <module#> command in order to soft-reset the module.

    3. Hard-reset the module with the set module power up|down <module#> command.

    If after the completion of one or more of these steps the card comes online and all the ports pass diagnostics , which is shown if you issue the show test <module#> command, and traffic starts to pass fine, then the Cisco bug ID CSCdu03935 (registered customers only) is possibly present. The fix is in these CatOS releases and later:

    • 5.5(18)

    • 6.3(10)

    • 7.4(3)

  2. You can see a message similar to one or more of these in the syslogs or show logging buff 1023 command output:

    • Coil Pinnacle Header Checksum

    • Coil Mdtif State Machine Error

    • Coil Mdtif Packet CRC Error

    • Coil Pb Rx Underflow Error

    • Coil Pb Rx Parity Error

    If you see one or more of these messages, and you have a group of 12 ports stuck and does not pass traffic, complete these steps:

    1. Disable and enable the ports.

    2. Issue the reset <module#> command in order to soft-reset the module.

    3. Hard-reset the module with the set module power up|down <module#> command.

    After the completion of steps b and/or c, contact the Cisco Technical Support with the previous information if you encounter one or more of these issues:

    • The module does not come online.

    • The module comes online, but a group of 12 ports fails diagnostics, which is seen in the output from the show test <module#> command.

    • The module is stuck in the other state when it boots up.

    • All port LEDs on the module become amber.

    • All ports are in the err-disabled state as seen when the show <module#> command is issued.

How to Troubleshoot Catalyst 6500/6000 WS-X6348 Module Port Connectivity

Complete these steps in order to perform port connectivity troubleshooting on the Catalyst 6500/6000 WS-X6348 module.

Step-by-step Instructions

Complete these steps:

  1. Check the software version in use and make sure there are no known WS-X6348 issues with that code. Verify the module is a WS-X6348 and that the status is ok. 

    esc-6509-c (enable) show module 6
Mod Slot Ports Module-Type               Model               Sub Status
--- ---- ----- ------------------------- ------------------- --- --------
6   6    48    10/100BaseTX Ethernet     WS-X6348-RJ-45      no  ok   
 
Mod Module-Name          Serial-Num
--- -------------------- -----------
6                        SAD04170FPY
 
Mod MAC-Address(es)                        Hw     Fw         Sw
--- -------------------------------------- ------ ---------- -----------------
6   00-01-97-15-03-a0 to 00-01-97-15-03-cf 1.1    5.3(1)     6.3(9)    
esc-6509-c (enable)

    In the previous command output, check the status of the module. It can be in one of these four states:

    • OK—Everything is fine.

    • power-deny—Not enough power is available to power the module.

    • other—Most likely the Serial Communication Protocol (SCP) communication does not work.

    • faulty/unknown—This indicates most likely a bad module or slot.

    • err-disabled—View the output from the show logging buffer command, as shown in step 3, in order to see if there are any messages on why the module is in the err-disabled state.

  2. Verify that the configuration for the module and its ports are correct. Make sure that options such as the set port host command, are enabled when appropriate. 

    esc-6509-c (enable) show config 6
This command shows non-default configurations only.
Use 'show config  all' to show both default and non-default configurations.
....................
begin
!
# ***** NON-DEFAULT CONFIGURATION *****
!
!
#time: Sun Oct 20 2002, 12:17:49 
!
# default port status is enable
!
!
#module 6 : 48-port 10/100BaseTX Ethernet
set vlan 175  6/1-2
end
esc-6509-c (enable)

  3. Issue the show logging buff 1023 command in order to check for any port-related error messages in the log.

    The output for this command is intentionally not shown as it is specific to each switch.

  4. Verify that dynamic Content Addressable Memory (CAM) entries are created for any traffic that enters the port to which you troubleshoot. Make sure that the CAM entry is associated with the correct VLAN. 

    esc-6509-c (enable) show cam dynamic 6/1
* = Static Entry. + = Permanent Entry. # = System Entry. R = Router Entry.
X = Port Security Entry $ = Dot1x Security Entry
 
VLAN  Dest MAC/Route Des    [CoS]  Destination Ports or VCs / [Protocol Type] 
----  ------------------    -----  -------------------------------------------
175   00-d0-06-26-f4-00             6/1 [ALL]
175   00-e0-1e-a4-88-af             6/1 [ALL]
175   00-90-6d-fb-88-00             6/1 [ALL]
175   08-00-2b-2f-f4-dc             6/1 [ALL]
175   aa-00-04-00-01-a4             6/1 [ALL]
175   08-00-2b-2f-f3-b4             6/1 [ALL]
175   00-00-0c-0b-f8-98             6/1 [ALL]
175   00-00-0c-ff-ec-c9             6/1 [ALL]
175   00-03-e3-48-a6-e0             6/1 [ALL]
175   00-05-74-19-59-8a             6/1 [ALL]
175   00-08-e2-c3-60-a8             6/1 [ALL]
175   00-50-54-7c-f2-e0             6/1 [ALL]
175   00-50-54-75-dd-74             6/1 [ALL]
175   00-50-0b-6c-b8-00             6/1 [ALL]
175   00-04-5a-6c-6a-3a             6/1 [ALL]
175   00-00-0c-34-7b-16             6/1 [ALL]
175   00-00-0c-0c-19-36             6/1 [ALL]
175   08-00-69-07-b1-c8             6/1 [ALL]
Total Matching CAM Entries Displayed  =18
esc-6509-c (enable)

  5. If a port is configured as a trunk, check to make sure it is in the correct status and that the appropriate VLANs are spanning-tree forwarding and not pruned by VLAN Trunk Protocol (VTP). For a dot1q trunk, also make sure that the native VLAN matches that of the device on the other side of the trunk. 

    esc-6509-e> (enable) show trunk 3/1
* - indicates vtp domain mismatch
Port      Mode         Encapsulation  Status        Native vlan
--------  -----------  -------------  ------------  -----------
 3/1      desirable    dot1q          trunking      1
 
Port      Vlans allowed on trunk
--------  ---------------------------------------------------------------------
 3/1      1-1005,1025-4094
 
Port      Vlans allowed and active in management domain 
--------  ---------------------------------------------------------------------
 3/1      1-50,79-81,175-176,997-999
 
Port      Vlans in spanning tree forwarding state and not pruned
--------  ---------------------------------------------------------------------
 3/1      1-50,79-81,175-176,997-999
esc-6509-e> (enable)

  6. Make sure that the port in question is forwarding for spanning-tree on the correct VLAN. Also, that portfast is enabled or disabled where appropriate. 

    esc-6509-c (enable) show spantree 6/1
Port                     Vlan Port-State    Cost      Prio Portfast Channel_id
------------------------ ---- ------------- --------- ---- -------- ----------
 6/1                     175  forwarding           19   32 disabled 0         
esc-6509-c (enable)

  7. If the port is connected to another Cisco device use Cisco Discovery Protocol (CDP) to check if the port can see the device.

    Note: CDP must be enabled on the switch and the other Cisco device. Also note that CDP is Cisco proprietary, and will not work with non-Cisco devices.

    esc-6509-c (enable) show cdp port 6/1
CDP               : enabled
Message Interval  : 60
Hold Time         : 180
Version           : V2
Device Id Format  : Other
 
Port      CDP Status
--------  ----------
 6/1      enabled     
esc-6509-c (enable)

    In this example, port 6/1 on the Catalyst 6509 switch connects to Fast Ethernet interface 0/4 on a Catalyst 3500XL.

    esc-6509-c (enable) show cdp neighbor 6/1 detail
Port (Our Port): 6/1     
Device-ID: esc-cat3500xl-1
Device Addresses:
  IP Address: 172.16.176.200
Holdtime: 150 sec
Capabilities: TRANSPARENT_BRIDGE SWITCH 
Version: 
  Cisco Internetwork Operating System Software 
  IOS (tm) C3500XL Software (C3500XL-C3H2S-M), Version 12.0(5.1)XW, MAINTENANCEE
  Copyright (c) 1986-2000 by cisco Systems, Inc.
  Compiled Thu 21-Dec-00 12:04 by devgoyal
Platform: cisco WS-C3548-XL
Port-ID (Port on Neighbors's Device): FastEthernet0/4
VTP Management Domain: sj-et
Native VLAN: unknown
Duplex: unknown
System Name: unknown
System Object ID: unknown
Management Addresses: unknown
Physical Location: unknown
esc-6509-c (enable)

    Since CDP is Cisco proprietary, care must be taken. CDP packets are sent to a well-known multicast destination MAC address 01-00-0C-CC-CC-CC. A Cisco switch not configured for CDP, or a non-Cisco switch, typically treats CDP packets like any multicast and floods them throughout the VLAN. If two Cisco switches with CDP enabled are connected through a non-CDP-capable switch, a possible result is that those two CDP-enabled switches think that they are CDP neighbors when, in fact, there is actually another switch in between them.

  8. Check the configuration, state, and health of the port in trouble. You can also issue the show port <module#> command in order to look at all the ports for a given module. 

    esc-6509-c (enable) show port 6/1
Port  Name                 Status     Vlan       Duplex Speed Type
----- -------------------- ---------- ---------- ------ ----- ------------
 6/1                       connected  175        a-full a-100 10/100BaseTX
 
Port  AuxiliaryVlan AuxVlan-Status     InlinePowered     PowerAllocated
                                   Admin Oper   Detected mWatt mA @42V
----- ------------- -------------- ----- ------ -------- ----- --------
 6/1  none          none           -     -      -        -     -
 
Port  Security Violation Shutdown-Time Age-Time Max-Addr Trap     IfIndex
----- -------- --------- ------------- -------- -------- -------- -------
 6/1  disabled  shutdown             0        0        1 disabled      99
 
Port  Num-Addr Secure-Src-Addr   Age-Left Last-Src-Addr     Shutdown/Time-Left
----- -------- ----------------- -------- ----------------- ------------------
 6/1         0                 -        -                 -        -         -
 
Port     Broadcast-Limit Multicast Unicast Total-Drop
-------- --------------- --------- ------- --------------------
 6/1                   -         -       -                    0
 
Port  Send FlowControl  Receive FlowControl   RxPause    TxPause
      admin    oper     admin     oper
----- -------- -------- --------- ---------   ---------- ----------
 6/1  off      off      off       off         0          0          
 
Port  Status     Channel              Admin Ch
                 Mode                 Group Id
----- ---------- -------------------- ----- -----
 6/1  connected  auto silent             34     0
 
Port  Align-Err  FCS-Err    Xmit-Err   Rcv-Err    UnderSize
----- ---------- ---------- ---------- ---------- ---------
 6/1           0          0          0          0         0
 
Port  Single-Col Multi-Coll Late-Coll  Excess-Col Carri-Sen Runts     Giants
----- ---------- ---------- ---------- ---------- --------- --------- ---------
 6/1           0          0          0          0         0         0         0
 
Port  Last-Time-Cleared
----- --------------------------
 6/1  Sun Oct 13 2002, 16:37:58
esc-6509-c (enable)

    Status—Can display the following states:

    • connected

    • notconnect

    • connecting

    • standby

    • faulty

    • inactive

    • shutdown

    • disabled

    • err-disabled

    • monitor

    • active

    • dot1p

    • untagged

    • inactive

    • onhook

    If a port is in the notconnect state, check the cabling as well as the device connected to the other end. If a port is in the faulty state, it indicates a hardware problem. Issue the show test <module#> command for module diagnostic results. If the port is in the inactive state, issue the show vlan command in order make sure that the VLAN of the port still exists and issue the set port enable <module#/port> in order to try to re-enable the port. VTP problems can sometimes cause a VLAN to be deleted, which results in ports associated with that VLAN to become inactive.

    • Vlan—This field displays trunk if it is a trunk port, or the VLAN number the port is a member of if it is an access port.

    • speed and duplex—These fields have an a in front of the value displayed, for example, a-full, if the value was obtained through auto-negotiation. If the port is hardcoded for speed and duplex the a is not present. While not in a connected state, an auto-negotiation-enabled port displays auto in these fields. Make sure that the device attached to this port has the same settings as the port in regards to either hard setting the speed and duplex or auto-negotiating the speed and duplex.

    • If port security is enabled, make sure the appropriate MAC addresses are allowed to pass through the port, and that the port is not shut down due to a security violation.

    • If broadcast suppression is enabled, check the number of dropped packets in order to make sure this is not the cause of traffic problems on the port.

    • If flow control is enabled, ensure that the other side of the link supports flow control as well, and make sure that the settings match on both ends.

    • If the port is configured as part of an EtherChannel, its state and the state of the other ports in the channel are displayed. Information on the neighbor device appears based on information obtained through CDP, if you assume that the CDP is enabled on both devices in the channel.

    • FCS-Err—The number of valid size frames with Frame Check Sequence (FCS) errors but no framing errors. This is typically a physical issue, for example, cabling, a bad port, or a bad Network Interface Card (NIC), but can also indicate a duplex mismatch.

    • Align-Err—This is the number of frames with alignment errors, which are frames that do not end with an even number of octets and have a bad Cyclic Redundancy Check (CRC), received on the port. These usually indicate a physical problem, for example, cabling, a bad port, or a bad NIC, but can also indicate a duplex mismatch. When the cable is first connected to the port, some of these errors can occur. Also, if there is a hub connected to the port, collisions between other devices on the hub can cause these errors.

    • Xmit-Err and Rcv-Err —This indicates that the internal port transmit (Tx) and receive (Rx) buffers are full. A common cause of Xmit-Err is traffic from a high bandwidth link that is switched to a lower bandwidth link, or traffic from multiple inbound links that is switched to a single outbound link. For example, if a large amount of bursty traffic comes in on a gigabit port and is switched out to a 100 Mbps port, this can cause the Xmit-Err field to increment on the 100 Mbps port. This is because that output buffer of the port is overwhelmed by the excess traffic due to the speed mismatch between the incoming and outgoing bandwidths.

    • Late-coll (late collisions)—The number of times that a collision is detected on a particular port late in the transmission process. For a 10 Mbit/s port, this is later than 512 bit-times into the transmission of a packet. Five hundred and twelve bit-times corresponds to 51.2 microseconds on a 10 Mbit/s system. This error can indicate a duplex mismatch among other things. For the duplex mismatch scenario, the late collision is seen on the half duplex side. As the half duplex side transmits, the full duplex side does not wait its turn and transmits simultaneously, which causes a late collision. Late collisions can also indicate an Ethernet cable or segment that is too long. Collisions should not be seen on ports configured as full duplex.

    • Single-coll (single collision)—The number of times one collision occurs before the port transmits a frame to the media successfully. Collisions are normal for ports configured as half duplex, but should not be seen on full duplex ports. If collisions are increasing dramatically, this points to a highly utilized link or possibly a duplex mismatch with the attached device.

    • Multi-coll (multiple collision)—This is the number of times multiple collisions occur before the port transmits a frame to the media successfully. Collisions are normal for ports configured as half duplex, but should not be seen on full duplex ports. If collisions increase dramatically, this points to a highly utilized link or possibly a duplex mismatch with the attached device.

    • Excess-coll (excessive collisions)—This is a count of frames for which transmission on a particular port fails due to excessive collisions. An excessive collision happens when a packet has a collision 16 times in a row. The packet is then dropped. Excessive collisions are typically an indication that the load on the segment needs to be split across multiple segments, but can also point to a duplex mismatch with the attached device. Collisions should not be seen on ports configured as full duplex.

    • Carri-Sen (carrier sense)—This occurs every time an Ethernet controller wants to send data on a half duplex connection. The controller senses the wire and checks if it is not busy before transmitting. This is normal on a half duplex Ethernet segment.

    • Undersize—The frames received that are smaller than the minimum IEEE 802.3 frame size of 64 bytes long, which excludes framing bits, but includes FCS octets, that are otherwise well formed, so it has a valid CRC. Check the device that sends out these frames.

    • Runts—The frames received that are smaller than the minimum IEEE 802.3 frame size (64 bytes for Ethernet), and with a bad CRC. This can be caused by a duplex mismatch and physical problems, such as a bad cable, port, or NIC on the attached device.

    • Giants—These are frames that exceed the maximum IEEE 802.3 frame size (1518 bytes for non-jumbo Ethernet), and have a bad FCS. Try to find the offending device and remove it from the network. In many cases it is the result of a bad NIC.

    • Issue the clear counters [all | mod/port] command in order to reset the statistics for the show port, show Mac, and show counters commands.

      Refer to the Catalyst 6500 Series Command Reference, 7.5 for more information and further explanation of the various fields in the show port command output.

  9. Check that traffic counters are incrementing both inbound and outbound on the port. You can also issue the show Mac<module#> command in order to look at the MAC info for all ports for a given module. 

    esc-6509-c (enable) show Mac 6/1
 
Port     Rcv-Unicast          Rcv-Multicast        Rcv-Broadcast
-------- -------------------- -------------------- --------------------
 6/1                    20890               894039                74883
 
Port     Xmit-Unicast         Xmit-Multicast       Xmit-Broadcast
-------- -------------------- -------------------- --------------------
 6/1                    12845                73660                  179
 
Port     Rcv-Octet            Xmit-Octet
-------- -------------------- --------------------
 6/1                 79498714              8738501
 
MAC      Dely-Exced MTU-Exced  In-Discard Out-Discard
-------- ---------- ---------- ---------- -----------
 6/1              0          0          0           0
 
Port  Last-Time-Cleared
----- --------------------------
 6/1  Sun Oct 13 2002, 16:37:58
esc-6509-c (enable)

    • The previous output shows the total unicast, multicast, and broadcast packets received (Rcv) and transmitted (Xmit) on a port.

      Note: If the port is an Inter-Switch Link Protocol (ISL) trunk, all traffic are multicast, for example, all ISL headers use the destination multicast address 01-00-0C-CC-CC-CC.

    • Dely-Exced —The number of frames discarded by this port due to an excessive transmit delay through the switch. This counter should never go up unless the port is under very high utilization.

    • MTU Exceed—This is an indication that one of the devices on that port or segment is transmitting more than the allowed frame size (1518 bytes for non-jumbo Ethernet).

    • In-Discard—The result of incoming valid frames that were discarded because the frame did not need to be switched. This can be normal if a hub is connected to a port and two devices on that hub exchange data. The switch port still sees the data but does not have to switch it, since the CAM table shows the MAC address of both devices associated with the same port, and so it is discarded. This counter can also increment on a port configured as a trunk if that trunk blocks for some VLANs, or on a port that is the only member of a VLAN.

    • Out-Discard—The number of outbound packets chosen to be discarded even though no packet errors are detected. One possible reason to discard such a packet can be to free up buffer space.

    • Issue the clear counters [all | mod/port] command in order to reset the statistics for the show port, show Mac, and show counters commands.

      Refer to the Catalyst 6500 Series Command Reference, 7.5 for more information and further explanation of the various fields in the show Mac command output.

  10. Check the detailed statistics for a specific port. 

    esc-6509-c (enable) show counters 6/1
64 bit counters
0  rxHCTotalPkts                      =               364517
1  txHCTotalPkts                      =                35104
2  rxHCUnicastPkts                    =                10281
3  txHCUnicastPkts                    =                 6678
4  rxHCMulticastPkts                  =               338957
5  txHCMulticastPkts                  =                28343
6  rxHCBroadcastPkts                  =                15279
7  txHCBroadcastPkts                  =                   83
8  rxHCOctets                         =             29291862
9  txHCOctets                         =              3460655
10 rxTxHCPkts64Octets                 =               181165
11 rxTxHCPkts65to127Octets            =               201314
12 rxTxHCPkts128to255Octets           =                 5546
13 rxTxHCPkts256to511Octets           =                11425
14 rxTxHCpkts512to1023Octets          =                   81
15 rxTxHCpkts1024to1518Octets         =                   89
16 txHCTrunkFrames                    =                    0
17 rxHCTrunkFrames                    =                    0
18 rxHCDropEvents                     =                    0
32 bit counters
0  rxCRCAlignErrors                   =          0
1  rxUndersizedPkts                   =          0
2  rxOversizedPkts                    =          0
3  rxFragmentPkts                     =          0
4  rxJabbers                          =          0
5  txCollisions                       =          0
6  ifInErrors                         =          0
7  ifOutErrors                        =          0
8  ifInDiscards                       =          0
9  ifInUnknownProtos                  =          0
10 ifOutDiscards                      =          0
11 txDelayExceededDiscards            =          0
12 txCRC                              =          0
13 linkChange                         =          4
14 wrongEncapFrames                   =          0
0  dot3StatsAlignmentErrors           =          0
1  dot3StatsFCSErrors                 =          0
2  dot3StatsSingleColFrames           =          0
3  dot3StatsMultiColFrames            =          0
4  dot3StatsSQETestErrors             =          0
5  dot3StatsDeferredTransmisions      =          0
6  dot3StatsLateCollisions            =          0
7  dot3StatsExcessiveCollisions       =          0
8  dot3StatsInternalMacTransmitErrors =          0
9  dot3StatsCarrierSenseErrors        =          0
10 dot3StatsFrameTooLongs             =          0
11 dot3StatsInternalMacReceiveErrors  =          0
0  txPause                            =          0
1  rxPause                            =          0
0  rxTotalDrops                       =          0
1  rxFIFOFull                         =          0
2  rxBadCode                          =          0
Last-Time-Cleared
--------------------------
Sun Oct 20 2002, 16:23:06
esc-6509-c (enable)

    This is a list of some of the non-generic counter details from the previous output:

    • RxFragmentPkts—The total number of packets received that do not end with an even number of octets (alignment error) or that has an FCS error, and is less than 64 octets in length, which excludes framing bits, but includes FCS octets.

    • dot3StatsInternalMacReceiveErrors—A count of frames for which reception on a particular port fails due to an internal MAC sublayer receive error. A frame is only counted if it is not counted by the corresponding instance of either dot3StatsFrameTooLongs, dot3StatsAlignmentErrors, or dot3StatsFCSErrors. In particular, an instance of this object can represent a count of receive errors on a particular port that are not otherwise counted.

    • dot3StatsInternalMacTransmitErrors—A count of frames for which transmission on a particular port fails due to an internal MAC sublayer transmit error. A frame is only counted if it is not counted by the corresponding instance of either dot3StatsLateCollisions, dot3StatsExcessiveCollisions, or dot3StatsCarrierSenseErrors.

    • RxJabbers—The total number of packets received that are longer than 1518 octets, which excludes framing bits, but incudes FCS octets, and do not end with an even number of octets (alignment error), or had an FCS error. The recommended action is to isolate the device that sends out these packets.

    • txDelayExceededDiscards—The number of frames discarded by this port due to an excessive transmit delay through the switch. This counter is the same as the Dely-Exced counter in the output from the show Mac command, and should never go up unless the port is under very high utilization.

    • IfInUnknownProtos—The number of inbound packets with unknown protocols.

    • TxCRC—This increments when frames are transmitted with a bad CRC, but it does not include frames aborted due to a late collision. This counter typically increments on an egress port when a frame is transmitted that is received as an ISL frame on an ingress port, but that carries an Ethernet packet with a bad CRC inside it, while the ISL packet itself has a good CRC. It can also be caused by bad switch hardware. A way to troubleshoot this is to send broadcast traffic on a port and see if the counter increments on all egress connected ports. If this happens independent of the port where you send traffic into, there is a failure in the switch hardware, most probably the chassis or supervisory module. If the counter is incrementing only when a certain module is used to send traffic into, this module has a hardware failure. If the counter is only incrementing on a few ports, the ports themselves have a problem. If the cause cannot be determined by the previous test, check the neighbor switches that are ISL connected, or check ISL connected end-devices. Contact Cisco Technical Support if you need further assistance.

    • dot3StatsSQETestErrors—A count of times that the SQE TEST ERROR message is generated by the physical signaling sublayer (PLS) for a particular interface. The SQE TEST ERROR message is defined in section 7.2.2.2.4 of the American National Standards Institute (ANSI)/IEEE 802.3-1985 and its generation is described in section 7.2.4.6 of the same document. This counter should never go up, since it is only of relevance to external Ethernet transceivers.

    • dot3StatsCarrierSenseErrors—The number of times that the carrier sense condition is lost or never asserted when you attempt to transmit a frame on a particular port. The count represented by an instance of this object is incremented at most once per transmission attempt, even if the carrier sense condition fluctuates during a transmission attempt. This counter is the same counter as the Carri-Sen field in the output of the show port command. This is normal on an half duplex Ethernet segment.

    • linkChange—The number of times the port toggles between a connected state to a non-connected state. If this counter is incrementing constantly it means there is something wrong with this port, the cable attached to this port, or the device at the other end of the cable.

    • dot3StatsFrameTooLongs—This is the count of frames received on a particular interface that exceeds the maximum permitted frame size. Check the device attached to the port.

    • dot3StatsFCSErrors—A count of valid frames received on a particular interface that end with an even number of octets but do not pass the FCS check. This is typically a physical issue, for example, cabling, bad port, or bad NIC card, but can also indicate a duplex mismatch. This is the same counter as the FCS-Err field in the output from the show port command.

    • dot3StatsSingleColFrames—A count of successfully transmitted frames on a particular port for which transmission is initially inhibited by exactly one collision. Collisions are normal for ports configured as half duplex, but should not be seen on full duplex ports. If collisions increase dramatically this points to a highly utilized link, or possibly a duplex mismatch with the attached device. This is the same counter as the Single-Coll field in the output from the show port command.

    • dot3StatsMultiColFrames—A count of successfully transmitted frames on a particular port for which transmission is initially inhibited by more than one collision. Collisions are normal for ports configured as half duplex, but should not be seen on full duplex ports. If collisions increase dramatically this points to a highly utilized link or possibly a duplex mismatch with the attached device. This is the same counter as the Multi-Coll field in the output from the show port command.

    • dot3StatsExcessiveCollisions—A count of frames for which transmission on a particular port fails due to excessive collisions. An excessive collision happens when a packet has a collision 16 times in a row. The packet is then dropped. Excessive collisions are typically an indication that the load on the segment needs to be split across multiple segments, but can also point to a duplex mismatch with the attached device. Collisions should not be seen on ports configured as full duplex. This is the same counter as the Excess-Coll field in the output from the show port command.

    • dot3StatsLateCollisions—The number of times that a collision is detected on a particular port late in the transmission process. For a 10 Mbit/sec port this is later than 512 bit-times into the transmission of a packet. 512 bit-times corresponds to 51.2 microseconds on a 10 Mbit/s system. A late collision is also considered a generic collision for purposes of other collision-related statistics. This counter is the same as the Late-Coll field in the output from the show port command, and can indicate a duplex mismatch among other things. For the duplex mismatch scenario, the late collision is seen on the half duplex side. As the half duplex side transmits, the full duplex side does not wait its turn and transmits simultaneously that causes a late collision. Late collisions can also indicate an Ethernet cable or segment that is too long. Collisions should not be seen on ports configured as full duplex.

    • dot3StatsDeferredTx—A count of frames for which the first transmission attempt on a particular port is delayed because the medium is busy. This count does not include frames involved in collisions. Deferred transmissions are normal in Ethernet, however, a high count can indicate a highly loaded segment.

    • rxBadCode—This is a count of received frames for which the preamble has a bad code. Check the device connected to the port.

    • IfInDiscards—This is a count of valid frames received, which are discarded by the forwarding process of the switch. This is the same counter as the In-Discard field in the output from the show Mac command. You see this when you receive traffic on a trunk for a specific VLAN while the switch does not have any other ports on that VLAN. You also see this counter increments when the destination address of the packet is learned on the port the packet is received on, or when a port is configured as a trunk and that trunk is blocking for VLANs.

    • rxUndersizedPkts—The total number of packets received that are less than 64 octets long, which excludes framing bits, but includes FCS octets, and are otherwise well formed. This counter is the same as the Undersize field in the output from the show port command. Check the device that sends out these frames.

    • RxOversizePkts—The total number of packets received that are longer than 1518 octets, which excludes framing bits, but includes FCS octets, and are otherwise well formed. Check the device connected to this port. This counter can increment when the device attached to the port has ISL encapsulation enabled, and the port itself does not. This counter also increments if you receive jumbo frames without the configuration of jumbo support on the port.

    • dot3StatsAlignmentErrors—The total number of packets received that have a length, which excludes framing bits, but includes FCS octets, of between 64 and 1518 octets, inclusive, but do not end with an even number of octets and have a bad FCS. This is the same counter as the Align-Err field in the output from the show port command. These errors usually indicate a physical problem , for example, bad port, or bad NIC card, but can also indicate a duplex mismatch. When the cable is first connected to the port, some of these errors can occur. Also, if there is a hub connected to the port, collisions between other devices on the hub can cause these errors.

    • rxTotalDrops—This counter includes a sum of these counters:

      • The number of bad packets because of a CRC error

      • A coding violation or sequence error.

      • The number of Color Blocking Logic (CBL) blocking drops

      • The number of instances of invalid encapsulation

      • The number of broadcast suppression drops

      • The number of drops because the packet length is less than 64 or greater than 1518 bytes

      CBL refers to the spanning-tree state of a particular VLAN (color) on the port in question. If the port is in a spanning-tree blocking state for a particular VLAN, it is normal to drop packets received on that port for that VLAN.

  11. Check for incrementing errors. Also, issue the show logging buffer 1023 command, as shown in step 3, which syslogs any of these errors that occurs on a port. Some errors cause the module to be reset by firmware in order to recover. This command was introduced in CatOS release 5.5(12), 6.3(4), and 7.x. 

    esc-6509-c (enable) show intcounters 6/1
MasterInt       : 0
PbUnderflow     : 0
Parity          : 0
InternalParity  : 0
PacketCRC       : 0
MdtifErr        : 0
CpuifErr        : 0
PnclChksum      : 0

    Issue the show log command in order to get the history of the module resets.

    esc-6509-c (enable) show log 6
 
Module 6 Log:
  Reset Count:   73
  Reset History: Sun Oct 13 2002, 15:51:18
                 Sun Oct 13 2002, 08:44:51
                 Sat Oct 12 2002, 22:48:11
                 Fri Oct 11 2002, 23:47:30

  12. The output from the show spantree [vlan] or show spantree [mod/port] can be used to verify the port is spanning-tree forwarding or blocking state. If the port is in blocking state, it does not forward traffic on that link.

    esc-6509-c (enable) show spantree 175       
VLAN 175
Spanning tree mode          PVST+
Spanning tree type          ieee
Spanning tree enabled
 
Designated Root             00-30-94-93-e5-80
Designated Root Priority    1
Designated Root Cost        76
Designated Root Port        6/1                     
Root Max Age   20 sec   Hello Time 2  sec   Forward Delay 15 sec
 
Bridge ID MAC ADDR          00-d0-02-ea-1c-ae
Bridge ID Priority          32768
Bridge Max Age 20 sec   Hello Time 2  sec   Forward Delay 15 sec
 
Port                     Vlan Port-State    Cost      Prio Portfast Channel_id
------------------------ ---- ------------- --------- ---- -------- ----------
 3/1                     175  forwarding            4   32 disabled 0         
 6/1                     175  forwarding           19   32 disabled 0         
 6/2                     175  blocking            100   32 disabled 0         
16/1                     175  forwarding            4   32 enabled  0

  13. Issue the show test <module#> command in order to check the results of the online diagnostic test performed at switch boot time or when a module is reset. The results of these tests can be used to determine if a hardware component failure is detected on the module. It is important to set the diagnostic mode to complete, otherwise all or some of the diagnostic tests are skipped. If a hardware component failure has occurred between now and the last switch or module reset, the diagnostics must be run again through a switch or module reset in order to detect the failure.

    Complete these steps in order to run the diagnostic tests for a module:

    1. Set the diagnostic mode to complete. 

      esc-6509-c (enable) set test diag complete 
Diagnostic level set to complete.

    2. Reset the module. 

      esc-6509-c (enable) reset 6

This command will reset module 6 and may disconnect your telnet session.

Do you want to continue (y/n) [n]? y

    3. View the diagnostic test result for the ports on the module for any indication of a failure. Also check for failures in groups of 12 ports, which suggests a Coil ASIC failure or Pinnacle port failure. 

      esc-6509-c (enable) show test 6
 
Diagnostic mode: complete   (mode at next reset: complete)
 
Module 6 : 48-port 10/100BaseTX Ethernet
 
Line Card Status for Module 6 : PASS
 
Port Status :
  Ports 1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  -----------------------------------------------------------------------------
        .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .
        25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
       ------------------------------------------------------------------------
        .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .
 
Line Card Diag Status for Module 6  (. = Pass, F = Fail, N = N/A)
 
 Loopback Status [Reported by Module 2] :
  Ports 1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  -----------------------------------------------------------------------------
        .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  . 
  Ports 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
  -----------------------------------------------------------------------------
        .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  . 
 
 InlineRewrite Status :
  Ports 1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  -----------------------------------------------------------------------------
        .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  . 
  Ports 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
  -----------------------------------------------------------------------------
        .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  . 
 
esc-6509-c (enable)

Command Outputs to Collect Before You Contact Cisco Technical Support

The Output Interpreter Tool (registered customers only) (OIT) supports certain show commands. Use the OIT to view an analysis of show command output.

This list of commands was used in the previous troubleshooting of the WS-X6348 module connectivity issues in this document. Use these commands in order to log the troubleshooting output collected before you open a casein order to provide to the TAC engineer for analysis.

  • show module <module#>

  • show config <module#>

  • show logging buffer 1023

  • show cam dynamic <module#/port>

  • show trunk <module#/port>

  • show spantree <module#/port>

  • show cdp neighbor <module#/port> detail

    Repeat these three commands three times in orderto monitor counter increments, steps 8 through 10 only.

  • show port <module#/port>

  • show mac <module#/port>

  • show counters <module#/port>

  • show intcounters <module#/port> (Introduced in CatOS release 5.5(12), 6.3(4), and 7.x.)

    show log <module#>

  • set test diag complete

    reset <module#>

    show test <module#>

This is list of additional commands, which can be collected before you open a case with Cisco Technical Support for further troubleshooting by the TAC engineers or development engineers. These commands are hidden commands and should be used exactly as shown in order to troubleshoot the WS-X6348 module issues by the TAC engineers. You can alternatively provide these commands at the request of the TAC engineer who handles the case.

  • show asicreg <module#/port> pinnacle errcounters

  • show asicreg <module#/port> pinnacle pointers

  • show asicreg <module#/port> pinnacle all

  • show asicreg <module#/port> coil errcounters

  • show asicreg <module#/port> coil pointers

  • show asicreg <module#/port> coil 129

  • show asicreg <module#/port> coil all

  • show asicreg <module#/port> mii_phy all

    Note: This Command Line Interface (CLI) is currently does not work from CatOS release 6.3(8) and later. Refer to Cisco bug ID CSCdz26435 (registered customers only) for more information.

  • show ltl <module#/port>

  • show cbl <module#>

Related Information

Revision History

Revision Publish Date Comments
1.0
24-Apr-2007
Initial Release

Was this Document Helpful?

FeedbackFeedback

Contact Cisco

This Document Applies to These Products