Troubleshooting

This chapter includes the following topics:

Diagnosing Problems

The switch LEDs provide troubleshooting information about the switch. They show power-on self-test (POST) failures, port-connectivity problems, and overall switch performance. You can also get statistics from the CLI, or a Simple Network Management Protocol (SNMP) workstation. For details, refer to one of the following guides:

Switch POST Results

Approximately 30 seconds after the switch powers on, it begins the POST, which can take up to 5 minutes to complete. During POST, the SYSTEM LED blinks green. After POST is complete, the SYSTEM LED turns solid green. The ACTV LED is green if the switch is acting as the active switch.

caut.gif

Caution blank.gif POST failures are usually fatal. Conta ct your Cisco technical support representative if your switch does not pass POST.

Switch LEDs

Look at the port LEDs for information when troubleshooting the switch. See the descriptions of the LED colors and their meanings in the “Ethernet Management Port” section.

Switch Connections

The following are some of the scenarios relating to switch connections that might require troubleshooting.

Bad or Damaged Cable

Always examine a cable for marginal damage or failure. A cable might be just good enough to connect at the physical layer, but it could corrupt packets as a result of subtle damage to the wiring or connectors. You can identify this situation because the port has many packet errors or the port constantly flaps (loses and regains link).

  • Ensure that the cables are recommended by Cisco.
  • Look for broken or missing pins on cable connectors.
  • Rule out bad patch panel connections or media convertors, if any, between the source and destination. If possible, bypass the patch panel or eliminate media convertors (fiber-optic-to-copper).
  • Try the cable in another port to see if the problem is caused by the cable.
  • For a Catalyst 3650 switch StackWise cable, remove and inspect the cable and StackWise port for bent pins or damaged connectors. If the StackWise cable is bad, replace the cable.

Ethernet and Fiber Cables

Make sure that you have the correct cable:

  • For Ethernet, use Category 3 copper cable for 10 Mbps UTP connections. Use either Category 5, Category 5e, or Category 6 UTP for 10/100 or 10/100/1000 Mbps connections.
  • Verify that you have the correct fiber-optic cable for the distance (100 meters or less) and port type. Make sure that the connected device ports match and use the same type encoding, optical frequency, and fiber type. For more information, see the “Connector Specifications” section.
  • Determine if a copper crossover cable was used when a straight-through cable was required, or the reverse. Enable automatic medium-depending interface crossover (auto-MDIX) on the switch, or replace the cable. For more information, see the “Connector Specifications” section.

Link Status

Verify that both sides have a link. A broken wire or a shut-down port might cause one side to show a link even though the other side does not have a link.

A port LED that is on does not guarantee that the cable is functional. It might have encountered physical stress, causing it to function at a marginal level. If the port LED does not turn on:

  • Connect the cable from the switch to a known functional device.
  • Make sure that both ends of the cable are connected to the correct ports.
  • Verify that both the devices have power.
  • Verify that you are using the correct cable type. For more information, see the “Connector Specifications” section.
  • Look for loose connections. Sometimes a cable appears to be seated but is not. Disconnect the cable and then reconnect it.

10/100/1000 Port Connections

If a port appears to malfunction:

PoE and PoE+ Port Connections

If a powered device connected to a PoE port does not receive power:

  • Verify the status of all the ports. For more information, see the “Port LEDs and Modes” section.
  • Use the show interfaces privileged EXEC command to see if the port is error-disabled, disabled, or shut down. Re-enable the port, if necessary.
  • Verify that the power supply installed in the switch meets the power requirements of your connected devices.
  • Verify the cable type. Many legacy-powered devices, including older Cisco IP phones and access points that do not fully support IEEE 802.3af might not support PoE when connected to the switch by a crossover cable. Replace the crossover cable with a straight-through cable.
caut.gif

Caution blank.gif Noncompliant cabling or powered devices might cause a PoE port fault. Use only compliant cabling to connect Cisco prestandard IP phones, wireless access points, or IEEE 802.3af-compliant devices.

note.gif

Noteblank.gif Ensure that the output of the PoE circuit has been evaluated as a limited power source (LPS) per IEC 60950.


SFP Modules

If SFP modules do not function correctly:

  • Use only Cisco SFP modules.
  • Inspect the uplink port and SFP module. If the module is faulty, replace the module.
  • Verify that the module is supported on Cisco Catalyst 3650 switches. (The switch’s release note on Cisco.com lists the SFP and SFP+ modules that the switch supports.) See the Cisco Catalyst 3650 Series Switches Release Notes at http://www.cisco.com/c/en/us/support/switches/catalyst-3650-series-switches/products-release-notes-list.html.
  • Use the show interfaces privileged EXEC command to see if the port or module is error-disabled, disabled, or shut down. Re-enable the port, if needed.
  • Make sure that all the fiber connections are clean and securely connected.
  • For CX1 module connections and fiber connections, make sure that cable routing does not violate the minimum allowed cable bend radius. See the documentation pertaining to the corresponding module for specific cabling requirements.
note.gif

Noteblank.gif When ordering or using CX1 cables, ensure that the version identifier is 2 or higher.


  • For long-wave SFP+ modules, a mode conditioning patch might improve performance over maximum link distances with MMF connections.
  • When you insert several SFPs in multiple switch ports, wait for 5 seconds between inserting each SFP. This will prevent the ports from going into error disabled mode. Similarly, when you remove an SFP from a port, wait for 5 seconds before reinserting it.

Interface Settings

Verify that the port or interface is not disabled or powered off. If a port or interface is manually shut down on either side of the link, it does not come up until you re-enable the interface. Use the show interfaces privileged EXEC command to see if the interface is error-disabled, disabled, or shut down on either side of the connection. If needed, re-enable the interface.

Ping End Device

Ping from the directly connected switch first, and then work your way back port by port, interface by interface, trunk by trunk, until you find the source of the connectivity issue. Make sure that each switch can identify the end device’s MAC address in its content-addressable memory (CAM) table.

Spanning Tree Protocol Loops

Spanning Tree Protocol (STP) loops might cause serious performance issues that resemble port or interface problems.

A unidirectional link might cause loops. A loop occurs when the traffic sent by a switch is received by the neighbor, but the traffic from the neighbor is not received by the switch. A broken cable, other cabling problems, or a port issue could cause this one-way communication.

You can enable UniDirectional Link Detection (UDLD) on the switch to help identify unidirectional link problems. For information about enabling UDLD on the switch, see the “Understanding UDLD” section in the Cisco Catalyst 3650 Series Switches Software Configuration Guides at http://www.cisco.com/c/en/us/support/switches/catalyst-3650-series-switches/products-installation-and-configuration-guides-list.html.

Switch Performance

The following are some of the scenarios relating to switch performance that might require troubleshooting.

Speed, Duplex, and Autonegotiation

Port statistics that show a large amount of alignment errors, frame check sequences (FCS), or late-collisions errors, might indicate a speed or duplex mismatch.

A common issue occurs when duplex and speed settings are mismatched between two switches, between a switch and a router, or between a switch and a workstation or server. Mismatches might occur when manually setting the speed and duplex, or from autonegotiation issues between the two devices.

To maximize switch performance and to ensure a link, follow one of these guidelines when changing the duplex or speed settings:

  • Let both the ports autonegotiate both the speed and the duplex.
  • Manually set the speed and duplex parameters for the interfaces on both ends of the connection.
  • If a remote device does not autonegotiate, use the same duplex settings on the two ports. The speed parameter adjusts itself even if the connected port does not autonegotiate.

Autonegotiation and Network Interface Cards

Sometimes, problems occur between the switch and third-party network interface cards (NICs). By default, the switch ports and interfaces autonegotiate. Laptops or other devices are commonly set to autonegotiate; yet, sometimes, issues occur.

To troubleshoot autonegotiation problems, try setting both sides of the connection manually. If this does not solve the problem, there could be a problem with the firmware or software on the NIC. You can resolve this by upgrading the NIC driver to the latest version.

Cabling Distance

If the port statistics show excessive frame check sequences, late-collisions or alignment errors, verify that the cable distance from the switch to the connected device meets the recommended guidelines..

Resetting the Switch

If you have configured a new switch with a wrong IP address, or if all of the switch LEDs start blinking when you try to enter Express Setup mode, you can clear the IP address that is configured on the switch.

note.gif

Noteblank.gif Resetting the switch reboots the switch.


To reset the switch


Step 1blank.gif Press and hold the Mode button.

Step 2blank.gif Start the terminal emulation program on the PC or the terminal. The program, frequently a PC application, such as HyperTerminal or ProcommPlus, makes communication between the switch and your PC or terminal possible.

The switch LEDs begin blinking after about 2 seconds. If the LEDs above the mode button turn solid green, you can release the Mode button and run Express Setup to configure the switch. If the LEDs do not turn solid green, continue with the next step.

Step 3blank.gif Continue holding down the Mode button. The LEDs stop blinking after an additional 8 seconds, and then the switch reboots


 

You can also configure the switch by using the CLI setup procedure.

Finding the Switch’s Serial Number

If you contact Cisco Technical Assistance, you should know the switch’s serial number. Figure 5-1 shows the serial number location. You can also use the show version privileged EXEC command to see the switch’s serial number.

Figure 5-1 Switch Serial Number Location

 

347807SN.eps

 

Replacing a Failed Data Stack Member

To replace a failed data stack member:


Step 1blank.gif Power down the failed switch. Remove the AC or DC input power.

Step 2blank.gif Make sure the replacement switch is powered off, and then connect it to the stack.

If you had manually set the member numbers for the switch stack, manually assign the member number of the failed switch to the replacement switch. To manually assign the stack member number, see the Cisco Catalyst 3650 Series Switches Software Configuration Guides at http://www.cisco.com/c/en/us/support/switches/catalyst-3650-series-switches/products-installation-and-configuration-guides-list.html.

Step 3blank.gif Make the same Gigabit Ethernet connections on the replacement switch as those that were on the failed switch.

Step 4blank.gif Reinstall modules and cable connections, if any.

Step 5blank.gif Power on the replacement switch


 

The replacement switch will have the same configuration for all the interfaces as the failed switch, and will function the same way as the failed switch.

note.gif

Noteblank.gif The replacement switch must be a Cisco Catalyst 3650 switch.