Troubleshooting
This chapter describes how to troubleshoot Cisco Nexus 7000 Series hardware problems.
This chapter includes the following sections:
If your system has problems related to the Cisco NX-OS operating system, see the Cisco NX-OS Troubleshooting Guide. If you cannot isolate a system problem, contact the Cisco Technical Assistance Center (TAC).
Getting Started
Start the Cisco Nexus 7000 Series switch by turning on the power at the power supply units to send power to the system fans, supervisor module, fabric modules, and I/O modules. During the startup phase, the STATUS LEDs on the supervisor and I/O modules are amber to indicate that the initialization process is in progress. When the system is operational, the component LEDs indicate that either the system is operational or you need to troubleshoot for system problems.
The key to troubleshooting the system is to isolate a problem to a specific system component. First, you compare what the system is doing with what it should be doing. Look at the chassis LEDs to see if there are any critical problems detected with the system components. Green chassis LEDs indicate that all components of a type are operational while amber LEDs indicate that at least one component of a type has a critical problem that requires troubleshooting.
For a listing of the system LEDs and the conditions that they indicate, see Table D-1.
Less critical problems, such as minor overtemperature conditions, are not indicated by the chassis LEDs. To find those problems, you must look for amber STATUS LEDs on the system modules.
Depending on the type of component that needs troubleshooting, see one of the following topics:
Troubleshooting the Fan Trays
The following conditions indicate that there is a problem with one or more fan tray modules:
- The FAN LED on the chassis is amber.
- The STATUS LED on the fan tray is flashing red.
While the system is running, each fan tray module should be exhausting air and its STATUS LED should be green, which indicates that all of the fans in that tray are running above the threshold speed.
For a listing of the fan tray LEDs and the conditions that they indicate, see Table D-7.
To troubleshoot a fan tray problem, follow these steps until the problem is resolved:
Step 1 Check if the STATUS LED is flashing red or is not lit on each fan tray. If the LED is flashing red, at least one fan in the tray is running at a slow speed or not running. If the LED is not lit, no power is going to the fan tray.
Step 2 Check if the power supply units are providing power to the chassis components. The Output LED on each power supply unit should be lit and green.
Step 3 Make sure that the fan tray is properly seated in the chassis by following these steps:
a. Loosen the captive screws on the fan tray until they are no longer in the chassis.
b. Place one hand on the handle for the fan tray and pull the fan tray part way out of the chassis.
c. Push the fan tray back into the chassis until it is fully seated on the midplane and make sure that each captive screw on the fan tray is aligned with its hole in the chassis.
d. Securely tighten each captive screw (to 8 in-lb [0.90 N·m]) to the chassis.
Step 4 Call Cisco TAC and describe the conditions that you are seeing with the fan tray. See the “Contacting Customer Service” section.
Troubleshooting an AC Power Supply Unit
The following conditions indicate that you need to troubleshoot one or more AC power supply units:
- The PSU LED on the chassis is amber.
- The Input 1, Input 2, or Output LED on a power supply unit is not green.
- The Fault LED on a power supply unit is flashing red.
Also, when there are power problems with other modules or the supervisor module PWR MGMT LED is amber, you need to troubleshoot the power supply units or check the seating of the modules that are not receiving power.
For a listing of the power supply unit LEDs and the conditions that they indicate, see Table D-6.
To troubleshoot a power supply problem, follow these steps:
Step 1 Determine which power supply unit has a problem. Check to see which power supply unit does not have its Input and Output LEDs lit.
Step 2 If one of the required Input LEDs is not lit, perform each of the following checks on the power cord used for that half of the power supply unit:
- Make sure that the power cord is securely connected to the AC source.
- For a 6-kW power supply unit, make sure that the power cord is securely connected to the power supply unit. Verify that the cable retention device on the power supply unit is securely holding the power cord.
- Turn the power supply switch to standby (STBY or 0) and then disconnect the power cord from the AC source and reconnect with another AC source. After you connect to the AC source, turn the power switch on (ON or 1).
Step 3 If the Output LED is not lit, verify that the power switch is turned on (ON or 1).
Step 4 Make sure that the power supply unit is properly seated in the power supply bay by performing the following steps:
a. Turn the power switch to standby (STBY or 0).
b. Loosen the four captive screws on the power supply unit so that they are no longer in contact with the chassis.
c. Pull the power supply unit part way out of the chassis and then push it back into the power supply bay. Make sure that the captive screws on the power supply unit align with their holes in the chassis.
d. Tighten the four captive screws to 8 in-lb (0.90 N·m).
e. Turn the power switch on (ON or 1).
Step 5 Verify that the power supply unit is functional by replacing it with another power supply (see the “Replacing an AC Power Supply Unit During Operations” section) and plugging its power cords into the same AC source. If the Input and Output LEDs turn green on the replacement power supply unit, contact Cisco TAC (see the “Contacting Customer Service” section) to replace the defective power supply unit.
If you cannot determine the problem with the power supply or determine which power supply unit has the problem, contact TAC.
Troubleshooting a DC Power Supply Unit
The following conditions indicate that you need to troubleshoot one or more DC power supply units:
- The PSU LED on the chassis is amber.
- The Input LEDs or Output LED on a power supply unit is not green.
- The Fault LED on a power supply unit is flashing red.
Also, when there are power problems with other modules or the supervisor module PWR MGMT LED is amber, you need to troubleshoot the power supply units or check the seating of the modules that are not receiving power.
For a listing of the power supply unit LEDs and the conditions that they indicate, see Table D-6.
To troubleshoot a power supply problem, follow these steps:
Step 1 Determine which power supply unit has a problem. Check to see which power supply unit does not have its Input and Output LEDs lit.
Note If a power supply unit has only one power cord attached to a DC source, only the Input LEDs (Input 1 and Input 2 or Input 3 and Input 4) for the connected half of the power supply unit should be lit.
Step 2 If the Output LED is not lit, verify that the power switch on the power supply unit is turned to ON. If not, turn it to ON and verify that the Output LED turns on.
Step 3 If the required Input LEDs are not lit, test the power connections as follows:
a. Turn the power switch on the power supply unit to standby (STBY or 0).
b. Turn off the DC circuit by manually shutting it down at the circuit breaker.
Warning Before performing any of the following procedures, ensure that power is removed from the DC circuit. Statement 1003
c. Verify that the power cord is securely connected to the DC source or the DC power interface unit (PIU).
Warning Hazardous voltage or energy may be present on DC power terminals. Always replace cover when terminals are not in service. Be sure uninsulated conductors are not accessible when cover is in place. Statement 1075
d. Verify that the power cord is securely connected to the power supply unit. Verify that the cable retention device on the power supply unit is securely holding the power cord.
e. Turn on the power at the circuit breaker.
f. Turn the power switch on the power supply unit to ON (or 1).
g. Check the Input LEDs to see if they are on yet.
Step 4 If the required Input LEDs are still not lit, test the power source as follows:
a. Turn the power switch on the power supply unit to standby (STBY or 0).
b. Turn off the DC circuit by manually shutting it down at the circuit breaker.
Warning Before performing any of the following procedures, ensure that power is removed from the DC circuit. Statement 1003
c. Disconnect the power cord from the DC source, and reconnect with another powered down DC source.
Warning Hazardous voltage or energy may be present on DC power terminals. Always replace cover when terminals are not in service. Be sure uninsulated conductors are not accessible when cover is in place. Statement 1075
d. Turn on the newly connected DC power source at the circuit breaker.
e. Turn the power switch on the power supply unit to ON (or 1).
f. Check the Input LEDs to see if they are on yet.
g. If the Input LEDs are not on, call TAC.
Step 5 If the Output LED is not lit, verify that the power switch on the power supply unit is turned to ON (or 1). If not, turn it to ON (or 1) and see if the LED turns on.
Step 6 If the Output LED is not lit, verify that the power supply unit is properly seated in the power supply bay by performing the following steps:
a. Turn the power switch on the power supply unit to standby (STBY or 0).
b. Turn off the DC circuit by manually shutting it down at the circuit breaker.
Warning Before performing any of the following procedures, ensure that power is removed from the DC circuit. Statement 1003
c. Pull the power supply unit part way out of the chassis and then push it back into the power supply bay. To remove a 6-kW power supply, you must first fully loosen its four captive screws. To remove a 3-kW, you must press and hold its release latch to the left. When you reinstall a 6-kW power supply, tighten its four captive screws to 8 in-lb (0.90 N·m).
d. Turn on the DC power source at the circuit breaker.
e. Turn the power switch to ON (or 1).
f. Verify that the Output LED turns on.
Step 7 If the Output LED is still not on, verify that the power supply unit is functional by replacing it with another power supply (see the “Replacing a DC Power Supply Unit During Operations” section) and plugging its power cords into the same DC source. If the Input and Output LEDs turn green on the replacement power supply unit, contact TAC (see the “Contacting Customer Service” section) to replace the defective power supply unit.
If you cannot determine the problem with the power supply or determine which power supply unit has the problem, contact TAC.
Troubleshooting an HVAC/HVDC Power Supply Unit
The following conditions indicate that you need to troubleshoot an HVAC/HVDC power supply unit:
- The PSU LED on the chassis is amber.
- The Input or Output LED on the power supply unit is not green.
- The Fault LED on the power supply unit is flashing red.
Also, when there are power problems with other modules or the supervisor module PWR MGMT LED is amber, you need to troubleshoot the power supply unit or check the seating of the modules that are not receiving power.
For a listing of the power supply unit LEDs and the conditions that they indicate, see Table D-6.
To troubleshoot a power supply problem, follow these steps:
Step 1 Check to see if the Input and Output LEDs are lit.
Step 2 If the Input LED is not lit, perform each of the following checks on the power cord used for the power supply unit:
- Make sure that the power cord is securely connected to the power source.
- Turn the power supply switch to standby to 0 and then disconnect the power cord from the power source and reconnect with another power source. After you connect to the power source, turn the power switch on (1).
Step 3 If the Output LED is not lit, verify that the power switch is turned on (1).
Step 4 Make sure that the power supply unit is properly seated in the power supply bay.
Step 5 Verify that the power supply unit is functional by replacing it with another power supply and plugging its power cords into the same power source.
If you cannot determine the problem with the power supply or determine which power supply unit has the problem, contact TAC.
Troubleshooting the Supervisor Modules
The following conditions indicate a problem with one or more supervisor modules:
- The SUP LED on the chassis is amber.
- The STATUS LED on a supervisor module is amber, red, or not lit.
- The CMP STATUS LED on a supervisor 1 module is amber or red.
- An ACT LED on a supervisor module is not lit.
The SYSTEM and PWR MGMT LEDs on the supervisor modules indicate that there could be problems with the supervisor module or any of the other switch modules.
When you start up the switch, the supervisor module STATUS and CMP STATUS LEDs are amber while the module runs diagnostic tests. When the module passes the diagnostic tests and becomes operational, the STATUS and CMP STATUS LEDs become green. For switches that have two supervisor modules, the ACTIVE LED is green for the active supervisor module or amber for the standby supervisor module. When the SYSTEM and PWR MGMT LEDs are green, the switch does not detect any critical system problems. Amber, red, or unlit LEDs indicate system problems that you need to resolve, possibly with other system modules.
For a listing of supervisor module LEDs and the conditions that they indicate, see Table D-2.
To troubleshoot for a hardware problem with the supervisor module, follow these steps until the problem is resolved:
Step 1 Check if the STATUS LED (all supervisors modules) or CMP STATUS LED (Supervisor 1 modules only) is amber, red, or unlit on each supervisor module as follows:
- If either the STATUS or CMP STATUS LED is amber or flashing red, a minor (amber) or critical (flashing red) overtemperature condition exists. Contact TAC for assistance (see the “Contacting Customer Service” section).
- If the STATUS LED is red, the initialization process detected a parity error condition. Contact TAC for assistance (see the “Contacting Customer Service” section).
Step 2 If the STATUS LED (all supervisors modules) and CMP STATUS LED (Supervisor 1 modules only) are unlit, check the module seating and power connections as follows:
a. Loosen the captive screws on the module so that they are no longer in contact with the chassis.
b. Unseat the module by pressing the ejector buttons on each end of the module and swinging out each ejector lever.
c. Place one hand on the handle for the module and pull the module part way out of the chassis.
d. Push the module back into the chassis until it is seated on the midplane.
e. Rotate both ejector levers until they both touch the front of the module. Make sure that each captive screw on the fan tray is aligned with its hole in the chassis.
f. Securely tighten each captive screw (to 8 in-lb [0.9 N·m]) to the chassis.
g. Verify that the Output LED on each power supply is lit. If the Output LED is not lit, troubleshoot the AC power supply units (see the “Troubleshooting an AC Power Supply Unit” section) or DC power supply units (see the “Troubleshooting a DC Power Supply Unit” section).
Step 3 If the LOG FLASH or EXPANSION FLASH (Supervisor 1) or SLOT0 (Supervisor 2 or 2E) LED is not lit, a compact flash slot (Supervisor 1) or USB port (Supervisor 2 or Supervisor 2E) is empty. If you need to load storage media in the slot or port, make sure that the media is formatted correctly for the slot.
Step 4 Contact TAC for assistance (see the “Contacting Customer Service” section).
Troubleshooting the Fabric Modules
The following conditions indicate problems with one or more fabric modules:
- The FAB LED on the chassis is amber.
- The STATUS LED on the fabric module is red.
Note Fabric modules are included with the Cisco Nexus 7009, 7010, and 7018 chassis but not with the Cisco Nexus 7004 chassis.
When you start up the switch or install a new fabric module, the STATUS LED on the module is amber while the module initializes. When the module becomes operational, the STATUS LED becomes green. If an overtemperature condition occurs, the STATUS LED flashes red.
For a listing of the fabric module LEDs and the conditions that they indicate, see Table D-5.
To troubleshoot a fabric module hardware problem, follow these steps until the problem is resolved:
Step 1 Check if a STATUS LED is flashing red or is unlit on each fabric module.
Step 2 Check if the power supply units are providing power to the chassis components.
Step 3 Make sure that the fabric module is properly seated in the chassis as follows:
a. Loosen the captive screws on the fabric module until they are no longer in the chassis.
b. Press the eject buttons on either end of the module and simultaneously rotate out both ejector levers until the module is unseated from the midplane.
c. Place one hand on the module handle and pull the module part way out of the chassis.
d. Push the module back into the chassis until it is seated on the midplane.
e. Simultaneously rotate both ejector levers until they both touch the front of the module. This action fully seats the module on the midplane.
f. Make sure that each of the captive screws on the module is aligned with its holes in the chassis.
g. Screw in each captive screw to the chassis and tighten to 8 in-lb (0.9 N·m).
Step 4 Contact TAC for assistance (see the “Contacting Customer Service” section).
Troubleshooting the I/O Modules
The following conditions indicate that there are problems with one or more I/O modules:
- The IOM LED on the chassis is amber.
- The STATUS LED on an I/O module is red.
During initialization, the STATUS LED is amber while the I/O module powers up and performs diagnostic tests. When the diagnostic tests are complete, the STATUS LED becomes green. If an overtemperature condition occurs, the STATUS LED becomes amber. If the module is resetting, ejectors are out, or if there is a major overtemperature condition, the LED flashes red.
For a listing of the I/O module LEDs and the conditions that they indicate, see Table D-3.
To troubleshoot an I/O module hardware problem, follow these steps until the problem is resolved:
Step 1 Determine which I/O module has a problem. Check if a STATUS LED that is flashing red or is not lit on a module.
Step 2 Check if the power supply units are providing power to the chassis components. See the “Troubleshooting the Fan Trays” section.
Step 3 Make sure that the I/O module is properly seated in the chassis by following these steps:
a. Loosen the captive screws on the I/O module until they are no longer in the chassis.
b. Press the eject buttons on either end of the module.
c. Simultaneously rotate out both ejector levers until the module is unseated from the midplane.
d. Place one hand on the module handle and pull the module part way out of the chassis.
e. Push the module back into the chassis until it is seated on the midplane.
f. Simultaneously rotate both ejector levers to the front of the module to fully seat the module.
g. Make sure that each of the captive screws on the module is aligned with its holes in the chassis.
h. Securely tighten each captive screw (to 8 in-lb [0.9 N·m]) to the chassis.
Step 4 Contact TAC for assistance (see the “Contacting Customer Service” section).
Contacting Customer Service
If you cannot solve a startup problem after using the troubleshooting suggestions in this chapter, contact the Technical Assistance Center (TAC) for assistance and additional instructions. Provide the following information to help your TAC representative address your problem as quickly as possible:
- Date that you received the switch
- Chassis serial number (located on a label on the right of the rear panel of the chassis)
- Type of software and release number
- Maintenance agreement or warranty information
- Brief description of the problem
- Brief explanation of the steps that you have already taken to isolate and resolve the problem
Note If you have CLI access, enter the show sprom all command to display the backplane contents.
To contact TAC, go to http://tools.cisco.com/ServiceRequestTool/create/launch.do.