The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter describes how to identify and resolve software problems related to the Cisco Industrial Ethernet 2000U Series (IE 2000U) and Connected Grid Switches, hereafter referred to as switch.
You can use the command-line interface (CLI) to identify and solve problems.
Additional troubleshooting information related to hardware is provided in the hardware installation guide.
NoteFor complete syntax and usage information for the commands used in this chapter, see the documents listed in the“Related Documents” section.
Note Recovery procedures require that you have physical access to the switch.
Switch software can be corrupted during an upgrade, by downloading the wrong file to the switch, and by deleting the image file. In all of these cases, the switch does not pass the power-on self-test (POST), and there is no connectivity.
This procedure uses the Xmodem Protocol at 115200 baud line speed to recover from a corrupt or wrong image file. There are many software packages that support the Xmodem Protocol, and this procedure is largely dependent on the emulation software that you are using.
This recovery procedure requires that you have physical access to the switch.
Step 1 From your PC, download the software image tar file ( image_filename.tar ) from Cisco.com.
The Cisco IOS image is stored as a bin file in a directory in the tar file. For information about locating the software image files on Cisco.com, see the release notes.
Step 2 Extract the bin file from the tar file.
1. Display the contents of the tar file by using the tar -tvf < image_filename.tar > UNIX command.
2. Locate the bin file, and extract it by using the tar -xvf < image_filename.tar > < image_filename.bin > UNIX command.
3. Verify that the bin file was extracted by using the ls -l < image_filename.bin > UNIX command.
Step 3 Connect your PC with terminal-emulation software supporting the Xmodem Protocol to the switch console port through a connection at 9600 bps. (The remaining steps assume the use of a HyperTerminal.)
Step 4 Ensure that the switch is in boot loader mode. If it is not, use one of these methods to the access boot loader mode:
On a PC, use Ctrl-Break. On a SUN work station running UNIX, use Ctrl-C.
Step 5 Use the set BAUD 115200 boot loader command to increase the baud rate of the switch console connection from 9600 bps to 115200 bps.
Step 6 Change the baud rate on the HyperTerminal to 115200 bps.
Step 7 On the switch, start the file transfer by using the Xmodem Protocol.
:
copy xmodem: flash:image_filename.bin
Step 8 After the Xmodem request appears, use the appropriate command on the terminal-emulation software to start the transfer and to copy the software image into flash memory.
Step 9 Boot the newly downloaded Cisco IOS image.
Step 10 Use the archive download-sw privileged EXEC command to download the software image to the switch.
Step 11 Use the reload privileged EXEC command to restart the switch and to verify that the new software image is operating properly.
Step 12 Delete the flash: image_filename.bin file from the switch.
This procedure uses the Xmodem Protocol at 9600 baud line speed and the Express Setup button to recover from a corrupt or wrong image file. There are many software packages that support the Xmodem Protocol, and this procedure is largely dependent on the emulation software that you are using.
This recovery procedure requires that you have physical access to the switch.
Step 1 From your PC, download the software image tar file ( image_filename.tar ) from Cisco.com.
The Cisco IOS image is stored as a bin file in a directory in the tar file. For information about locating the software image files on Cisco.com, see the release notes.
Step 2 Extract the bin file from the tar file.
1. Display the contents of the tar file by using the tar -tvf < image_filename.tar > UNIX command.
2. Locate the bin file, and extract it by using the tar -xvf < image_filename.tar > < image_filename.bin > UNIX command.
3. Verify that the bin file was extracted by using the ls -l < image_filename.bin > UNIX command.
Step 3 Connect your PC with terminal-emulation software supporting the Xmodem Protocol to the switch console port.
Step 4 Set the line speed on the emulation software to 9600 baud.
Step 5 Unplug the switch power cord.
Step 6 Press the Express Setup button and at the same time, reconnect the power cord to the switch.
You can release the Express Setup button a second or two after the LED above port 1 goes off. Several lines of information about the software appear along with instructions:
Step 7 Initialize the flash file system:
:
flash_init
Step 8 If you had set the console port speed to anything other than 9600, it has been reset to that particular speed. Change the emulation software line speed to match that of the switch console port.
:
load_helper
Step 10 Start the file transfer by using the Xmodem Protocol.
:
copy xmodem: flash:image_filename.bin
Step 11 After the Xmodem request appears, use the appropriate command on the terminal-emulation software to start the transfer and to copy the software image into flash memory.
Step 12 Boot the newly downloaded Cisco IOS image.
Step 13 Use the archive download-sw privileged EXEC command to download the software image to the switch.
Step 14 Use the reload privileged EXEC command to restart the switch and to verify that the new software image is operating properly.
Step 15 Delete the flash: image_filename.bin file from the switch.
If you lose or forget your password, you can delete the switch password and set a new one.
Before you begin, make sure that:
To delete the switch password and set a new one, follow these steps:
Step 1 Press the Express Setup button until the SETUP LED blinks green and the LED of an available downlink port blinks green.
If no switch downlink port is available for your PC or laptop connection, disconnect a device from one of the other downlink ports. Press the Express Setup button again until the SETUP LED and the port LED blink green.
Step 2 Connect your PC or laptop to the port with the blinking green LED.
The SETUP LED and the switch downlink port LED stop blinking and stay solid green.
Step 3 Press and hold the Express Setup button. Notice that the SETUP LED starts blinking green again. Continue holding the button until the SETUP LED turns solid green (approximately 5 seconds). Release the Express Setup button immediately.
This procedure deletes the password without affecting any other configuration settings. You can now access the switch without a password through the console port or by using the device manager.
Step 4 Enter a new password through the device manager by using the Express Setup window or through the command line interface by using the enable secret global configuration command.
The IEEE 802.3ab autonegotiation protocol manages the switch settings for speed (10, 100, and 1000 Mbps, excluding SFP module ports) and duplex (half or full). There are situations when this protocol can incorrectly align these settings, reducing performance. A mismatch occurs under these circumstances:
To maximize switch performance and ensure a link, follow one of these guidelines when changing the settings for duplex and speed:
NoteIf a remote device does not autonegotiate, configure the duplex settings on the two ports to match. The speed parameter can adjust itself even if the connected port does not autonegotiate.
These sections describe how to troubleshoot Power over Ethernet (PoE) ports.
If a powered device (such as a Cisco IP Phone) that is connected to a PoE switch port and is powered by an AC power source loses power from the AC power source, the device might enter an error-disabled state. To recover from an error-disabled state, enter the shutdown interface configuration command, and then enter the no shutdown interface command. You can also configure automatic recovery on the switch to recover from the error-disabled state. The errdisable recovery cause loopback and the errdisable recovery interval seconds global configuration commands automatically take the interface out of the error-disabled state after the specified period of time.
If a Cisco powered device is connected to a port and you configure the port by using the power inline never interface configuration command, a false link up can occur, placing the port into an error-disabled state. To take the port out of the error-disabled state, enter the shutdown and the no shutdown interface configuration commands.
Do not connect a Cisco powered device to a port that has been configured with the power inline never command.
Cisco small form-factor pluggable (SFP) modules have a serial EEPROM that contains the module serial number, the vendor name and ID, a unique security code, and cyclic redundancy check (CRC). When an SFP module is inserted in the switch, the switch software reads the EEPROM to verify the serial number, vendor name and vendor ID, and recompute the security code and CRC. If the serial number, the vendor name or vendor ID, the security code, or CRC is invalid, the software generates a security error message and places the interface in an error-disabled state.
NoteThe security error message references the GBIC_SECURITY facility. The switch supports SFP modules and does not support GBIC modules. Although the error message text refers to GBIC interfaces and modules, the security messages actually refer to the SFP modules and module interfaces. For more information about error messages, seeCisco System Messages.
If you are using a non-Cisco SFP module, remove the SFP module from the switch, and replace it with a Cisco module. After inserting a Cisco SFP module, use the errdisable recovery cause gbic-invalid global configuration command to verify the port status, and enter a time interval for recovering from the error-disabled state. After the elapsed interval, the switch brings the interface out of the error-disabled state and retries the operation. For more information about the errdisable recovery command, see the Cisco IOS Configuration Fundamentals Command Reference .
If the module is identified as a Cisco SFP module, but the system is unable to read vendor-data information to verify its accuracy, an SFP module error message is generated. In this case, you should remove and re-insert the SFP module. If it continues to fail, the SFP module might be defective.
You can check the physical or operational status of an SFP module by using the show interfaces transceiver privileged EXEC command. This command shows the operational status, such as the temperature and the current for an SFP module on a specific interface and the alarm status. You can also use the command to check the speed and the duplex settings on an SFP module. For more information, see the show interfaces transceiver command in the Cisco IOS Interface and Hardware Component Command Reference .
The switch monitors the temperature conditions to determine the health of the power supplies. The temperature value is the temperature in the switch (not the external temperature). Enter the show env temperature or show env all privileged EXEC command to see if the temperature is okay or exceeds a temperature threshold.
This is an example of the output from the show env temperature command:
This is an example of output from the show env all command:
The power supply temperature is monitored every 30 seconds. There are two temperature thresholds for power supplies:
If the critical threshold is surpassed, a syslog message is displayed every 30 seconds. When the warning threshold is crossed, a syslog message is displayed first, and if the condition persists, the syslog message is displayed every 2 minutes. If the power supply temperature reaches below the warning threshold, a syslog message is also displayed.
These are examples of syslog message that could be displayed:
Mar 1 00:04:50.203: %HARDWARE-5-PSU_THERMAL_NORMAL: Power Supply 1A Temperature is within the acceptable limit
Mar 1 00:04:53.207: %HARDWARE-2-PSU_THERMAL_WARNING: Power Supply 1B temperature has reached warning threshold
Mar 1 00:04:56.210: %HARDWARE-5-PSU_THERMAL_NORMAL: Power Supply 1B Temperature is within the acceptable limit
Mar 1 00:04:56.210: %HARDWARE-5-PSU_THERMAL_CRITICAL: Power Supply 1B temperature has reached critical threshold
You can use the CISCO-ENVMON-MIB and IDENTITY-MIB to receive traps that display information about temperatures, temperature thresholds, temperature sensors, and other related information. You must configure SNMP on the switch to access CISCO-ENVMON-MIB and IDENTITY-MIB objects. For more information, see the “Configuring SNMP” chapter in the System Management Software Configuration Guide for Cisco IE 2000U and Connected Grid Switches .
The switch supports IP ping, which you can use to test connectivity to remote hosts. Ping sends an echo request packet to an address and waits for a reply.
The switch also provides the Control Plane Security feature, which by default drops ping response packets received on user network interfaces (UNIs) or enhanced network interfaces (ENIs). However, methods are available to ping successfully from the switch to a host connected to a UNI or ENI.
Control Plane Security does not drop ping response packets to or from network node interfaces (NNIs), and no special configuration is required to enable pings to or from hosts connected to NNIs.
Beginning in privileged EXEC mode, use the ping command to ping another device on the network from the switch:
|
|
---|---|
Ping a remote host by supplying the hostname or IP network address. Note Though other protocol keywords are available with the ping command, they are not supported in this release. |
NotePing is not supported on a UNI or ENI configured as an IEEE 802.1Q tunnel port.
Ping is supported on NNIs on all software images.
It is important to note that the software images available for the switch provide different options for pinging a host connected to a UNI or ENI:
The next sections apply to both access ports and trunk ports.
For all software images for the switch, you can use a Layer 3 service policy to enable pings from the switch to a host connected to a UNI or ENI.
NoteFor a switch running the IP services image, IP routing is not enabled by default and does not have to be enabled to use a Layer 3 service policy.
This example is one possible configuration:
When your switch is running the IP services image, you can use any of these methods:
For a sample configuration of how to add a Layer 3 service policy to a UNI or ENI, see the “All Software Versions” section.
For examples using IP routing and pinging from an SVI or a routed port, see the next sections.
IP routing is only supported when the switch is running the IP services image.
You can use this configuration to enable IP routing and enable pings from an SVI to a host connected to a UNI or ENI.
With this configuration, a host with an IP address of 192.168.1.2 can be pinged from the switch.
You can use this configuration to enable IP routing, change a switchport to a routed port, and permit pings from the switch to a connected host:
This response is typical of a successful ping to a host:
An unsuccessful ping results in this message:
Keep these guidelines in mind while pinging:
If your switch is running the IP services image, use one of these methods to ping a host connected to a UNI or ENI:
To end a ping session, simultaneously press and release the Ctrl , Shift , and 6 keys, and then press the X key.
The Layer 2 traceroute feature allows the switch to identify the physical path that a packet takes from a source device to a destination device. Layer 2 traceroute supports only unicast source and destination MAC addresses. It finds the path by using the MAC address tables of the switches in the path. When the switch detects a device in the path that does not support Layer 2 traceroute, the switch continues to send Layer 2 trace queries and lets them time out.
NoteLayer 2 traceroute is available only on NNIs.
The switch can only identify the path from the source device to the destination device. It cannot identify the path that a packet takes from source host to the source device or from the destination device to the destination host.
Note CDP is enabled by default on NNIs. You can enable CDP on ENIs, but UNIs do not support CDP.
For a list of switches that support Layer 2 traceroute, see the “Layer 2 Traceroute Usage Guidelines” section. If any devices in the physical path are transparent to CDP, the switch cannot identify the path through these devices. For more information about enabling CDP, see the Configuring CDP” in the System Management Software Configuration Guide for Cisco IE 2000U and Connected Grid Switches .
– If an ARP entry exists for the specified IP address, the switch uses the associated MAC address and identifies the physical path.
– If an ARP entry does not exist, the switch sends an ARP query and tries to resolve the IP address. If the IP address is not resolved, the path is not identified, and an error message appears.
You can display the physical path that a packet takes from a source device to a destination device by using one of these privileged EXEC commands:
NoteLayer 2 traceroute is available only on NNIs.
For more information, see the Cisco IOS Configuration Fundamentals Command Reference .
You can use IP traceroute to identify the path that packets take through the network on a hop-by-hop basis. The command output displays all network layer (Layer 3) devices, such as routers, that the traffic passes through on the way to the destination.
Your switches can participate as the source or destination of the traceroute privileged EXEC command and might or might not appear as a hop in the traceroute command output. If the switch is the destination of the traceroute, it is displayed as the final destination in the output. Intermediate switches do not show up in the output if they are only bridging the packet from one port to another within the same VLAN. However, if the intermediate switch is a multilayer switch that is routing a particular packet, this switch shows up as a hop in the output.
The traceroute privileged EXEC command uses the Time To Live (TTL) field in the IP header to cause routers and servers to generate specific return messages. Traceroute starts by sending a User Datagram Protocol (UDP) datagram to the destination host with the TTL field set to 1. If a router finds a TTL value of 1 or 0, it drops the datagram and sends an Internet Control Message Protocol (ICMP) time-to-live-exceeded message to the sender. Traceroute finds the address of the first hop by examining the source address field of this message.
To identify the next hop, traceroute sends a UDP packet with a TTL value of 2. The first router decrements the TTL field by 1 and sends the datagram to the next router. The second router sees a TTL value of 1, discards the datagram, and returns the time-to-live-exceeded message to the source. This process continues until the TTL is incremented to a value large enough for the datagram to reach the destination host (or until the maximum TTL is reached).
To learn when a datagram reaches its destination, traceroute sets the UDP destination port number in the datagram to a very large value that the destination host is unlikely to be using. When a host receives a datagram destined to itself containing a destination port number that is unused locally, it sends an ICMP port-unreachable error to the source. Because all errors except port-unreachable errors come from intermediate hops, the receipt of a port-unreachable error means that this message was sent by the destination port.
Beginning in privileged EXEC mode, follow this step to trace that the path packets take through the network:
|
|
---|---|
NoteThough other protocol keywords are available with thetraceroute privileged EXEC command, they are not supported in this release.
This example shows how to perform a traceroute to an IP host:
The display shows the hop count, IP address of the router, and the round-trip time in milliseconds for each of the three probes that are sent. The following table describes the traceroute output display characters:
|
|
---|---|
Administratively unreachable. Usually, this output means that an access list is blocking traffic. |
|
To end a trace in progress, enter the escape sequence ( Ctrl-^ X by default). Simultaneously press and release the Ctrl , Shift , and 6 keys, and then press the X key.
You can use the Time Domain Reflector (TDR) feature to diagnose and resolve cabling problems. When running TDR, a local device sends a signal through a cable and compares the reflected signal to the initial signal.
On the switch, TDR is supported only on the copper Ethernet 10/100 ports or on dual-purpose ports configured as 10/100/100 ports by using the RJ-45 connector.
TDR can detect these cabling problems:
If one of the twisted-pair wires is open, TDR can find the length at which the wire is open.
Use TDR to diagnose and resolve cabling problems in these situations:
To run TDR, enter the test cable-diagnostics tdr interface interface-id privileged EXEC command:
To display the results, enter the show cable-diagnostics tdr interface interface-id privileged EXEC command. For a description of the fields in the display, see the Cisco IOS Interface and Hardware Component Command Reference .
NoteTDR is supported only on the copper Ethernet 10/100 ports or on dual-purpose ports configured as 10/100/100 ports by using the RJ-45 connector.
NoteFor complete syntax and usage information for specificdebug commands, see the Cisco IOS Debug Command Reference.
All debug commands are entered in privileged EXEC mode, and most debug commands take no arguments. For example, beginning in privileged EXEC mode, enter this command to enable the debugging for Switched Port Analyzer (SPAN):
The switch continues to generate output until you enter the no form of the command.
If you enable a debug command and no output appears, consider these possibilities:
To disable debugging of SPAN, enter this command in privileged EXEC mode:
Alternately, in privileged EXEC mode, you can enter the undebug form of the command:
To display the state of each debugging option, enter this command in privileged EXEC mode:
Beginning in privileged EXEC mode, enter this command to enable all-system diagnostics:
The no debug all privileged EXEC command disables all diagnostic output. Using the no debug all command is a convenient way to ensure that you have not accidentally left any debug commands enabled.
By default, the network server sends the output from debug commands and system error messages to the console. If you use this default, you can use a virtual terminal connection to monitor debug output instead of connecting to the console port.
Possible destinations include the console, virtual terminals, internal buffer, and UNIX hosts running a syslog server. The syslog format is compatible with 4.3 Berkeley Standard Distribution (BSD) UNIX and its derivatives.
NoteBe aware that the debugging destination you use affects system overhead. Logging messages to the console produces very high overhead, whereas logging messages to a virtual terminal produces less overhead. Logging messages to a syslog server produces even less, and logging to an internal buffer produces the least overhead of any method.
For more information about system message logging, see the “Configuring System Message Logging” chapter in the System Management Software Configuration Guide for Cisco IE 2000U and Connected Grid Switches .
The output from the show platform forward privileged EXEC command provides some useful information about the forwarding results if a packet entering an interface is sent through the system. Depending upon the parameters entered about the packet, the output provides lookup table results and port maps used to calculate forwarding destinations, bitmaps, and egress information.
Most of the information in the output from the command is useful mainly for technical support personnel, who have access to detailed information about the switch ASICs. However, packet forwarding information can also be helpful in troubleshooting.
This is an example of the output from the show platform forward command on Gigabit Ethernet port 1 in VLAN 5 when the packet entering that port is addressed to unknown MAC addresses. The packet should be flooded to all other ports in VLAN 5.
This is an example of the output when the packet coming in on Gigabit Ethernet port 1 in VLAN 5 is sent to an address already learned on the VLAN on another port. It should be forwarded from the port on which the address was learned.
This is an example of the output when the packet coming in on Gigabit Ethernet port 1 in VLAN 5 has a destination MAC address set to the router MAC address in VLAN 5 and the destination IP address unknown. Because there is no default route set, the packet should be dropped.
This is an example of the output when the packet coming in on Gigabit Ethernet port 1 in VLAN 5 has a destination MAC address set to the router MAC address in VLAN 5 and the destination IP address set to an IP address that is in the IP routing table. It should be forwarded as specified in the routing table.
The crashinfo file saves information that helps Cisco technical support representatives to debug problems that caused the Cisco IOS image to fail (crash). The switch writes the crash information to the console at the time of the failure, and the file is created the next time you boot the Cisco IOS image after the failure (instead of while the system is failing).
The information in the file includes the Cisco IOS image name and version that failed, a list of the processor registers, and a stack trace. You can provide this information to the Cisco technical support representative by using the show tech-support privileged EXEC command.
All crashinfo files are kept in this directory on the flash file system:
flash:/crashinfo/crashinfo_ n where n is a sequence number.
Each new crashinfo file that is created uses a sequence number that is larger than any previously existing sequence number, so the file with the largest sequence number describes the most recent failure. Version numbers are used instead of a timestamp because the switches do not include a real-time clock. You cannot change the name of the file that the system will use when it creates the file. However, after the file is created, you can use the rename privileged EXEC command to rename it, but the contents of the renamed file will not be displayed by the show tech-support privileged EXEC command. You can delete crashinfo files by using the delete privileged EXEC command.
You can display the most recent crashinfo file (that is, the file with the highest sequence number at the end of its filename) by entering the show tech-support privileged EXEC command. You also can access the file by using any command that can copy or display files, such as the more or the copy privileged EXEC command.
You can use the on-board-failure logging (OBFL) feature to collect information about the switch. The information includes uptime, temperature, and voltage information and helps Cisco technical support representatives to troubleshoot switch problems.
This section has this information:
By default, OBFL is enabled. It collects information about the switch and small form-factor pluggable (SFP) modules. The switch stores this information in the flash memory:
You should manually set the system clock or configure it by using Network Time Protocol (NTP).
When the switch is running, you can retrieve the OBFL data by using the show logging onboard privileged EXEC commands. If the switch fails, contact your Cisco technical support representative to find out how to retrieve the data.
When an OBFL-enabled switch is restarted, there is a 10-minute delay before logging of new data begins.
To enable OBFL, use the hw-module module logging onboard [ message level level ] global configuration command. Use the message level level parameter to specify the severity of the hardware-related messages that the switch generates and stores in the flash memory.
To copy the OBFL data to the local network or a specific file system, use the copy logging onboard module 1 destination privileged EXEC command.
Beginning in privileged EXEC mode, follow these steps to enable and configure OBFL. Note that OBLF is enabled by default; you need to enable it only if it has been disabled.
|
|
|
---|---|---|
hw-module module [ slot-number ] logging onboard [ message level ] |
||
(Optional) Copy the OBFL data to the local network or a specific file system. |
||
To disable OBFL, use the no hw-module module 1 logging onboard [ message level ] global configuration command.
To clear all the OBFL data in the flash memory except for the uptime and CLI command information, use the clear logging onboard privileged EXEC command.
To display the OBFL information, use one or more of the privileged EXEC commands in the following table.
NoteWhen an OBFL-enabled switch is restarted, there is a 10-minute delay before logging of new data begins.
These are examples of output from the show logging onboard commands:
|
|
---|---|