Verifying Hardware Diagnostics and Logging
This chapter contains the Cisco NX-OS recommended features and procedures for managing and troubleshooting potential hardware faults.
This chapter includes the following sections:
•Online Diagnostics
•Onboard Failure-Logging
Online Diagnostics
Generic Online Diagnostics (GOLD) provides hardware testing verification that is useful for detecting hardware faults. If a fault is detected, corrective action is taken to mitigate the fault to reduce potential network outages. GOLD tests are executed when a chassis is powered up, for an online insertion and removal (OIR) event, as health checks occur in the background (continuous testing), and on demand from the CLI.
Enabling GOLD
Introduced: Cisco NX-OS Release 4.0(1)
Generic Online Diagnostics are enabled by default. (We do not recommend disabling online diagnostics.) In the event online diagnostics have been disabled, they can be enabled using the following command.
n7000(config)# diagnostic bootup level complete
Understanding Diagnostic Contents (Per Module)
Introduced: Cisco NX-OS Release 4.0(1)
The show diagnostic content command displays the available tests for a module and the associated attributes for each test. This is useful for determining what tests are available for a module and if the tests are disruptive prior to running an on-demand test.
n7000# show diagnostic content module 1
Module 1: 10/100/1000 Mbps Ethernet Module
Diagnostics test suite attributes:
B/C/* - Bypass bootup level test / Complete bootup level test / NA
M/S/* - Only applicable to active / standby unit / NA
D/N/* - Disruptive test / Non-disruptive test / NA
H/* - Always enabled monitoring test / NA
F/* - Fixed monitoring interval test / NA
X/* - Not a health monitoring test / NA
E/* - Sup to line card test / NA
L/* - Exclusively run this test / NA
T/* - Not an ondemand test / NA
A/I/* - Monitoring is active / Monitoring is inactive / NA
ID Name Attributes (hh:mm:ss)
____ __________________________________ ____________ _________________
1) EOBCPortLoopback--------------> C**N**X**T* -NA-
2) ASICRegisterCheck-------------> ***N******A 00:01:00
3) PrimaryBootROM----------------> ***N******A 00:30:00
4) SecondaryBootROM--------------> ***N******A 00:30:00
5) PortLoopback------------------> CP*N***E**A 00:15:00
On-Demand Tests
Introduced: Cisco NX-OS Release 4.0(1)
On Demand tests should be executed anytime hardware is suspected to be faulty. An on-demand test is executed from Exec mode. GOLD tests can be disruptive and non-disruptive, so caution should be taken to prevent any network outages. If a GOLD test is disruptive the administrator will be prompted to continue.
n7000# diagnostic start module 1 test 6 port 1
Verifying GOLD Test Results (Per Module)
Introduced: Cisco NX-OS Release 4.0(1)
The following command checks the GOLD test results for module 1. The detail option provides timestamp information for each test, which is useful for determining when a test may have passed or failed.
n7000# show diagnostic result module 1
Current bootup diagnostic level: complete
Module 1: 10/100/1000 Mbps Ethernet Module
Test results: (. = Pass, F = Fail, I = Incomplete,
U = Untested, A = Abort, E = Error disabled)
1) EOBCPortLoopback--------------> .
2) ASICRegisterCheck-------------> .
3) PrimaryBootROM----------------> .
4) SecondaryBootROM--------------> .
Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
-----------------------------------------------------
U U U U U U U U U U U U U U U U
Port 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
-----------------------------------------------------
U U U U U U U U U U U U U U U U
Port 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
-----------------------------------------------------
Onboard Failure-Logging
Onboard Failure Logging (OBFL) provides persistent event logging per module that contains detailed information that can be useful when troubleshooting. OBFL is enabled by default and should not be disabled. The data in some of the OBFL logs may be difficult to understand, but the logs are useful when the Cisco TAC is diagnosing a potential hardware issue.
Enabling/Disabling OBFL
Introduced: Cisco NX-OS Release 4.0(1)
This section was included for reference and may not be required.
An OBFL log can be disabled per system or per module. The following example illustrates how to enable an OBFL log per module in the event it was previously disabled. This action is typically not required since they are enabled by default.
n7000(config)# hw-module logging onboard module 1 environmental-history
Module: 1 Enabling environmental-history ... was successful.
Viewing Log Contents
Introduced: Cisco NX-OS Release 4.0(1)
OBFL logs can be viewed per system (all logs), per log type (i.e. environmental-history) for all modules, and per module/log type. The "|" option can be used to redirect output to a file if the contents need to be sent to a remote destination. Since logs are persistent they can contain a large amount of data.
n7000# show logging onboard module 1 environmental-history
----------------------------
----------------------------
========== Sensor Temperature History Log ==========
----------------------------------------------------
Fri Apr 9 11:20:24 2010 sensor 13 temperature 53
Fri Apr 9 11:36:25 2010 sensor 14 temperature 54
Clearing Log Contents
Introduced: Cisco NX-OS Release 4.0(1)
The following clear log onboard command can be used to clear the contents for all logs, a log type for all modules, or a specific log type for a specified module.
n7000# clear log onboard ?
counter-stats Clear OBFL counter statistics
environmental-history Clear OBFL environmental history
error-stats Clear OBFL error statistics
exception-log Clear OBFL exception log
fex Clear OBFL information for FEX
internal Clear Logging Onboard Internal
interrupt-stats Clear OBFL interrupt statistics
module Clear OBFL information for Module
obfl-logs Clear OBFL (boot-uptime/device-version/obfl-history).
stack-trace Clear OBFL stack trace