Table Of Contents
Router Field Diagnostics
Diagnostics Overview
Downloading the Diagnostic Image
Field-Programmable Gate Array Overview
Upgrading an FPGA Image on a Line Card
Using the diag Command
Diagnostic Commands for Cisco 12000 Series Line Cards
Diagnostic Commands for Cisco 12000 Series Rout Processors
Diagnostic Commands for Cisco 12000 Series Switch Fabric Cards and Clock Scheduler Cards
Diagnostic Command Reference Table
diag Command Output Examples
Testing Without the verbose Option
Testing Using the verbose Option
Displaying Previous Test Results
Router Field Diagnostics
This chapter describes the diagnostic commands you can use to troubleshoot line cards, Route Processor cards (PRP and GRP), Switch Fabric Cards (SFC), and Clock Scheduler Cards (CSC) installed in Cisco 12010, Cisco 12410, and Cisco 12810 routers.
You can run field diagnostics to help isolate a faulty line card without disrupting the operation of the system. Field diagnostics are not designed to identify specific components within the router. They simply determine whether a particular card is operational or defective.
Note
Starting with Cisco IOS Release 12.0(22)S, the line card field diagnostics image is not bundled with the main Cisco IOS image. Line card field diagnostics are now stored and maintained as a separate image that must be available on a flash memory card or TFTP boot server before you enter the diagnostic commands. Route processor and switch fabric card field diagnostics continue to be bundled with the main Cisco IOS image and do not need to be launched from a separate image.
Unbundled field diagnostics provide the following benefits:
•
In-service testing—Field diagnostics can be run on an in-service router running Cisco IOS and only take the card being tested out of service. The tested card is returned to normal operation if the diagnostic tests pass. If there is a hardware fault, the card remains out of service after the testing is completed.
•
Smaller Cisco IOS image—The smaller IOS image helps accommodate customers with 20 MB flash memory cards.
•
Easy access to the most current diagnostics software—Because the field diagnostic software is now maintained as a separate image, the most recent version is always available on Cisco.com, without regard to the Cisco IOS software release currently in use.
This chapter describes the following topics:
•
Diagnostics Overview
•
Field-Programmable Gate Array Overview
•
Using the diag Command
•
diag Command Output Examples
Diagnostics Overview
More than 100 diagnostic operations for each router line card exist, in addition to diagnostics for the switch fabric and route processor cards. These operations include the following:
•
Processor tests
•
Memory tests
•
Component tests
•
Major data path tests
•
Field-programmable gate array (FPGA) image updates (on some line cards)
Note
When using Cisco IOS Release 12.0(21)S or 12.0(21)ST or later, the default download method changes from the MBus to the switch fabric to make the process faster. It takes about 1 minute to obtain test results from the switch fabric compared to 15 minutes from the MBus.
While diagnostics are running, the line card being tested is taken offline and is controlled by the diagnostic software. The diagnostics affect only the line card being tested; the other line cards remain online and continue to pass traffic normally. Diagnostics do not affect system performance.
Diagnostic testing stops when all of the tests are completed, are terminated by the user, or by default if an error is encountered. If multiple cards are specified for the test cycle, the diagnostics stop testing a card when it fails a test, but then continues testing the remaining cards.
When testing is finished, a pass or fail message appears on the console, and on the alphanumeric LED display on the card being tested.
Downloading the Diagnostic Image
Before you can use the diagnostic commands, you must download a valid diagnostic image to a flash memory card installed in the router or to an TFTP boot server.
Field diagnostic images are 10 to 18 MB in size, and Cisco IOS images are slightly larger. One 64 MB flash memory card can store both images, or the images can be stored individually on two 20 MB memory cards.
The name of the field diagnostic image is c12k-fdiagsbflc-mz and the latest version is always available on Cisco.com. The image name is combined with the Cisco IOS release number, for example: c12k-fdiagsbflc-mz 120-25.S
In this example, 120-25.S is the release number of the image that corresponds to the Cisco IOS image: Cisco IOS Release 12.0(25)S.
Note
All Cisco field diagnostic images available on Cisco.com are compatible with Cisco IOS Release 12.0(22)S and later. We recommend that you use the latest available diagnostic image for testing and verifying line cards.
Field-Programmable Gate Array Overview
Some Cisco 12000 series line cards store hardware information as software code in the form of a field-programmable gate array, or FPGA. This code is permanently placed in flash memory on the line card. Occasionally, this flash memory code must be updated with new FPGA code containing feature enhancements or code improvements.
Some FPGA images are bundled within the Cisco IOS image and are updated along with a new release. These type of FPGA images do not require flash memory storage and are developed and expanded within Cisco IOS feature sets. FPGA images stored in flash memory on the line card receive updates occasionally. When the line card boots, Cisco IOS software may use an FPGA image bundled within itself, or it may use the FPGA image stored in flash memory on the line card.
Cisco IOS checks the revision of the FPGA code in flash memory during the boot process and informs you if an upgrade is necessary.
The following example of Cisco IOS software boot text shows an FPGA upgrade message (see underlined section):
*Aug 19 14:51:06 UTC: %MBUS-6-FABCONFIG: Switch Cards 0x1F (bitmask)
Fabric Clock is Redundant
Bandwidth Mode : Full Bandwidth
*Aug 19 14:51:23 UTC: %MBUS-6-NO_FPGA_IMG: FPGA image is not
appropriate or corrupted for slot 0. Please run Field Diagnostics
image on slot 0 to upgrade the FPGA image.
*Aug 19 14:51:31 UTC: %GRPGE-6-SYNC_LOSS: Interface
GigabitEthernet2/0: Loss of Sync
*Aug 19 14:51:31 UTC: %GRPGE-6-RX_LOS: Interface
GigabitEthernet2/0: Detected RX Loss of Signal
SLOT 2:00:00:12: %SYS-5-RESTART: System restarted --
Cisco Internetwork Operating System Software
IOS (tm) GS Software (GLC1-LC-M), Experimental Version
12.0(20030605:093502) [mliflian-25-thr-work-nog-lke 115]
Copyright (c) 1986-2003 by cisco Systems, Inc.
Compiled Tue 19-Aug-03 17:33 by mliflian
If an upgrade message appears, you can run the field diagnostics to update the image in flash memory. No actual tests are initiated when using field diagnostics to update the FPGA; it only performs the update to the FPGA code. See the "Upgrading an FPGA Image on a Line Card" section.
When field diagnostics run, the target line card stops passing traffic and goes offline (the rest of the router continues passing traffic normally). The line card is loaded with the field diagnostics program, the FPG update is run, and if successful, the card reboots to Cisco IOS. The entire process takes approximately 2 minutes and must be done one line card at a time.
Note
Before updating the next line card, wait about 30 seconds after the Cisco IOS software reloads on the line card to give the router a chance to synchronize all of its internal functions.
Upgrading an FPGA Image on a Line Card
When the Cisco IOS image boots, it verifies that a compatible FPGA image is running on the router. The major version number of the FPGA image must be the same as that expected by the Cisco IOS image; the minor version number on the FPGA image must be the same as or greater than the minor version number expected by the Cisco IOS image.
For example, if the Cisco IOS image expects a minimum FPGA image of 03.02, the software verifies that the actual major version number of the FPGA image in the line card bootflash is 03, and that the minor version number is 02 or above.
If a line card does not boot and an error message indicates that there is a problem with the FPGA image, or if the line card alphanumeric LED display remains stuck in the IOS STRT stage, you must upgrade the FPGA image using the diag command.
Use the following procedure to upgrade the FPGA image on a line card.
Step 1
Enter the enable command and password (if prompted).
Step 2
Enter the diag slot-number update-fpga source {tftp | flash} source-path command to update the flash memory.
Where:
•
slot-number is the slot number of the line card you are upgrading.
•
source-path is the path to the diagnostic image.
–
For flash memory cards, the source path is typically:
slot0:c12k-fdiagsbflc-mz.120-25.S or slot1:c12k-fdiagsbflc-mz.120-25.S.
–
For TFTP boot servers, the source path is typically: tftp://tftp_server_ip_address/my_directory/c12k-fdiagsbflc-mz.120-25.S.
Caution 
Do not unplug the line card or terminate the field diagnostics session during this test.
The following example shows the output of a line card with the update-fpga option specified.
Router# diag 7 verbose source tftp
tftp://223.255.254.254/c12k-fdiagsbflc-mz.120-25.S update-fpga
Running DIAG config check
Fabric Download for Field Diags chosen: If timeout occurs, try 'mbus'
option.
Verbose mode: Test progress and errors will be displayed
UUT will update FPGA's flash
Runnning Diags will halt ALL activity on the requested slot.
[confirm]
PID of f_diag_run is 121, set test_pid[3]
Launching a Field Diagnostic for slot 7
Loading c12k-fdiagsbflc-mz.new_fpga from 223.255.254.254 (via
Ethernet0):
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Calling enable_slot_and_type_on_fabric(DISABLE) for slot 7
Calling enable_slot_and_type_on_fabric(ENABLE) for slot 7
FD 3> *****************************************************
FD 3> GSR Field Diagnostics V6.9
FD 3> Compiled by ckhuong on Fri Jan 17 15:51:36 PST 2003
FD 3> view: ckhuong-conn_isp.inti4
FD 3> *****************************************************
Diagnostics have been downloaded to slot 7
Executing all diagnostic tests in slot 7
(total/indiv. timeout set to 2000/600 sec.)
FD 3> BFR_CARD_TYPE_4P_GE_E48 testing...
FD 3> Available test types 2
FD 3> Completed f_diags_board_discovery() (0x32)
FD 3> Verbosity now (0x00000011) TESTSDISP FATL
FD 3> Test list selection received: Test ID 3, Device 0
FD 3> running in slot 7 (1 tests from test list ID 3)
FD 3> Just into idle state
FDIAG_STAT_IN_PROGRESS(3): test #1 Tetra Flash Mem Upgrading
FD 3> Upgraded FPGA image in Flash from version 0xcc53 to 0xcc54
Calling enable_slot_and_type_on_fabric(DISABLE) for slot 7
Calling enable_slot_and_type_on_fabric(ENABLE) for slot 7
FD 3> Changed current_status to FDIAG_STAT_IDLE
COMPLETED Field Diags: pid 121, status 6, test_num 1, error_code 0
Field Diagnostic ****PASSED**** for slot 7
Field Diag results from eeprom before updating slot 7, run# 0x10 were
0x0
previous field diag eeprom values: run 16 fail mode 0 (PASS)
last test failed was 0, error code 0
Field Diag eeprom values: run 17 fail mode 0 (PASS) slot 7
last test failed was 0, error code 0
Shutting down diags in slot 7
Using the diag Command
You can use the diag command to run field diagnostics on a line card, Route Processor card (PRP and GRP), Switch Fabric Card (SFC), or Clock and Scheduler Card (CSC) in Cisco 12000 Series routers.
Caution 
Performing field diagnostics on a line card stops all activity on the line card. Before the
diag command begins running diagnostics, you are prompted to confirm the request to perform field diagnostics on the line card.
Caution 
Some line cards include components that are unable to isolate internal line card testing traffic from customer premise connections. You are warned to disconnect any connections to these line cards before testing to achieve the most reliable results and minimize traffic disruption.
Refer to Table 6-1 for a description of the command variables.
Diagnostic Commands for Cisco 12000 Series Line Cards
The following diagnostic commands apply to line cards.
diag slot-number source {tftp | flash} source-path [verbose] [wait] [full]
[coe]
[dl-timeout-plus <1-2000>] [device] [messaging]
diag slot-number previous
diag slot-number halt
diag slot-number update-fpga source {tftp | flash} source-path
[dl-timeout-plus <1-2000>]
Diagnostic Commands for Cisco 12000 Series Rout Processors
The following diagnostic commands apply to GRP and PRP route processors.
diag slot-number [verbose] [wait] [full] [coe]
diag slot-number previous
diag slot-number halt
Diagnostic Commands for Cisco 12000 Series Switch Fabric Cards and Clock Scheduler Cards
The following diagnostic command applies to SFC and CSC fabric cards.
diag slot-number [verbose]
Diagnostic Command Reference Table
Table 6-1 lists the diag command keywords, options, variables, and their descriptions.
Table 6-1 diag Command Reference
Command
|
Description
|
slot-number
|
Specifies the slot number of the card you want to test.
|
source
|
Specifies the source path of the line card diagnostic image. The source keyword must be followed by either the tftp or flash keyword.
|
tftp
|
Specifies that a TFTP server is the source of the diagnostic image. The tftp keyword must be followed by the source-path variable. To manipulate the download time for slow TFTP connections, see the dl-timeout-plus option.
|
flash
|
Specifies that a flash memory card is the source of the diagnostic image. The flash keyword must be followed by the source-path variable.
|
source-path
|
Specifies the path to the diagnostic image file c12k-fdiagsbflc-mz.IOS Release
Where IOS Release corresponds to the Cisco IOS release number. For example:
c12k-fdiagsbflc-mz.120-25.S
In this example 120-25.S corresponds to Cisco IOS Release 12.0(25)S.
The following are examples of the source path for flash cards:
• slot0:c12k-fdiagsbflc-mz.120-25.S
• slot1:c12k-fdiagsbflc-mz.120-25.S.
The following is an example of the source path for a TFTP server:
• tftp://tftp_server_ip_address/my_directory/c12k-fdiagsbflc-mz.120-25.S.
|
halt
|
(Optional) Stops diagnostic testing on the line card.
If you stop the field diagnostic test using the diag halt command, the line card remains down (in an unbooted state). Generally, you stop testing to remove or replace the line card. If this is not the case, and you need to bring the line card back up (online), use the microcode reload slot global configuration command, or power cycle the line card by removing and replacing it in its slot.
|
previous
|
(Optional) Displays previous test results (if any) for the card.
|
verbose
|
(Optional) Enables progress and error messages to be displayed on the console.
By default, only the minimum status messages appear on the console, along with the final result. Use the verbose option to view all of the tests performed.
Due to the comprehensive nature of the diagnostics, testing without the verbose option can cause a delay of up to 15 minutes before any results are displayed. We recommend that you specify the verbose option to capture results when communicating with Cisco TAC.
|
wait
|
(Optional) Stops the automatic reloading of the Cisco IOS software on the line card after the successful completion of diagnostic testing.
If you use the wait keyword, you must use the microcode reload slot global configuration command to reload the Cisco IOS software image to the line card, or you can manually remove and insert the line card in the slot (to initialize it) so the RP recognizes the line card.
|
coe
|
(Optional) Continue On Error.
Normally the field diagnostics stop immediately upon failing any one test during a test session. coe forces the testing to continue to the end of the internal test list, even if a failure occurs.
Caution  If you use this option and a cascade of failures is found, it may require that you reload the router, affecting all RPs and line cards.
|
full
|
(Optional) Forces a line card or RP to complete the most extensive set of tests.
The default set of diagnostics emphasize memory and data path tests. Use the full option to run the complete set of tests.
|
dl-timeout-plus <1-2000 seconds>
|
(Optional) Manipulates the download timeout value for users with slow TFTP boot paths.
300 seconds is the baseline value. Any value from 1 to 2000 adds that value to 300. For example, a dl-timeout-plus value of 12 makes the total value 312 seconds.
|
update-fpga
|
(Optional) Updates flash memory with FPGA images from the current field diagnostics download image.
This option limits the field diagnostics session to one task; updating the FPGA images in flash memory on the line card. No other testing is performed during this session. This process is non-selective in cases where a line card contains multiple FPGA images.
Caution  Do not unplug the line card or terminate the field diagnostics session during an FPGA upgrade.
|
messaging
|
Provides additional troubleshooting information.
|
After all diagnostic tests are completed on a line card, a PASSED or TEST FAILURE message appears on the console.
•
If the line card passes, the Cisco IOS software image on the line card is automatically reloaded (unless the wait keyword is specified).
•
If the line card fails, the Cisco IOS software image on the line card is not automatically reloaded.
If the line card fails the test, the line card is defective and must be replaced. Under certain circumstances, TAC engineers may direct you to replace field-replaceable memory modules and then retest. For example, if the DRAM test fails, you might only need to replace the DRAM on the line card. However, this should only be performed under the guidance of a TAC engineer and only when observing tightly controlled static-sensitive device handling procedures.
Caution 
Do not replace memory components without properly grounding both the board and yourself.
diag Command Output Examples
This section contains output examples using diag commands.
Caution 
Performing field diagnostics on a line card stops all activity on the line card. Before the
diag command begins running diagnostics, you are prompted to confirm the request to perform field diagnostics on the line card.
Testing Without the verbose Option
The following example shows the output of diagnostic testing performed on a line card in slot 7 without the verbose option specified. After the line card passes all field diagnostic tests, Cisco IOS software automatically reloads on the card.
Note that the total/indiv. timeout set to 2000/600 sec. message indicates that 2000 seconds are allowed to perform all field diagnostics tests, and that no one test should exceed 600 seconds to complete.
Router# diag 7 source tftp
tftp://192.164.5.4/images/c12k-fdiagsbflc-mz.120-25.S
Running DIAG config check
Fabric Download for Field Diags chosen: If timeout occurs, try 'mbus'
option.
Running Diags will halt ALL activity on the requested slot. [confirm]
Launching a Field Diagnostic for slot 7
Downloading diagnostic tests to slot 7 via fabric (timeout set to 300
sec.)
5d20h: %GRP-4-RSTSLOT: Resetting the card in the slot: 7,Event:
EV_ADMIN_FDIAG
Loading images/award/c12k-fdiagsbflc-mz from 192.164.1.1 (via
Ethernet0): !!!!!
5d20h: Downloading diags from tftp file
tftp://192.164.1.1/images/award/c12k-fdiagsbflc-mz
!!!!![OK - 13976524 bytes]
FD 7> *****************************************************
FD 7> GSR Field Diagnostics V6.05
FD 7> Compiled by award on Tue Jul 30 13:00:41 PDT 2002
FD 7> view: award-conn_isp.FieldDiagRelease
FD 7> *****************************************************
Executing all diagnostic tests in slot 7
(total/indiv. timeout set to 2000/600 sec.)
FD 7> BFR_CARD_TYPE_OC12_4P_POS testing...
FD 7> Available test types 2
FD 7> Completed f_diags_board_discovery() (0x1)
FD 7> Test list selection received: Test ID 1, Device 0
FD 7> running in slot 7 (30 tests from test list ID 1)
FD 7> Skipping MBUS_FDIAG command from slot 2
FD 7> Just into idle state
Field Diagnostic ****PASSED**** for slot 7
Shutting down diags in slot 7
5d20h: %GRP-4-RSTSLOT: Resetting the card in the slot: 7,Event:
EV_ADMIN_FDIAG
SLOT 7:00:00:09: %SYS-5-RESTART: System restarted --
Cisco Internetwork Operating System Software
IOS (tm) GS Software (GLC1-LC-M), Experimental Version
12.0(20020509:045149) [award-conn_isp.f_diag_new 337]
Copyright (c) 1986-2002 by cisco Systems, Inc.
Compiled Tue 25-Jun-02 15:51 by award
Testing Using the verbose Option
The following example shows the output of diagnostic testing performed on a line card in slot 7 with the verbose option specified (recommended).
Router# diag 7 verbose tftp
tftp://192.164.1.1/images/award/c12k-fdiagsbflc-mz.120-25.S
Running DIAG config check
Fabric Download for Field Diags chosen: If timeout occurs, try 'mbus'
option.
Verbose mode: Test progress and errors will be displayed
Runnning Diags will halt ALL activity on the requested slot.
[confirm]
Launching a Field Diagnostic for slot 7
Downloading diagnostic tests to slot 7 via fabric (timeout set to 300
sec.)
00:07:41: %GRP-4-RSTSLOT: Resetting the card in the slot: 7,Event:
EV_ADMIN_FDIAG
Loading images/award/c12k-fdiagsbflc-mz from 192.164.1.1 (via
Ethernet0): !!!!!! (...)
00:08:24: Downloading diags from tftp file
tftp://192.164.1.1/images/award/c12k-fdiagsbflc-mz
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!
FD 7> *****************************************************
FD 7> GSR Field Diagnostics V6.05
FD 7> Compiled by award on Tue Jul 30 13:00:41 PDT 2002
FD 7> view: award-conn_isp.FieldDiagRelease
FD 7> *****************************************************
Executing all diagnostic tests in slot 7
(total/indiv. timeout set to 2000/600 sec.)
FD 7> BFR_CARD_TYPE_OC12_4P_POS testing...
FD 7> Available test types 2
FD 7> Completed f_diags_board_discovery() (0x1)
FD 7> Verbosity now (0x00000011) TESTSDISP FATL
FD 7> Test list selection received: Test ID 1, Device 0
FD 7> running in slot 7 (30 tests from test list ID 1)
FD 7> Just into idle state
FDIAG_STAT_IN_PROGRESS(7): test #1 Dram Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #2 Dram Datapins
FDIAG_STAT_IN_PROGRESS(7): test #3 Dram Busfloat
FDIAG_STAT_IN_PROGRESS(7): test #4 RBM SDRAM Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #5 RBM SDRAM Datapins
FDIAG_STAT_IN_PROGRESS(7): test #6 RBM SSRAM Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #7 RBM SSRAM Datapins Memory
FDIAG_STAT_IN_PROGRESS(7): test #8 TBM SDRAM Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #9 TBM SDRAM Datapins
FDIAG_STAT_IN_PROGRESS(7): test #10 TBM SSRAM Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #11 TBM SSRAM Datapins Memory
FDIAG_STAT_IN_PROGRESS(7): test #12 PSA TLU SDRAM Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #13 PSA TLU SDRAM Datapins
FDIAG_STAT_IN_PROGRESS(7): test #14 PSA PLU SDRAM Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #15 PSA PLU SDRAM Datapins
FDIAG_STAT_IN_PROGRESS(7): test #16 PSA SRAM Marching Pattern
FDIAG_STAT_IN_PROGRESS(7): test #17 PSA SRAM Datapins
FDIAG_STAT_IN_PROGRESS(7): test #18 To Fabric SOP FIFO SRAM Memory
FDIAG_STAT_IN_PROGRESS(7): test #19 From Fabric SOP FIFO SRAM Memory
FDIAG_STAT_IN_PROGRESS(7): test #20 RBM to SALSA Packet
FDIAG_STAT_IN_PROGRESS(7): test #21 TBM to SALSA Packet
FDIAG_STAT_IN_PROGRESS(7): test #22 RBM to TBM SLI Packet Loopback
FDIAG_STAT_IN_PROGRESS(7): test #23 TBM to PSA Packet - Framer
Loopback
FDIAG_STAT_IN_PROGRESS(7): test #24 TBM to TX SOP Packet
FDIAG_STAT_IN_PROGRESS(7): test #25 TBM to RX SOP Packet - 4302
Terminal Loopback
FDIAG_STAT_IN_PROGRESS(7): test #26 TBM to RX SOP Packet - Framer
System Bus Loop
FDIAG_STAT_IN_PROGRESS(7): test #27 RBM to TBM Fabric Packet Loopback
FDIAG_STAT_IN_PROGRESS(7): test #28 TBM to RBM Packet, RBM page
crossing
FDIAG_STAT_IN_PROGRESS(7): test #29 TBM to TX SOP Packet Simultaneous
FDIAG_STAT_IN_PROGRESS(7): test #30 TBM to PSA Multicast Packets -
Framer Loopbac
FD 7> Changed current_status to FDIAG_STAT_IDLE
Field Diagnostic ****PASSED**** for slot 7
Field Diag eeprom values: run 62 fail mode 0 (PASS) slot 7
last test failed was 0, error code 0
Shutting down diags in slot 7
Displaying Previous Test Results
The following example shows the previous test results of a line card in slot 7. Diagnostics had been run 64 times on this line card. Because the board PASSED the last field diagnostics session, the fail mode was 0, and the last test that failed was 0.
Field Diag eeprom values: run 64 fail mode 0 (PASS) slot 7
last test failed was 0, error code 0