Field Notice: FN74095 - Cisco Nexus N9K-C93108TC-FX3P and N9K-C93108TC-FX3H NX-OS Standalone Interface down after reload: EPLD - BIOS/Firmware Upgrade Recommended
Available Languages
Notice
THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Products Affected
| Affected Software Product | Affected Release | Affected Release Number | Comments |
|---|---|---|---|
| NX-OS System Software | 10 | 10.1(1), 10.1(2), 10.1(2t), 10.1(3), 10.2(1), 10.2(1q), 10.2(3), 10.2(3t), 10.2(3u), 10.2(3v), 10.2(4), 10.2(5), 10.2(6), 10.3(1), 10.3(2), 10.3(3), 10.3(3o), 10.3(3p), 10.3(3w), 10.3(4), 10.3(4a), 10.4(1) |
10.x(X): For ALL releases: Device is affected only if EPLD MI FPGA version is 0x10 and lower: Example of EPLD MI FPGA version verification: n9k# show version module 1 epld |
| NX-OS System Software | 9 | 9.3(1), 9.3(10), 9.3(11), 9.3(12), 9.3(1z), 9.3(2), 9.3(3), 9.3(4), 9.3(5), 9.3(5w), 9.3(6), 9.3(7), 9.3(7a), 9.3(7k), 9.3(8), 9.3(9) |
9.3(X): For ALL releases: Device is affected only if EPLD MI FPGA version is 0x10 and lower: Example of EPLD MI FPGA version verification: n9k# show version module 1 epld |
Defect Information
| Defect ID | Headline |
| CSCwh84282 | After reload of 93108TC-FX3P some copper interfaces might not come up |
Problem Description
After a switch reload, Cisco Nexus N9K-C93108TC-FX3P and N9K-C93108TC-FX3H Switches might fail to bring link up on some front panel ports.
This problem is related to RJ45 front-panel interfaces Ethernet 1/1-48 only. Optical ports Ethernet 1/49-54 and MGMT0 port are not affected.
Until a fix is implemented, the link on affected ports will stay down. Further reboots might increase the number of impacted ports or change which ports are impacted.
Background
In Cisco Nexus N9K-C93108TC-FX3P and N9K-C93108TC-FX3H Switches, front panel RJ45 ports are connected to six separate octal PHYs. An octal PHY is a component inside the switch that is used to connect front panel ports to the main forwarding ASIC. The PHYs are mapped to the front ports in this sequence: First PHY front port:1-8; second PHY front port: 9-16; and so on.
Due to CSCwh84282, a reboot of the switch causes communication between Cisco NX-OS Software and PHY to become impaired. As a result, Cisco NX-OS Software is unable to bring link up on some ports that are connected to the affected PHY.
During the issue state, impacted ports will be down.
Successive reboots can cause the issue to move to other ports in the same PHY.
Problem Symptom
After a reload, some RJ45 interfaces on Cisco Nexus N9K-C93108TC-FX3P and N9K-C93108TC-FX3H Switches might not come up, even when connected.
Note: This issue can be triggered by a switch reload that occurs for any reason, including a software upgrade, software reset, system crash, or the power being turned up or down.
The problem is related only to the front-panel interfaces Ethernet 1/1-48. Optical ports Ethernet 1/49-54 and MGMT0 port are not affected.
Workaround/Solution
There is no workaround available to recover the impacted ports. To prevent the issue, do not reload the switch.
Updating the electronic programmable logic devices (EPLD) proactively will also prevent the issue from occurring.
Solution
The solution is to upgrade the EPLD to Cisco NX-OS System software release independent EPLD n9000-epld.CSCwh84282.img.
Note: While available to download under the 10.2(6)M release tree, the EPLD n9000-epld.CSCwh84282.img image is software release agnostic and can be installed on devices with Cisco NX-OS System software releases other than Release 10.2(6). This EPLD is supported on Cisco NX-OS Software releases 9.3.x and 10.2.x and higher (such as Release 10.3.x, Release 10.4.x and so on).
The following NX-OS EPLD updates that were released with Cisco NX-OS System software versions 9.3(13) and 10.4(2)F already contain the fix for this problem:
Note: This issue will not be resolved by a Cisco NX-OS Software update; the EPLD must be upgraded. (An EPLD upgrade requires a reload.)
To install the EPLD, follow standard EPLD upgrade procedure using these steps:
- Copy the EPLD image to bootflash.
- Install epld bootflash:n9000-epld.CSCwh84282.img module 1
After the EPLD has been upgraded, affected interfaces will be recovered.
To verify whether the correct EPLD was installed, use the CLI command show version module 1 epld.
In the command output, MI FPGA versions must be 0x11 and higher.
The following example shows the output of a fixed MI FPGA version:
n9k# show version module 1 epld
EPLD Device Version
---------------------------------------
MI FPGA 0x11
IO FPGA 0x8
How to Identify Affected Products
To identify whether ports are affected by this problem, log in to the Cisco Nexus N9K-C93108TC-FX3P switch and take the following steps:
i) Verify version of the MI FPGA. All MI FPGA versions 0x10 and lower are affected, as shown in the following example:
n9k# show version module 1 epld
EPLD Device Version
---------------------------------------
MI FPGA 0x10
IO FPGA 0x8
ii) Check Firmware Version and Device ID values of the port. When a port is affected, Device ID will have all values set to 0x00000000 and Firmware Version set to 0.00.0, as seen in the following example:
n9k# attach module 1
module-1# show hardware internal tah 10GBaseT fp-port<front panel port number>
Firmware Version 1e.20 1e.c885: 0.00.0
Device ID 01.0002 | 0003 : (0x00000000)
Device ID 03.0002 | 0003 : (0x00000000)
Device ID 07.0002 | 0003 : (0x00000000)
Device ID 1E.0002 | 0003 : (0x00000000)
Super Isolate bit 1.9.0 : (0x0000) Clear
System link status 04.E812: (0x0000) Tx Ready: N Rx Status:Down
Line link status 04.E800: (0x0000) line status:Down
Line link speed 7.C800 : (0x0000)
speed : 10BASE-T
Duplex : Half
Line link status 7.C810 : (0x0000)
Line Status:
Tx PAUSE : Disabled
Rx PAUSE : Disabled
Dup Link Partner AutoNeg Ability :
MDI/MDX : MDI
Conn State : Reserved/Invalid
data >> 9 = 0x0000
10G Auto Neg 07.0000: (0x0000) AN Off, Restart Off
10G Advertisement 07.0020: (0x0000) Off
1G Auto Neg 07.C400: (0x0000) 1G Advertisement Off
1G Auto Neg 07.10 : (0x0000) 100M Advertisement Off
NxtPageAbility Off ExtNxtPageAbility Off
Line Partner Ability 07.13 : (0x0000)
10G AutoNeg Status 07.21 : (0x0000)
Parallel detect 07.E410: (0x0)
A port is affected if both of the following conditions are met:
a. EPLD MI FPGA version must be 0x10 or lower.
b. Internal port status returns the following status:
- All Device ID values are 0x00000000
- Firmware Version is not populated and set to 0.00.0
Example of a port that is affected:
n9k# show inter status | grep Eth1/20
Eth1/20 -- notconnec routed auto auto 10g
Verification of internal port state: When a port is affected, Device ID will have all values set to 0x00000000 and Firmware Version set to 0.00.0, as shown in the following example:
n9k# attach module 1
module-1# show hardware internal tah 10GBaseT fp-port 20
Firmware Version 1e.20 1e.c885: 0.00.0
Device ID 01.0002 | 0003 : (0x00000000)
Device ID 03.0002 | 0003 : (0x00000000)
Device ID 07.0002 | 0003 : (0x00000000)
Device ID 1E.0002 | 0003 : (0x00000000)
Super Isolate bit 1.9.0 : (0x0000) Clear
System link status 04.E812: (0x0000) Tx Ready: N Rx Status:Down
Line link status 04.E800: (0x0000) line status:Down
Line link speed 7.C800 : (0x0000)
speed : 10BASE-T
Duplex : Half
Line link status 7.C810 : (0x0000)
Line Status:
Tx PAUSE : Disabled
Rx PAUSE : Disabled
Dup Link Partner AutoNeg Ability :
MDI/MDX : MDI
Conn State : Reserved/Invalid
data >> 9 = 0x0000
10G Auto Neg 07.0000: (0x0000) AN Off, Restart Off
10G Advertisement 07.0020: (0x0000) Off
1G Auto Neg 07.C400: (0x0000) 1G Advertisement Off
1G Auto Neg 07.10 : (0x0000) 100M Advertisement Off
NxtPageAbility Off ExtNxtPageAbility Off
Line Partner Ability 07.13 : (0x0000)
10G AutoNeg Status 07.21 : (0x0000)
Parallel detect 07.E410: (0x0)
Example of port that is NOT affected:
n9k# show inter status | grep Eth1/20
Eth1/20 -- notconnec routed auto auto 10g
Verification of internal port state: Device ID is populated and NOT 0x000000 and Firmware Version is populated and NOT 0.00.0. Therefore, the port is not affected, as shown in the following example:
n9k# attach module 1
module-1# show hardware internal tah 10GBaseT fp-port 20
Firmware Version 1e.20 1e.c885: 5.0X.X
Device ID 01.0002 | 0003 : (0x31c31cb2)
Device ID 03.0002 | 0003 : (0x31c31cb2)
Device ID 07.0002 | 0003 : (0x31c31cb2)
Device ID 1E.0002 | 0003 : (0x31c331c3)
Super Isolate bit 1.9.0 : (0x0000) Clear
System link status 04.E812: (0x0048) Tx Ready: N Rx Status:Down
Line link status 04.E800: (0x0000) line status:Down
Line link speed 7.C800 : (0x0007)
speed : 10GBASE-T
Duplex : Full
Line link status 7.C810 : (0x0414)
Line Status:
Tx PAUSE : Disabled
Rx PAUSE : Disabled
Dup Link Partner AutoNeg Ability : Link Partner Capable of AuntoNeg
MDI/MDX : MDI
Conn State : AutoNeg
data >> 9 = 0x0002
10G Auto Neg 07.0000: (0x3000) AN On, Restart Off
10G Advertisement 07.0020: (0x1181) On
1G Auto Neg 07.C400: (0xcc40) 1G Advertisement On
1G Auto Neg 07.10 : (0x9181) 100M Advertisement On
NxtPageAbility On ExtNxtPageAbility On
Line Partner Ability 07.13 : (0x0000)
10G AutoNeg Status 07.21 : (0x0000)
Parallel detect 07.E410: (0x0)
Revision History
| Version | Description | Section | Date |
| 1.0 | Initial Release | — | 2024-JAN-29 |
For More Information
For further assistance or for more information about this field notice, contact the Cisco Technical Assistance Center (TAC) using one of the following methods:
Receive Email Notification About New Field Notices
To receive email updates about Field Notices (reliability and safety issues), Security Advisories (network security issues), and end-of-life announcements for specific Cisco products, set up a profile in My Notifications
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)