Cisco GSR 12008 Chassis
This field notice addresses two separate issues on the Cisco GSR 12008. These issues are being grouped together in one field notice so that customers have the option of combining any activities that are needed into a single maintenance action rather than two separate ones.
Issue 1: GSR 12008 Cooling
If the ambient (room) air temperature is above 40�‹ C (104�‹ F), some existing Cisco GSR 12008s may not be able to adequately cool themselves. This is true regardless of the Cisco GSR 12008 configuration (types and numbers of cards, AC or DC power, and so on). Cisco normally certifies its equipment up to 50�‹ C (122�‹ F). Ambient air temperature above 40�‹ C is most likely be associated with an air conditioning equipment failure affecting the entire room or building. If ambient air temperature remains below 40�‹ C, the Cisco GSR 12008 is able to adequately cools itself regardless of configuration.
Issue 2: CSC-8
There is a potential problem with CSC-8 cards in the Cisco GSR 12008's main fan tray. CSC-8 cards provide power to the Cisco GSR 12008 main fan tray. This potential problem can cause a loss of power to the fan tray and cause the Cisco GSR 12008 to overheat and eventually shutdown. The problem symptoms and the risk of the Cisco GSR 12008 overheating differ depending on whether the Cisco GSR 12008 is equipped with redundant or non-redundant CSC-8 cards.
Case #1: Single CSC-8 in failure (or redundant CSC-8s with both in failure):
In Cisco GSR 12008s equipped with a single CSC-8 card that has failed (or redundant CSC-8 cards that have both failed), power to the Cisco GSR 12008's main fan tray will be lost and the Cisco GSR 12008 will begin to overheat. This could ultimately result in the Cisco GSR 12008 shutting itself down because of an overtemperature condition.
Case #2: Redundant CSC-8 Cards with one in failure:
In Cisco GSR 12008s equipped with redundant CSC-8 cards, both CSC-8 cards load share (as shown by the red arrows in the illustration below) to provide power to the main fans using a "diode-OR" circuit:
Note:?Cisco GSR 12008 software will not switch from an active CSC-8 card to a standby if the DC-DC converter has failed because of the location of the fan voltage sense line (after the "diode-OR"), and because a failure of the DC-DC converter does not affect other functions on the CSC-8 cards. This can lead to the unusual configuration shown in the illustration below where the standby CSC-8 card is actually providing power to the fans (as shown by the red arrow):
This could be very important during maintenance activities (such as upgrading the CSC-8 cards as described in this field notice), because removing a standby CSC-8 card that is providing power to the fans could cause the fans to stop unexpectedly and risk overheating the Cisco GSR 12008. The node will not overheat if a working CSC-8 card is immediately re-installed in the GSR 12008
Note:?Because of the location of the fan voltage sense line, it is not possible to tell which CSC-8 card is bad, or even if a CSC-8 card is bad using only the voltage sense displayed in the show env fans output. To determine if and which CSC-8 card is bad, use the technique described in the Problem Symptoms section.
Issue 1: Cisco GSR 12008 Cooling
The Cisco GSR 12008 was not originally designed to cool itself to 50�‹ C (122�‹ F).
Issue 2: CSC-8
There is a design problem in the DC-DC power supply circuitry on the CSC-8 card that provides power for the main fan tray. When the DC-DC converter fails, power is lost to the fans, but other CSC-8 functions are not affected.
Issue 1: Cisco GSR 12008 Cooling
If the Cisco GSR 12008 is in an environment with an ambient air temperature above 40�‹ C (104�‹ F), users may be made aware of this problem either proactively or through unsolicited alarm messages, both of which are described below.
Proactive
Users may proactively determine the situation by executing the show environment temp command. The command output contains inlet and hot temperature sensor readings. The "inlet" temperature corresponds to the room air temperature entering the GSR 12008. The "hot" temperature corresponds to the temperature of the air being exhausted from the GSR 12008 to the room. A portion of the output from this command is shown below:
Slot # Hot Sensor Inlet Sensor (deg C) (deg C) 0 52.5 43.5 7 64.0 48.5 16 50.0 46.0 17 56.0 49.0 18 45.0 44.5 19 45.0 43.5 20 45.0 41.5 24 NA 47.0 26 NA 48.0
Non-Proactive
If the inlet air temperature exceeds a preset threshold, the GSR 12008 may generate an unsolicited alarm on the console. An example of this alarm is shown below:
%GSR_ENV-2-WARNING: Slot 17 Inlet sensor temperature at 49 deg C > 42 deg C
Note:? The Cisco GSR 12008 will not generate an unsolicited alarm if the user has configured it no to display environmental alarms by executing the no env command. To re-activate the generation of alarms, use the env command.
To view the preset temperature thresholds, use the show environment table command. A portion of the output from this command is shown below:
Inlet Sensor Temperature Limits (deg C): Warning Critical Shutdown GRP/GLC (Slots 0-15) 35 40 52 CSC (Slots 16-17) 40 45 59 SFC (Slots 18-20) 37 42 54
Issue 2: CSC-8
Case #1: Single CSC-8 in failure (or redundant CSC-8s with both in failure)
In GSR 12008s equipped with a non-redundant CSC-8 card that has failed (or redundant CSC-8 cards that have both failed), users can check for this problem either proactively or through unsolicited alarm messages, both of which are described below.
Proactive
Users can proactively check for this problem by executing the show environment fans command, which displays the voltage going to the fans and the fans' RPM. A portion of the output from this command is shown below.
Slot # Fan Information 16 Voltage 2427V Speed fast: Main Fan 0 broken Main Fan 1 broken Main Fan 2 broken Main Fan 3 broken Main Fan 4 broken Main Fan 5 broken Power Supply Fan 0 broken Power Supply Fan 1 broken Power Supply Fan 2 broken Power Supply Fan 3 broken
Note:?The fan speed (in this case "fast"), is not an indication of the actual fan speed, but rather an indication of the fan speed setting (the speed at which the GSR 12008 would like to run the fans). The other information displayed is more indicative of proper fan operation or failure.
Note:?The voltage reading of "2427V" appears to be a software bug and should be interpreted as "0V". This value has been observed to revert to 0V after about 60 seconds on some GSR 12008s.
Non-Proactive
If the CSC-8 is not providing power to the main fans, the Cisco GSR 12008 may generate an unsolicited alarm on the console. An example of this is shown below.
%GSR_ENV-1-CRITICAL_FAN: Slot 17 Main fan tray, 6 fans broken; Chassis may overheat
Case #2: Redundant CSC-8 Cards with one in failure
In Cisco GSR 12008s equipped with redundant CSC-8 cards with one in failure, users can check for this problem proactively. The Cisco GSR 12008 will not, however, notify the user with a console alarm because the both the fan voltage sense line and the fan speed sense lines will appear normal .
Proactive
It is possible to proactively check for a failed CSC-8 card in a redundant pair, but the procedure should be performed only by a qualified engineer. Please contact Cisco's Technical Assistance Center (TAC) for more information and refer to this Field Notice.
Issue 1: GSR 12008 Cooling
Most Cisco GSR 12008s are installed in computer rooms with air conditioning where this will not be a problem. For customers who wish to guarantee adequate cooling above 40�‹ C (104�‹ F) up to 50�‹ C (122�‹ F), a cooling upgrade kit is available for purchase through Cisco Marketplace under the product ID "GSR8-FLTASM-UPG=".
Issue 2: CSC-8?
The part number of the CSC-8 card was rolled to incorporate this change:
Old CSC-8 part number: |
800-03030-01 |
New CSC-8 part number: |
800-03030-02 |
Any CSC-8 with an 800-03030-01 part number at any revision may exhibit the problem described in this field notice. Any CSC-8 with an 800-03030-02 part number at any revision will not exhibit this problem. To see the part number from the console, use the show diag command. Sample output is shown below (with the old part number):
SLOT 17 (CSC 1 ): Clock Scheduler Card(8) MAIN: type 20, 800-3030-01 rev C0 dev 16777215 HW config: 0xFF SW key: FF-FF-FF PCA: 73-2607-03 rev B0 ver 3 HW version 1.0 S/N CAB02510BQT MBUS: MBUS Agent (1) 73-2146-07 rev B0 dev 0 HW version 1.2 S/N CAB025007W9 Test hist: 0xFF RMA#: FF-FF-FF RMA hist: 0xFF DIAG: Test count: 0xFFFFFFFF Test results: 0xFFFFFFFF MBUS Agent Software version 01.35 (RAM) (ROM version is 01.33) Using CAN Bus A
Customers who wish to upgrade their CSC-8 cards from the -01 version to the -02 version can do so by submitting a return materials authorization (RMA) for the -01 units. These unites will be replaced with the -02 or higher (if applicable) version.
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods: