Resolve ASR 5500 Series "FanFailed" Error Condition

Introduction

This document describes the problem and solution related to fan failure errors reported by Simple Network Management Protocol (SNMP) traps, seen on the Cisco Aggregation Services Router (ASR) 5500 platform. This normally happens, though rare, in normal operation of the platform.

Problem

In some cases the FanFailed alarm is seen on the ASR 5500 platform in regards to fan numbers 1 and 2 as shown here:

Tue Apr 01 16:07:07 2014 Internal trap notification 22 (FanFailed)
 fan number 2 fan status Communication Error
Tue Apr 01 16:07:27 2014 Internal trap notification 22 (FanFailed)
 fan number 1 fan status Communication Error

The lower rear and lower front fans currently report an Unknown state, and the general temperature of the node cards appears to be OK at the moment.

[local]ASR5500# show fan
Tuesday April 01 16:47:04 BST 2014
Lower Rear Fan Tray:
  State:       Unknown
  Speed:       0%
  Temperature: Unknown

Upper Rear Fan Tray:
  State:       Normal
  Speed:       50%
  Temperature: 27C

Lower Front Fan Tray:
  State:       Unknown
  Speed:       0%
  Temperature: Unknown

Upper Front Fan Tray:
  State:       Normal
  Speed:       50%
  Temperature: 27C

Solution

This issue is usually resolved when you reseat the fan modules. Complete these steps in order to verify that the status is normal:

1. Slide out the fan module. Wait two to three minutes and then slide it back into the slot.
2. Enter this command in order to verify that the fans are in the normal state:

   [local]ASR5500# show fans verbose
   Thursday May 28 19:16:54 EDT 2015
    Location State Speed Temp Version WD Tach Current
   ----------- ----------------------- ----- --------
   Lower Rear Normal 50% 24C 0.2.5 E -- --
    Fan 00 Normal -- -- -- -- 3280 386mA
    Fan 01 Normal -- -- -- -- 3280 406mA
    Fan 02 Normal -- -- -- -- 3320 416mA
    Fan 03 Normal -- -- -- -- 3240 396mA
    Fan 04 Normal -- -- -- -- 3280 440mA
    Fan 05 Normal -- -- -- -- 3280 430mA
    Fan 06 Normal -- -- -- -- 3240 460mA
    Fan 07 Normal -- -- -- -- 3280 493mA
    Fan 08 Normal -- -- -- -- 3240 470mA
   
   Upper Rear Normal 50% 39C 0.2.5 E -- --
    Fan 00 Normal -- -- -- -- 3240 430mA
    Fan 01 Normal -- -- -- -- 3240 416mA
    Fan 02 Normal -- -- -- -- 3280 416mA
    Fan 03 Normal -- -- -- -- 3320 460mA
    Fan 04 Normal -- -- -- -- 3280 416mA
    Fan 05 Normal -- -- -- -- 3320 416mA
    Fan 06 Normal -- -- -- -- 3240 430mA
    Fan 07 Normal -- -- -- -- 3280 430mA
    Fan 08 Normal -- -- -- -- 3320 430mA
   
   Lower Front Normal 50% 23C 0.2.5 E -- --
    Fan 09 Normal -- -- -- -- 2520 146mA
    Fan 10 Normal -- -- -- -- 2520 146mA
    Fan 11 Normal -- -- -- -- 2400 146mA
   
   Upper Front Normal 50% 35C 0.2.5 E -- --
    Fan 09 Normal -- -- -- -- 2400 146mA
    Fan 10 Normal -- -- -- -- 2520 146mA
    Fan 11 Normal -- -- -- -- 2400 146mA

3. Enter this command in order to verify that the fan modules run the correct firmware version:

   [local]ASR5500# show hardware version fans 
   Thursday May 28 19:16:54 EDT 2015
   Fan Tray Controller Version 
   Upper Fan Tray: 0.2.5
   Lower Fan Tray: 0.2.5

4. Enter this command in order to verify that the SNMP trap errors were cleared and that there are no errors after the fan module was reseated:

   [local]ASR5500# show snmp trap hist ver | grep -i fan
   Saturday February 07 08:22:32 UTC 2015
   Sat Feb 07 07:56:38 2015 Internal trap notification 1509 (CiscoFruFanStatusChanged)
    FRU entity Fan tray fan status changed to Communication Error  at Lower Front
   Sat Feb 07 07:56:38 2015 Internal trap notification 22 (FanFailed) fan number 1
    fan status Communication Error
   Sat Feb 07 07:56:38 2015 Internal trap notification 1503 (EntStateOperDisabled)
    Fan Tray(Lower Front) Admin state:"Unknown", Alarm severity:"Major"
   Sat Feb 07 07:56:58 2015 Internal trap notification 1509 (CiscoFruFanStatusChanged)
    FRU entity Fan tray fan status changed to Normal at Lower Front
   Sat Feb 07 08:09:06 2015 Internal trap notification 1509 (CiscoFruFanStatusChanged)
    FRU entity Fan tray fan status changed to Communication Error  at Lower Front
   Sat Feb 07 08:09:06 2015 Internal trap notification 22 (FanFailed) fan number 1
    fan status Communication Error
   Sat Feb 07 08:09:06 2015 Internal trap notification 1503 (EntStateOperDisabled)
    Fan Tray(Lower Front) Admin state:"Unknown", Alarm severity:"Major"
   Sat Feb 07 08:10:26 2015 Internal trap notification 1509 (CiscoFruFanStatusChanged)
    FRU entity Fan tray fan status changed to Normal at Lower Rear
   Sat Feb 07 08:10:26 2015 Internal trap notification 1509 (CiscoFruFanStatusChanged)
    FRU entity Fan tray fan status changed to Normal at Lower Front

Related Information

• ASR5000 SGSN Administration Guide - Cisco Systems
• Technical Support & Documentation - Cisco Systems