Field Notice: FN63916 - Some Cisco Aironet 1530, 1550, 1600, 1700, 2600, 2700, 3500, 3600 and 3700 Series Might Be Unable to Join a Wireless LAN Controller - Software Upgrade Recommended

Some Wireless Access Points (APs) manufactured between August 2014 and October 2014 might have an incorrectly programmed SHA-2 certificate. 

The affected product families are:

• Cisco Aironet 1530 Series
• Cisco Aironet 1550 Series
• Cisco Aironet 1600 Series
• Cisco Aironet 1700 Series
• Cisco Aironet 2600 Series
• Cisco Aironet 2700 Series
• Cisco Aironet 3500 Series
• Cisco Aironet 3600 Series
• Cisco Aironet 3700 Series

Issue 1

After you upgrade a Wireless LAN Controller (WLC) to software version 8.0.100.0 or 3.6.0E

AND

after the Wireless APs download the new software version, any Wireless AP with an incorrectly programmed SHA-2 certificate disconnects from the WLC and is not able to rejoin the WLC if the WLC has a SHA-2 certificate.

Issue 2

Any new Wireless AP with software version 8.0.100.0 and with an incorrectly programmed SHA-2 certificate fails to validate the image downloaded from the WLC. The result is that the AP is unable to establish a connection to a WLC with version 8.0.100.0 software.

If the AP has an incorrectly programmed SHA-2 certificate and the WLC has version 8.0.100.0 or 3.6.0E, the likelihood of this issue being observed is 100%.

Between August and October 2014, a manufacturing change was added to support SHA-2 certificates. In the certificate chain transition, some APs were manufactured with incorrect certificate information. Prior to this change, the APs only had a SHA-1 device ID certificate. After the change the APs had both SHA-1 and SHA-2, but the SHA-2 was incorrectly programmed on the affected units.

The available fixed code ensures that the APs continue to function as APs that were manufactured prior to August 2014.

The affected APs are fully functional and equivalent to APs manufactured prior to August 2014.

In the future, Cisco will provide support for SHA-2 authentication between APs and more recently manufactured WLCs.

New Aironet APs with factory installed recovery Cisco IOS® are able to join the controller that runs software version 8.0.100.0 or 3.6.0E and download version 15.3(3)JA or 15.3(3)JN IOS. However after the AP reload, the APs are unable to join the controller. On the AP, logs similar to these are seen:

*Oct 16 12:39:06.231: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.
*Oct 16 13:14:56.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: ***.***.***.*** peer_port: 5246Peer certificate verification failed FFFFFFFF
*Oct 16 13:14:56.127: DTLS_CLIENT_ERROR: ../capwap/base_capwap/capwap/base_capwap_wtp_dtls.c:496 Certificate verified failed!
*Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to ***.***.***.***:5246
*Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to ***.***.***.***:5246

Another symptom of this issue is that the AP might be able to join the software version 8.0.100.0 controller, download a new Cisco IOS code, and boot up and join the controller correctly; however when it goes to upgrade to the newer 8.x code it gets stuck in a loop and fails the download.

*Nov 11 10:13:53.003: Currently running a Release Image
*Nov 11 10:13:53.027: Using SHA-2 signed certificate for image signing validation.
*Nov 11 10:13:53.091: Image signing certificate validation failed (FFFFFFFF).
*Nov 11 10:13:53.091: Failed to validate signature
*Nov 11 10:13:53.091: Digital Signature Failed Validation (flash:/update/ap3g2-k9w8-mx.v153_80mr.201410311616/final_hash)
*Nov 11 10:13:53.091: AP image integrity check FAILED
Aborting Image Download
Download image failed, notify controller!!! From:8.0.100.0 to 8.0.102.34, FailureCode:3
archive download: takes 339 seconds
*Nov 11 10:14:02.399: capwap_image_proc: problem extracting tar file

AireOS

In order to avoid this issue, if the WLC runs software version 7.6 or earlier and you have APs affected by this issue, do not upgrade to version 8.0.100.x train. Wait for the next Cisco Connection Online (CCO) release.

Workaround for AireOS

If the WLC has been upgraded to version 8.0.100.x and the APs are supported in AireOS 7.6, downgrade to this version.

Solution for AireOS

If the WLC has software version 7.6 or earlier, upgrade the WLC to version 8.0.120.0 or later.

If the WLC has software version 8.0.100.x, follow these steps:

1. Upgrade the WLC to software version 8.0.104.0:
   • All controllers
   • WISM2
2. Allow all APs to join the WLC and upgrade to software version 8.0.104.0.
3. Upgrade the WLC to software version 8.0.120.0 or later.

Note: Step 2 is required to push the 8.0.104.0 special software version onto the APs in order to allow all future upgrades.

Cisco IOS-XE

In order to avoid this issue, if the WLC has software version 3.3.x or earlier and you have APs affected by this issue, do not upgrade to version 3.6.0E.

Workaround for Cisco IOS-XE

If the WLC has been upgraded to version 3.6.0E and APs are supported in Cisco IOS-XE Version 3.3.x, downgrade to this version.

Solution for Cisco IOS-XE

If the WLC has software version 3.6.0E, follow these steps:

1. Upgrade to version 3.6.1 or 3.7.0 or later.
2. Enter the wireless security certificate force-sha1-cert command from the prompt.

From the AP CLI, enter the show version command and look for the "Top Assembly Serial Number". An example of a Top Assembly Serial Number is FTX1613GJGA.

If the AP is joined to an AireOS controller:

• From the CLI, enter the show ap inventory APNAME command.
• From the GUI, select Wireless > All APs > APNAME > Inventory in order to view the serial number.

If the AP is joined to a Cisco IOS-XE controller:

• From the controller CLI , enter the show ap name APNAME inventory command and look for the "Cisco AP" serial number.
• From the GUI, select Configuration > Wireless > Access Points > All APs > APNAME > Inventory in order to view the serial number.

Alternately, the serial number can be found on the back/bottom of the AP:

Confirm that your serial number is affected with the Serial Number Validation Tool.