Field Notice: FN - 62879 - OTBU Database Corruption with ONS 15454 Releases 7.x (7.00, 7.01, 7.02, 7.03, 7.04, 7.20, 7.21, 7.22, 7.23)

Available Languages

Updated:August 10, 2009

Document ID:FN62879

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Revised August 10, 2009

October 29, 2007

NOTICE:

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Date	Comment
1.1	10-AUG-2009	Replaced all verbiage under Problem Description section; and added EXAMPLE, steps, and screenshot under the Background section.
1.0	29-OCT-2007	Initial Public Release

Products Affected

Products Affected	Comments
15454-ONS - R7.X.X	See Title

Problem Description

The following issue was introduced in release 7.0 and exists only in the 7.xx releases.

The database can become corrupted on an ONS 15454 when provisioning, or pre-provisioning the AIC-I cards and AEP.

This issue affects any ONS 15454 that supports the AIC-I and Alarm Expansion Panel (AEP).

Note: A corrupted database can lead to a complete outage of the node.

Background

The AIC-I card and the AEP on the ONS 15454 provide direct external alarm inputs, and external control output alarm contacts, routed through the backplane to wire-wrap pins, accessible from the back of the shelf. Under certain scenarios, provisioning of the AIC-I card can cause the database to become corrupt. The database load process on the ONS 15454 occurs during a TCC reboot, TCC protection switch, software activation, or database restore.

If a database has a specific AIC-I provisioning, a node may actually remain operational until there is a TCC reset. Upon reset, the existing database is read in by the software, and the error is detected, causing the corrupt database to be deleted from the TCC flash.

A database backup may contain the erroneous provisioning, and upon restore there is a TCC reset, which leads to the same set of events described above. The database backup contains the erroneous entries because it was taken after the erroneous provisioning and prior to a TCC reset.

EXAMPLE:

The AICI provisioning state that leads to the database corruption. Is as follows:

Notice: Steps 3 and 4 cannot be combined, or re-ordered.

1. Have an AICI card installed or pre-provisioned (it is not necessary to have the hardware physically installed).

2. Enter the card view. Select the "Provision" tab, then select the "Card" tab.

3. Select the "External Control" radio button, then click on the "Apply" button.

4. Check the "Add Extension" check box, then click on the "Apply" button.

The following screenshot shows the upper left corner of the screen.

Problem Symptoms

In the 7.x releases listed above, when a database is loaded containing this corruption, the load process fails, causing the corrupt database to be deleted from the TCC flash memory. If this issue occurs, you will see a loss of all provisioning after a TCC switch or reset.

Workaround/Solution

To detect this issue on a Network Element (NE), check that the AIC-I card is provisioned with the "Add extension" option checked, and that the card is provisioned as "External control". Avoid provisioning the AIC-I card with "External Control" first, and then selecting "Add extension".

If this provisioning exists, simply delete the AIC-I card and re-enter it. Once the AIC-I has cleared its restart, and a grace period is taken to allow the database to be written to flash memory (30 seconds), a new backup can be made.

Do not attempt to undo the AIC-I provisioning as this may cause an autonomous TCC reset, and lead to database deletion.

Releases 8.0 and above contain the fix for this issue.

DDTS

To follow the bug ID link below and see detailed bug information, you must be a registered user and you must be logged in.

DDTS	Description
CSCse41969 (registered customers only)	AIC-I failed to retrieve AEP info in 7.0

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Cisco Notification Services (CNS) - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.