Cisco ONS 15600 Series

January 18, 2007

NOTICE:

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Products Affected

Problem Description

It has been found that a standing database out of sync (DBOSYNC) condition, coupled with an autonomous reset of the Standby TSC, followed closely by a reset of the Active TSC may cause the out of sync TSC to become active on default database and cause an outage.

Background

Given that the Active and Standby TSC databases are out of sync, the Standby TSC upon startup deletes the out of sync database in preparation to obtain the current database from the Active TSC. If, within the interval of time between the Standby TSC deleting the unknown database and obtaining a known database, the Active TSC resets, then the Standby TSC may become active on an empty database. In the process of becoming active, the TSC uses the system default values and messages this to the subtending cards, thereby causing an outage.

Problem Symptoms

A database out-of-sync condition accompanied with consecutive TSC resets resulting in an outage.

An indication that the system is using a default database is that one is unable to log into a node with any other account than the default CISCO15 Superuser with default password.

The sequence of events has always been preceded by simultaneous DBOSYNC and MEM-LOW or MEM-GONE.

Workaround/Solution

It is suggested that the alarm severity of DBOSYNC, MEM-LOW and MEM-GONE be raised to Critical. If any of these alarms present themselves on the 15600, the customer should take immediate corrective and preventative action.

There are established recovery procedures documented for both MEM-LOW/MEM-GONE and DBOSYNC conditions. The following procedures are alternative recommendations allowing for TAC participation. The recovery procedure recommendations are as follows:

If there is only MEM-LOW or MEM-GONE, do the following:

1. Take or ensure a current database backup of the active TSC via CTC. Also, verify DB is nonzero length on PC/UNIX file system.

2. Check for a DBOSYNC condition on the system. If identified, go to the recovery procedure below for handling both a DBOSYNC and a MEM-LOW/MEM-GONE condition.

3. Pull the Standby TSC, leave it out, and work with TAC to troubleshoot.

If there is only a DBOSYNC do the following:

1. Take or ensure a current database backup of the active TSC via CTC. Also, verify DB is nonzero length on PC/UNIX file system.

2. Check for a MEM-LOW or MEM-GONE condition on the system. If identified, go to the recovery procedure below for handling both a DBOSYNC and a MEM-LOW/GONE condition.

3. Using CTC or TL1, attempt a simple provisioning change and wait roughly 30 seconds and see if the DBOSYNC clears. Also, a CTC/TL1/telnet login will force a DB save and attempt a DB re-sync in 30 seconds.

4. If at this point this issue is not resolved, pull the Standby TSC, leave it out, and work with TAC to troubleshoot.

If there is both a DBOSYNC and a MEM-LOW/MEM-GONE, do the following:

Pull the Standby TSC, leave it out, and work with TAC to troubleshoot.

This issue will be corrected in Releases 6.24 (Supported by CTM R7.x and higher), 7.22, and 8.0. Upgrading to one of these releases as they become available is highly recommended. Until an upgrade can be performed use the steps indicated above.

DDTS

To follow the bug ID link below and see detailed bug information, you must be a registered user and you must be logged in.

Revision History

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email
• By telephone

Receive Email Notification For New Field Notices

Product Alert Tool - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.