Guest

Cisco ONS 15300 Series

Field Notice: ONS15327 Bus Termination on Some Older Revision XTC14, XTC28, E10/100-T, OC12, and OC48 Cards


Revised March 6, 2004

December 13, 2002


Products Affected

Product

Top Assembly

Comments

Part Number

Rev.

15327-XTC-14

800-07109-01

A0

Affected Card Population

15327-XTC-14

800-07109-01

B0

Affected Card Population

15327-XTC-14

800-07109-01

C0

Affected Card Population

15327-XTC-14

800-07109-02

A0

Affected Card Population

15327-XTC-14

800-07109-02

B0

Affected Card Population

15327-XTC-14

800-07109-02

C0

Affected Card Population

15327-XTC-28-3

800-07661-01

A0

Affected Card Population

Upgrade available May 30, 2003

15327-XTC-28-3

800-07661-01

B0

Affected Card Population

Upgrade available May 30, 2003

15327-XTC-28-3

800-07661-01

C0

Affected Card Population

Upgrade available May 30, 2003

15327-XTC-28-3

800-07661-01

D0

Affected Card Population

Upgrade available May 30, 2003

15327-XTC-28-3

800-07661-02

A0

Affected Card Population

Upgrade available May 30, 2003

15327-XTC-28-3

800-07661-02

B0

Affected Card Population

Upgrade available May 30, 2003

15327-XTC-28-3

800-07661-02

C0

Affected Card Population

Upgrade available May 30, 2003

15327-XTC-28-3

800-07661-02

D0

Affected Card Population

Upgrade available May 30, 2003

15327-E10/100T-4

800-7019-01

A0

Affected Card Population

15327-E10/100T-4

800-7019-01

B0

Affected Card Population

15327-E10/100T-4

800-7019-01

C0

Affected Card Population

15327-E10/100T-4

800-7019-02

A0

Affected Card Population

15327-E10/100T-4

800-7019-02

B0

Affected Card Population

15327-E10/100T-4

800-7019-02

C0

Affected Card Population

15327-OC12-1-IR

800-07273-01

A0

Affected Card Population

Upgrade available May 30, 2003

15327-OC12-1-IR

800-07273-01

B0

Affected Card Population

Upgrade available May 30, 2003

15327-OC12-1-IR

800-07273-01

C0

Affected Card Population

Upgrade available May 30, 2003

15327-OC12-1-IR

800-07273-02

A0

Affected Card Population

Upgrade available May 30, 2003

15327-OC12-1-IR

800-07273-02

B0

Affected Card Population

Upgrade available May 30, 2003

15327-OC12-1-IR

800-07273-02

C0

Affected Card Population

Upgrade available May 30, 2003

15327-OC48-1-IR

800-07662-01

A0

Affected Card Population

15327-OC48-1-IR

800-07662-01

B0

Affected Card Population

15327-OC48-1-IR

800-07662-01

C0

Affected Card Population

15327-OC48-1-IR

800-07662-02

A0

Affected Card Population

15327-OC48-1-IR

800-07662-02

B0

Affected Card Population

15327-OC48-1-IR

800-07662-02

C0

Affected Card Population

15327-OC48-1-IR

800-07662-02

D0

Affected Card Population

Problem Description

This field notice is issued to alert customers about initialization/boot issues that can occur on some older version cards of the ONS 15327 product. It is intended to both alert customers about suspected cards that may exhibit certain symptoms and provide a resolution when these symptoms are observed.

These cards include various older versions of the XTC-14, XTC-28-3, E10/100T-4, OC-12IR, and OC-48IR. Please refer to the Products Affected section in this field notice for a complete list of specific affected card versions. These older version cards do not have the proper SCL bus line terminations implemented and thus could exhibit initialization/bootup issues as indicated in the Problem Symptoms section.

It should be noted, however, that this issue is not service-affecting. Symptoms can occur with initial system startup with initial firmware download from the working XTC to the standby XTC, or from the working XTC to the affected line card(s), as indicated in the descriptions in the Problem Symptoms section.

For units in service, the upgrade procedures (provided in the section Symptomatic Details and Upgrade Procedures) may be followed to minimize the impact on network traffic, generally with only a 50-ms XTC switching time in protected XTC systems. However, when an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

This does not impact the ONS15454 or any other Cisco products.

Note: Replacements of the XTC-28-3 and OC-12 cards will be Industrial Temperature (I-Temp) rated. I-Temp operation is -40°C tp +65°C.

Note: XTC-14, E10/100T, and OC-48 cards will be repaired for proper bus termination.

Background

Some ONS15327 cards with earlier versions did not have a proper SCL line termination implemented on them. In the Products Affected section of this field notice, the Top Assembly Number version -01 has the non-Class B faceplates. Version -02 has the Class B faceplates.

Problem Symptoms

Symptoms can occur with initial system startup with initial firmware download from the working XTC to the standby XTC or from working XTC to the affected line card(s) during initial system initialization/bootup or initialization of a line card into a working system. If this download occurs properly, the system can be placed into service. Symptoms can also occur anytime a new card is initialized in a system.

General XTC-to-XTC firmware download symptoms include:

  • General Symptom 1: In a protected/redundant XTC configuration, if an XTC with the proper termination is paired with an XTC without proper termination, it may boot up only, but may not become active.

  • General Symptom 2: In a protected/redundant XTC configuration, if an XTC without proper termination is paired with an XTC without proper termination, the XTCs may continually attempt to initialize each other, back and forth, and neither can become active.

  • In a protected/redundant XTC configuration, if an XTC with proper termination is paired with another XTC with proper termination, there should not be any issues.

  • Systems with a single XTC may have potential symptoms if attempting an upgrade to a protected configuration.

XTC to OCn or Ethernet line card firmware download symptoms include:

  • During initialization, the firmware is downloaded from the XTC to the OCn or Ethernet line card. During this process, some line cards without proper termination may take longer than normal to download or may not download at all.

  • This may affect protected/redundant and unprotected/nonredundant XTC configurations. XTC revision has no impact on the bootup of the line cards, but instead the line card revision is the question.

Symptomatic Details and Upgrade Procedures

These XTC upgrade procedures may be followed in-service within normal 50-ms Automatic Protection Switch (APS) time for protected XTC systems. There is no in-service upgrade for unprotected XTC systems. Unprotected systems must be taken out of service for upgrade.

Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

  • 1.0-Two newer XTCs installed in a 15327

    • Both XTCs are operating normally. There is nothing to do for this scenario.

  • 2.0-One newer and one older XTC installed in a 15327

    • 2.1-Symptomatic case

      For the symptomatic case, only one XTC can see the SCL clock from the other XTC. The older XTC cannot see the SCL clock from the newer XTC. Therefore, if the older XTC is on standby, it resets the newer XTC and takes over the system. The older XTC becomes the active XTC. It cannot see the newer XTC and reports an equipment failure alarm for that XTC. The newer XTC boots and goes into standby. It sees the older XTC is alive and active and does not reset it. The system stays in this state as long as it is not disturbed. If the system gets power cycled or manually reset, it may go through at least one more reset until the older XTC can become the active XTC.

      The recommended upgrade procedure is:

      • Back up the provisioning database.

      • Verify that no software or database copy between the XTCs is in progress. (See 8.0-XTC Notes, below.)

      • Remove the older XTC. A switchover occurs and the newer XTC becomes active as soon as the older XTC is removed.

      • Verify that the newer XTC becomes active.

      • Install a new XTC to replace the older XTC.

      • Verify that the new XTC boots and becomes standby, and that software and/or the database copies to the new XTC.

      • Return the older XTC.

      Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

    • 2.2-Asymptomatic case

      For the asymptomatic case, both XTCs can see the other SCL clock, and the system behaves normally.

      The recommended upgrade procedure is:

      • Back up the provisioning database.

      • Verify that no software copy or database copy between the XTCs is in progress. (See 8.0-XTC Notes, below.)

      • Remove the older XTC. If the older XTC was standby, the newer XTC remains active. If the older XTC was active, a switchover occurs and the newer XTC becomes active as soon as the older XTC is removed.

      • Install a new XTC to replace the older XTC.

      • Verify that the new XTC boots and becomes standby, and that software and/or the database copies to the new XTC.

      • Return the older XTC.

      Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

  • 3.0-Two older XTCs installed in a 15327

    • 3.1-Symptomatic cases

      There are two possible symptomatic cases when there are two older XTCs installed in a 15327 chassis:

      • Symptomatic Case A-Neither XTC sees the SCL clock from the other XTC.

        In this case, the two older XTCs keep resetting each other over and over and the system never comes up. This condition is service-affecting. This case is also most likely to occur during a new install and turn up when there is no provisioning database to preserve and no services to maintain. Therefore, the two older XTCs can be pulled and replaced with two newer XTCs.

        The recommended upgrade procedure is:

        • Remove the older XTCs.

        • Install new XTCs to replace the older XTCs.

        • Verify that the new XTCs boot and that one XTC becomes standby and the other XTC becomes active.

        • If necessary, install the appropriate software version on the new XTCs.

          Note: This step is only necessary if the new XTCs have a software version that is not standard to the network.

        • Create the desired circuits and provisioning.

        • Return older XTCs.

      • Symptomatic Case B-Only one XTC can see the SCL clock from the other XTC.

        In this case, one XTC cannot detect the SCL clock from the other XTC. The XTC that cannot detect the SCL clock always resets the other XTC, if that XTC ever becomes active. An equipment failure alarm is generated for the XTC with the undetected SCL clock.

        The recommended upgrade procedure is:

        • Back up the provisioning database.

        • Verify that no software and/or database copy between XTCs is in progress. (See 8.0-XTC Notes, below.)

        • Remove the standby older XTC.

        • Install the first new XTC to replace the older XTC.

        • Verify that the first new XTC boots and becomes standby and that software and/or the database copies to the new XTC.

        • Remove the active older XTC. A switchover occurs and the first new XTC, previously installed, becomes active.

        • Install the second new XTC to replace the second older XTC.

        • Verify that the second new XTC boots and becomes standby and that software and/or the database copies from the first new XTC (active) to the second new XTC (standby).

        • Return both older XTCs.

        Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

    • 3.2-Asymptomatic case

      For the asymptomatic case, both XTCs can see the other SCL clock, and the system behaves normally.

      The recommended upgrade procedure is:

      • Back up the provisioning database.

      • Verify that no software copy or database copy between XTCs is in progress. (See 8.0-XTC Notes, below.)

      • Remove the standby older XTC.

      • Install the first new XTC to replace the older XTC.

      • Verify that the first new XTC boots and becomes standby, and that software and/or the database copies to the new XTC.

      • Remove the active older XTC. A switchover occurs and the first new XTC, previously installed, becomes active.

      • Install the second new XTC to replace the second older XTC.

      • Verify that the second new XTC boots and becomes standby, and that software and/or the database copies from the first new XTC (active) to the second new XTC (standby).

      • Return both older XTCs.

      Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

  • 4.0-Older XTC in an unprotected configuration in a 15327

    • 4.1-Intended unprotected operation

      In this situation, the 15327 is intended to operate with a single unprotected XTC. Since there is no second XTC in the 15327, there are no problems relating to XTC-to-XTC communications over the SCL bus. Therefore, this is a stable configuration, and there is no immediate need to upgrade from the older XTC to a newer XTC. However, should the system configuration be changed in the future to include a second XTC, the older XTC will need to be upgraded at that time using Procedure 4.2, below.

      • As indicated above, a single XTC system is stable since no problem exists with XTC-to-XTC communication.

      • If it is desired to upgrade unprotected systems and leave them as unprotected systems, then remove the system from service and replace the older XTC and any line card(s).

      • If the provisioning database is loaded on the XTC prior to installing it in the target 15327 chassis (such as in a lab environment), the service hit will be approximately 10 minutes.

      • If you are installing a new XTC without the desired provisioning database already loaded, the database must be restored after the XTC boots.

      Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

    • 4.2-Intended protected operation

      In this situation, the 15327 is originally intended to operate in protected mode with two XTCs installed. However, it is being operated with a single unprotected XTC because of system instability that resulted from using two old XTCs with improperly terminated SCL buses. In this case, both XTCs were symptomatic and could not work together in the same 15327. Since the second XTC was removed from the 15327, there are no longer problems relating to XTC-to-XTC communications over the SCL bus. This temporary unprotected configuration is stable.

      The recommended upgrade procedure is:

      • Obtain two newer XTCs.

      • If the nonutilized XTC was left installed in the 15327 (but not inserted into the backplane), remove it. Leave the active XTC operational at this time.

      • Back up the provisioning database from the installed XTC.

      • Install one newer XTC in the available slot and wait for it to boot and go standby.

      • Verify that software and the database copy from the active XTC to the standby XTC. Wait for this step to complete before proceeding. (See 8.0-XTC Notes, below.)

      • Remove the older XTC. A switchover occurs and the newer XTC becomes active as soon as the older XTC is removed.

      • Install the second new XTC to replace the older XTC.

      • Verify that the new XTC boots and becomes standby, and that software and/or the database copies to the new XTC.

      • Return both older XTCs.

      Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

  • 5.0-Combining hardware upgrade with 15327 software upgrade

    For the case in which it is desired to upgrade the 15327 hardware and software during the same maintenance window, this upgrade sequence is recommended to preserve the provisioning data and minimize system downtime:

    • Upgrade the older XTCs first. Follow the applicable recommended upgrade procedure that is outlined above.

    • Verify that the 15327 is stable and operating normally.

    • Upgrade the older line cards second.

    • Verify that all the line cards have completed their bootup sequence and are operating normally.

    • Upgrade the system software last.

    Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

  • 6.0-OCn or Ethernet line cards

    • During initialization, the firmware is downloaded from the XTC to the OCn or Ethernet line card. During this process, some line cards without proper termination may take longer than normal to download or may not download at all.

    • This may affect protected/redundant and unprotected/nonredundant XTC configurations. XTC revision has no impact on the bootup of the line cards but, instead, the line card revision.

    • To upgrade a working OCn card, perform a Cisco Transport Controller/Manager (CTC/CTM) switchover to the optical protection channel. Then, change out the optical card and return the older version unit(s).

    • To upgrade a protection OCn card, manually remove the card, and replace and return the older unit. This OCn upgrade procedure should result in no downtime.

    • For an Ethernet card upgrade, there is no in-service upgrade. The newer card should be inserted into an available slot, provisioned exactly like the older card, and changed out as quickly as possible, preferably during a maintenance window to minimize lost packets. Approximately 5 minutes is required for spanning tree reconvergance.

    Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

  • 7.0-Older XTCs with older line cards in the same 15327

    For the case in which one or two XTCs and one or more line cards are older in the same 15327, the upgrade procedure is:

    • Replace the older XTC(s) first by following the applicable recommended XTC upgrade procedure, which is outlined above. This is based on:

      • How many XTCs are older (none, one, or two)?

      • Are the older XTCs symptomatic or asymptomatic?

      Once these questions are answered, you can determine which XTC upgrade procedure to follow.

    • The last step is to upgrade the older line cards. The fastest and easiest way is to pull the card and replace it with a new one.

    Upgrade the optical cards first, then the Ethernet cards, if required.

    Downtime for a protection card is typically less than 5 minutes.

    There should not be any downtime for replacing the XTCs in a redundant configuration, except in the case of Ethernet traffic. Resetting the active XTC causes as much as a 5-minute outage while the spanning tree reconverges.

    Note: When an active XTC resets, any Ethernet traffic can experience as much as a 5-minute traffic outage while the spanning tree reconverges.

  • 8.0-XTC Notes

    • If software is being copied, then the XTC "FAIL" LED blinks red until the copy completes.

    • If the database is being copied, then the XTC "STBY/ACT" LED blinks amber and green until the copy completes.

Workaround/Solution

Customers who are experiencing the symptoms noted in this field notice should obtain replacement unit(s) and upgrade the system to newer versions by contacting Cisco Technical Support to request a Return Materials Authorization (RMA).

Note: Replacements of the XTC-28-3 and OC-12 cards will be Industrial Temperature (I-Temp) rated. I-Temp operation is -40°C tp +65°C.

XTC-14, E10/100T, and OC-48 cards will be repaired for proper bus termination.

Returned unit(s) will be upgraded to include the proper termination, tested, and returned to the pool of replacement products for continued upgrades.

How To Identify Hardware Levels

Note: Top Assembly Numbers (TAN) begin with "800". Revisions/versions of affected CTemp cards are indicated with "-01" or "-02" and can apply to rev. A0 though D0. Refer to the Products Affected section to determine the revision/version that specifically applies to the affected CTemp cards.

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Product Alert Tool - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.