Field Notice: FN - 70326 - cBR-8 - Potential Line Card Reload during Operation - Software Upgrade Recommended

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Publish Date	Comments
1.0	31-Jan-19	Initial Release

Products Affected

Affected OS Type	Affected Software Product	Affected Release	Affected Release Number	Comments
NON-IOS	IOSXE	16	16.4.1, 16.4.2, 16.4.3, 16.5.1, 16.5.1a, 16.5.1b, 16.5.2, 16.5.3, 16.6.1, 16.6.1a, 16.6.2, 16.6.3, 16.6.4, 16.6.4a, 16.7.1, 16.7.1a, 16.7.1b, 16.7.2, 16.8.1, 16.8.1a, 16.8.1b, 16.8.1c, 16.8.1d, 16.9.1, 16.9.1a, 16.9.1b, 16.9.1d, 16.9.2

Defect Information

Defect ID	Headline
CSCvf84094	PKO_ERROR, cBR-8 line card unexpectedly reloads

Problem Description

There is the potential for a cBR-8 line card reload during operation due to an Outbound Packet Data Error (PKO_ERROR).

Background

This issue might be observed when a cBR-8 system runs a heavy video load in addition to Data Over Cable Service Interface Specification (DOCSIS) traffic. When the line card encounters the issue, it core dumps and subsequently reloads. Line Card Redundancy (LCHA) recovers the system. In the case of Integrated Converged Cable Access Platform (I-CCAP) cBR-8, the system recovers in 100 seconds. However, Remote Phy (R-PHY) cBR-8 requires manual intervention.

It needs to be noted that Cisco IOS^® XE Software Releases 16.7.2, 16.8.1d, and 16.9.1 have different behavior. These releases have a workaround committed via Cisco bug ID CSCvj12365 - "Trigger LCHA when persistent PKO error is detected", where the DOCSIS downstream control traffic as well as the video traffic stops and then the line card reloads unexpectedly. This message is shown for both I-CCAP and R-PHY:

%VIDMAN-2-VIDEO_ILK_PKO_ERROR: CLC3: vidman: Video output exceeds system capacity. All video output STOPPED

The software triggers an LCHA event three seconds after this message. The system recovers automatically with no core dump for both I-CCAP and R-PHY.

Problem Symptom

The problem symptoms for the two scenarios are explained in this section.

I-CCAP cBR-8

The DOCSIS downstream control traffic and the video traffic stops on the line card. Also, the line card misses the upstream PHY health checks. As a result, the line card CDMAN process no longer functions and the line card unexpectedly reloads. In the case of CBR-8 I-CCAP, LCHA recovers the system. Recovery takes 100 seconds.

These messages might be observed:

%VIDMAN-2-VIDEO_ILK_PKO_ERROR: CLC3: vidman: Video output exceeds system capacity. All video output STOPPED ...

%IOSXE-3-PLATFORM: CLC3: cdman: US-PHY PHY0 BCM3142 missed 5 consecutive GMAC1 health checks ...

%CMCC_CBR-4-PROC_DOWN: CLC3: cmcc: Process us-sched (cdman) is down

R-PHY cBR-8

The line card output video traffic stops as well as the DOCSIS downstream control traffic. These messages display:

%VIDMAN-2-VIDEO_ILK_PKO_ERROR: CLC2: vidman: Video output exceeds system capacity. All video output STOPPED

%UBRCCE_CDM_INFRA-3-TEST_FAIL_ERR: CLC2: cdman: CDM_PT_PKOPKI_SENDRCV_TEST SANITY TEST FAILED, COUNT: 11

%UBRCCE_CDM_INFRA-3-TEST_FAIL_ERR: CLC2: cdman: CDM_PT_PKOPKI_SENDRCV_TEST SANITY TEST FAILED, COUNT: 12

For the R-PHY cBR-8 case, given that there is no onboard PHY on the line card, a line card switchover command needs to be entered in order to restore service. This message is shown for both I-CCAP and R-PHY scenarios:

%VIDMAN-2-VIDEO_ILK_PKO_ERROR: CLC3: vidman: Video output exceeds system capacity. All video output STOPPED

Workaround/Solution

For the I-CCAP cBR-8 scenario, the software automatically triggers LCHA in order to prevent a service outage.

For the R-PHY cBR-8 application, the LCHA switchover command needs to be initiated in order to restore service.

Cisco recommends that customers who encounter this issue upgrade to a later release which has the fix, as shown in this table.

cBR-8 Cisco IOS XE Software Release	Does the issue exist?	Cisco IOS XE Software Release with the Fix
3.18SP	No
16.4.x	Yes	16.7.3
16.5.x	Yes	16.7.3
16.6.x	Yes	16.7.3
16.7.x (16.7.1a, 16.7.1b, 16.7.2)	Yes	16.7.3
16.8.x (16.8.1, 16.8.1d)	Yes	16.8.1e
16.9.1a	Yes	16.10.1c
16.10.1 and later	No

 

Refer to Upgrading the Cisco cBR-8 Router for instructions on how to upgrade the cBR-8 Cisco IOS XE Release.

How To Identify Affected Products

The hardware information can be determined as indicated in this image:

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email
• By telephone

Receive Email Notification For New Field Notices

Cisco Notification Service—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.