Detect the Increase in StarOS Error Port Datalink and NPU Counters

Introduction

This document describes the script which detects the increase in error Datalink or NPU counters per port.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• StarOs

Components Used

This document is not restricted to specific software and hardware versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Problem

Error counters at the port level can be a great source of information in order to troubleshoot various issues with a StarOS node.

The most valuable information, in this case, is the variation of those counters during a certain period of time.

Static values that are available in the output of a single "show" command aren't providing sufficient information to make meaningful conclusions.

A typical approach is to collect several outputs of show commands and then make the difference manually.

This can be a difficult task, especially when it is not known what port exactly is impacted.

This script simplifies this process by providing the variation of error counters over a certain period of time per port.

Examples of issues that can be detected:

• MTU mismatches
• VLAN misconfiguration
• DataLink level errors

How does the Script Work?

In the SSD file, there are two outputs of show port npu counters and show port datalink counters taken at several minutes interval.

This permits to see the port level counters at a certain moment in time and also see their dynamics.

This script is checking the error counters from the command outputs and generates an alert when an increase in the counter is observed.

Usually, this indicates a problem at the physical or network level. Proceed with the steps to troubleshoot depending on the situation.

NPU Counters

These NPU counters are being observed:

Datalink Counters

These datalink counters are analyzed:

There is a series of datalink counters seen only for STM interfaces:

 Example Output

Increase in some of the error or drop counters fromshow port npu countersorshow port datalink countersoutputs are observed in the provided SSD.
The script highlights all the counters being checked, but only the ones with increase must be analyzed, that is the ones that contain the 'Following increase observed for port'statement
Note that such increases aren't necessarily pointing to an issue with the node. Usually, it is a problem with a cable, SFP, misconfiguration or network level problem.
Check the definition of the affected counter(s) and proceed forward with the steps to troubleshoot based on this.

########################## NPU COUNTERS ######################################

 No errors increase found during monitoring period

######################## DATALINK COUNTERS ###################################

 Errors observed in the output of 'show port datalink counters' between  Monday October 01 12:29:49 CDT 2018 and Monday October 01 13:03:24 CDT 2018 on the ports 6/10,6/16,5/15 

 - Following increase in errors is seen on port 6/10:

      RX OverSize frames:Frames: 404 



 - Following increase in errors is seen on port 6/16:

      RX OverSize frames:Frames: 402 


 - Following increase in errors is seen on port 5/15:
    
     RX OverSize frames:Frames: 3

How to Understand the Output?

If no variation was seen in any of the counters of our interest on any ports, the script returns nothing.

If there is a variation with at least one counter of our interest, on, at least, one port - the script would not generate an alert.

The alerts are grouped per type (NPU or Datalink) and then per port.

First, there would be a statement summarizing all findings and the monitoring period.

 Errors observed in the output of 'show port datalink counters' between  Monday October 01 12:29:49 CDT 2018 and Monday October 01 13:03:24 CDT 2018 on the ports 6/10,6/16,5/15 

Above it is between Monday, October 01 12:29:49 CDT 2018 and Monday, October 01 13:03:24 CDT 2018, i.e. it is around half an hour.

The timestamps are taken from the outputs of show port datalink counters or, respectively, show port npu counters

Afterwards, there is a summary of problematic counters identified per port.

 - Following increase in errors is seen on port 6/16:

      RX OverSize frames:Frames: 402 

In the example mentioned, there were 402 oversized frames received on the 6/16 port during the monitoring period (around half an hour).