ONS 15454 MSTP SNMP Fault Management

Introduction to SNMP Trap Notification:

An SNMP trap is basically an asynchronous notification set from SNMP agent to a network mangement system. Like other messages in SNMP, traps are sent using UDP.

A trap is a bundle of data that's defined by a managed information base (MIB). Traps falls into catagories: Generic and enterprise-specific.

Prerequisites

• Basic SNMP knowedge.
• Overview on MSTP Cisco 15454.

SNMP Fault Management Management Architecture:

SNMP Setup on multi-shelf environment:

• When a Multi-shelf node is OSC (or GCC) connected only, it sends the traps to the LAN anyway.
• To avoid this issue, the gateway node, i.e. the LAN connected node has to be set:

      - Either “proxy-only”

      - Or GNE.

• Below is some description about Socks Proxy:

     Socks Proxy (GNE, ENE, Proxy only, LNE)

         If Enable Socks Proxy button is selected.  And, GNE is selected.

            --- Creates a SOCKS tunnel to be built between the PC running CTC and the Lan connected node.

            --- It means it is the Gateway node (GNE) that uses SOCKS PROXY.

            --- This optionTurns on Firewall. (GNE = SOCKS PROXY + FIREWALL)

            --- This node is LAN connected and has ENE's behind it. 

            --- ENE's behind GNE cannot advertise across the LAN.

            --- You can ping, telnet and CTC to GNE and be able to see all the ENE's behind the GNE.

            --- You cannot ping, telnet or CTC to ENE.

       If Enable Socks Proxy button is selected.  And ENE is selected.

           --- Creates a SOCKS tunnel to be built between the PC running CTC and the Lan connected node.

           --- This is for nodes that are DCC connected only.

           --- This setting prevents the ENE node from adding any routes to the routing table with a next hop of the LAN         interface (motfcc0 for 15xxx).

           --- You can ping ENE's unless the LAN connected node is a SOCKS GNE.

           --- A tech can ping, telnet or CTC to the ENE if connected to the node with a pc in the same subnet as the NE.

      If Enable Socks Proxy button is selected.  And Socks Proxy Only is selected.

         --- Creates a SOCKS tunnel to be built between the PC running CTC and the Lan connected node.

         --- Same as GNE except it does not turn on Firewall.  
         --- Firewall is turned off.

         --- You can ping and telnet to the nodes.

      If Enable Socks Proxybutton is selected.  And Socks Proxy Only is selected.

         --- Creates a SOCKS tunnel to be built between the PC running CTC and the Lan connected node.

         --- Same as GNE except it does not turn on Firewall.  
         --- Firewall is turned off.

         --- You can ping and telnet to the nodes.

SNMP Setup on multi-shelf environment Socks proxy:

• The LNE must have a static route to advertise itself as gateway in the DCC area.
• A sample static route is a default one, destination 0.0.0.0, next hop the DCN router, cost=10.
• ENE nodes must send traps to LNE, port 391.

Proprietary MIBs:

Cisco ONS 15454 implements enterprise-specific MIBs as well as standard MIBs from IETF, while the IETF MIBs are common for both Cisco 15327 as well as Cisco 15454, the same is not true for Proprietary MIBs. Each product has a set of three proprietary MIB files.

The enterprise-specific MIB files are available in SMIv2 (Commonly referred to as 'SNMP Version 2 MIB') as well as SMIv1 (Commonly referred to as 'SNMP Version 1 MIB'). Depending on what is required by the NMS, appropriate set of MIB files should be loaded in the NMS.

Note that there are no differences between SMIv2 and SMIv1 MIB files except the syntax, hence it would not impact the NMS if SMIv1 MIB files are loaded instead of SMIv2 MIB files, or vice versa.

An unique feature of SNMP is that a particular version of a mib file is always compatible with all the earlier versions of the same file. For instance, CERENT-454-MIB.mib file is compatible with software version R2.2.3, R2.2.1, R2.0, etc. This is a mandatory property of every SNMP mib and Cisco ONG proprietary MIB files are no exception. Hence, when there is a choice, load the latest MIB files into the NMS for worry-free operations.

SMIv1/SMIv2 MIB files for Cisco ONS 15454:
 

1. CERENT-GLOBAL-REGISTRY.mib
2. CERENT-TC.mib
3. CERENT-454-MIB.mib
4. CERENT-MSDWDM-MIB.mib
5. CERENT-OPTICAL-MONITOR-MIB.mib
6. CISCO-SMI.mib*
7. CISCO-OPTICAL-MONITOR-MIB.mib*
8. CISCO-VOA-MIB.mib

Loading MIB files in NMS platforms:

Following these guidelines while loading the SNMP MIB files into a Network Management System would make things easier and quicker.

• First ensure that you are loading the acceptable version of the MIB files. For instance, some NMS platforms still will accept SMIv1 (or 'SNMP version 1') mib files only. 
• The mib files should be loaded in the order as they appear above. If this order is not strictly followed, then one or more mib files will not compile. The loading order specified in the MIB dependency table. This table facilitate loading of only a subset of IETF standard MIB files if desired so.
• If one or more IETF MIB files result in errors in the NMS while loading, the vendor of the NMS should be contacted to resolve the issue.

MIB Loading in HPOV Network Node Manager

Pick up SMIv2 mib files from above and load the HPOV NNM (HP OpenView Network Node Manager) in the correct order. 
 

• Ensure that proprietary mib files are loaded in the NNM. Look under Optionsin the main NNM panel and follow the options to load the mib files.
• Next open the Event Configuration
• From any Alarm Browser window, select Actions:Configure Event.
• In the top half of the window, select the cerent454Event if it is Cisco 15454 system and cerentGenericEvent if it is Cisco 15327 system.
• In the bottom half of the window, select a proprietary Alarm that you want to configure on the NNM's alarm Browser window.
• Select Edit:Events->Modify, then select the Event Message
• Make a selection in the Category
• Make a selection in the Severityfield, e.g., Major
• In Event Log Messagefield, enter the following:
  $N $2 Object:$3 Index:$4 Slot:$5 Port:$6 AID:$8
• Hit OK, and save the work under File
• Now every trap from the node will appear with a message and the varbinds which came with it.
  For instance, 
  "lossOfSignal cleared Object:ds3 Index:28449 Slot:3 Port:1 AID:FAC-5-1"
• You may want to configure an action to be executed when this trap is received. It may also be desired to have this above message in a different form. These options can be exercised using the above panel, if desired.

MIB Dependency Table:

The following table shows the dependencies of a mib file that are to be resolved while being loaded in a NMS.

Trap handling:

1.Read the trap

2.Identify the following:

Trap: TrapId

cerent454AlarmObjectType attrValue: ObjectType

cerent454AlarmState attrValue: ServiceAffecting/ No service Affecting

cerent454AlarmObjectName: Level/Slot/Port

Type = IpAddress, Value = 10.105.142.205 (V2 only)

3.Browse through  the Troubleshooting guidelines / alarm troubleshooting, select the proper TrapId, and browse to the related section.

4.Use the information on Level / Slot / Port to identify the card and port affected

5.Implement the procedure and clear the alarm.

SNMP V1 Traps example:

SNMPv1 Trap: lossOfSignalForOpticalChannel

(Wed May 05 11:20:49 2014) SNMPv1 Trap: Agent IP = 10.105.142.205, with time = 18 hrs : 31 min : 16.37 sec (6667637)

Enterprise = 1.3.6.1.4.1.3607.6.10.30, Generic = enterpriseSpecific, Specific = lossOfSignalForOpticalChannel

AttrOid1 = cerent454NodeTime.0 , AttrType = OctetString, AttrValue = 20051128022020S

AttrOid2 = cerent454AlarmState.8195.5600 , AttrType = Integer, AttrValue = criticalServiceAffecting(100)

AttrOid3 = cerent454AlarmObjectType.8195.5600 , AttrType = Integer, AttrValue = dwdmTrunk(170)

AttrOid4 = cerent454AlarmObjectIndex.8195.5600 , AttrType = Integer, AttrValue = 8195

AttrOid5 = cerent454AlarmSlotNumber.8195.5600 , AttrType = Integer, AttrValue = 2

AttrOid6 = cerent454AlarmPortNumber.8195.5600 , AttrType = Integer, AttrValue = port2(20)

AttrOid7 = cerent454AlarmLineNumber.8195.5600 , AttrType = Integer, AttrValue = 0

AttrOid8 = cerent454AlarmObjectName.8195.5600 , AttrType = OctetString, AttrValue = CHAN-2-2

Varbind 1 in trap 1.3.6.1.4.1.3607.6.10.100.10.20.0: Matches MIB definition.

Varbind 2 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.80.8195.5600: Matches MIB definition.

Varbind 3 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.20.8195.5600: Matches MIB definition.

Varbind 4 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.60.8195.5600: Matches MIB definition.

Varbind 5 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.30.8195.5600: Matches MIB definition.

Varbind 6 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.40.8195.5600: Matches MIB definition.

Varbind 7 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.50.8195.5600: Matches MIB definition.

Varbind 8 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.100.8195.5600: Matches MIB definition.

Is the trap a ServiceAffecting alarm ?

SNMPv1 Trap: lossOfSignalForOpticalChannel

(Wed May 05 11:20:49 2014) SNMPv1 Trap: Agent IP = 10.105.142.205, with time = 18 hrs : 31 min : 16.37 sec (6667637)

 Enterprise = 1.3.6.1.4.1.3607.6.10.30, Generic = enterpriseSpecific, Specific = lossOfSignalForOpticalChannel

 AttrOid1 = cerent454NodeTime.0 , AttrType = OctetString, AttrValue = 20051128022020S

AttrOid2 = cerent454AlarmState.8195.5600 , AttrType = Integer, AttrValue =criticalServiceAffecting(100)

 AttrOid3 = cerent454AlarmObjectType.8195.5600 , AttrType = Integer, AttrValue = dwdmTrunk(170)

 AttrOid4 = cerent454AlarmObjectIndex.8195.5600 , AttrType = Integer, AttrValue = 8195

 AttrOid5 = cerent454AlarmSlotNumber.8195.5600 , AttrType = Integer, AttrValue = 2

 AttrOid6 = cerent454AlarmPortNumber.8195.5600 , AttrType = Integer, AttrValue = port2(20)

 AttrOid7 = cerent454AlarmLineNumber.8195.5600 , AttrType = Integer, AttrValue = 0

 AttrOid8 = cerent454AlarmObjectName.8195.5600 , AttrType = OctetString, AttrValue = CHAN-2-2

The highlighted output above, has to be considered as URGENT alarm.

What to do --> Go to Cisco troubleshooting guideline \ Alarm Troubleshooting:
http://www.cisco.com/c/en/us/td/docs/optical/15000r9_6/dwdm/troubleshooting/guide/b_454d96_ts.html

Identify the Probable Cause and browse to it:

Troubleshooting Guide

SNMP V2 Traps example:

SNMPv2 Trap: lossOfSignalForOpticalChannel

(Wed May 05 11:20:49 2014): SNMPv2 Trap : Request Id = 254, Error Status = No Error , Error Index = 0

Oid1 = sysUpTime.0 , Type = TimeTicks, Value = 116 hrs : 48 min : 23.38 sec (42050338)

Oid2 = snmpTrapOID.0 , Type = ObjectID, Value = 1.3.6.1.4.1.3607.6.10.30.0.5600

Oid3 = cerent454NodeTime.0 , Type = OctetString, Value = 20051128031653S

Oid4 = cerent454AlarmState.65544.5600 , Type = Integer, Value = criticalServiceAffecting(100)

Oid5 = cerent454AlarmObjectType.65544.5600 , Type = Integer, Value = ots(3210)

Oid6 = cerent454AlarmObjectIndex.65544.5600 , Type = Integer, Value = 65544

Oid7 = cerent454AlarmSlotNumber.65544.5600 , Type = Integer, Value = 16

Oid8 = cerent454AlarmPortNumber.65544.5600 , Type = Integer, Value = port1(10)

Oid9 = cerent454AlarmLineNumber.65544.5600 , Type = Integer, Value = 0

Oid10 = cerent454AlarmObjectName.65544.5600 , Type = OctetString, Value = LINE-16-1-RX

Oid11 = 1.3.6.1.6.3.18.1.3.0 , Type = IpAddress, Value = 10.105.142.205

Varbind 3 in trap 1.3.6.1.4.1.3607.6.10.100.10.20.0: Matches MIB definition.

Varbind 4 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.80.65544.5600: Matches MIB definition.

Varbind 5 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.20.65544.5600: Matches MIB definition.

Varbind 6 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.60.65544.5600: Matches MIB definition.

Varbind 7 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.30.65544.5600: Matches MIB definition.

Varbind 8 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.40.65544.5600: Matches MIB definition.

Varbind 9 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.50.65544.5600: Matches MIB definition.

Varbind 10 in trap 1.3.6.1.4.1.3607.6.10.20.30.20.1.100.65544.5600: Matches MIB definition.

Same procedure:

• The only difference is in Source IP address: how to identify the node:

Oid11 = 1.3.6.1.6.3.18.1.3.0 , Type = IpAddress, Value = 10.105.142.205

• This provides the IP address of the source node.

Relevant documentation:

• Troubleshooting Guide for DWDM:

http://www.cisco.com/c/en/us/td/docs/optical/15000r9_6/dwdm/troubleshooting/guide/b_454d96_ts.html

• This link contains also a very useful explanation on how the 15454 provides the SNMP Management:

http://www.cisco.com/c/en/us/td/docs/optical/15000r9_0/dwdm/reference/guide/454d90_ref/454d90_snmp.html

• The MIBs are on CCO Cisco Connection Online.
• The Following link contains the module for objects and events for the Cisco ONS 15454 received trap:

http://issues.opennms.org/secure/attachment/10480/CERENT-454-MIB.txt