Table Of Contents
Trap Management
Fault Management Interface
Traps
Access Methods
Process for Fault Management
Manager Registration
Example of Registering with RTM Service Agent
Example of Unregistering with the RTM Service Agent
Extending the Expiration Time
Manager Entry Configuration
Trap Group Limitations
Register Agents When Using HPOV
Trap Processing
Trap Discovery Process
Trap Recovery
Trap Recovery Process
Special Case of Manager Registration
Set Special Registration
Trigger RTM Protocol
Trap Management
This chapter describes how to manage traps through CWM. The following topics are included in this section:
•
Fault Management Interface
•
Process for Fault Management
•
Special Case of Manager Registration
Fault Management Interface
In Release 12 of CWM fault management by external OSSs is based on robust SNMP traps. This trap mechanism, known as Robust Trap Mechanism (RTM), is supported by the SNMP Service Agent running in conjunction with a CWM management system. Through the Service Agent, up to 16 external SNMP Managers (with one reserved for CWM) can register to receive network and service related traps.
In CWM Release 12 some network resources utilize SNMP traps to relay fault information up to CWM, while others utilize proprietary robust alarms and string events. Robust alarms, in Cisco terms, refer to the state representations of network resources at a specific time. String or textual events refer to descriptive events triggered by certain activities and are sent up to the management system.
The two fault management mechanisms have no correlation. Textual events are sent only to port 162 on the CWM workstation.
Note
The BPX sends only textual events.
Actions occurring in the network, such as resource failures, might result in the generation of textual events as well as a change in alarm status.
Other actions might result in no alarm status; however, they generate a textual event. Furthermore, the order in which robust alarms are received might not reflect the chronological sequence of network events; robust alarms reflect state changes at time <x>. These issues must be considered when constructing Managers that receive these faults.
Traps
The primary interface for fault management is through SNMP traps emitted by the SNMP Service Agent. This agent provides a uniform interface that partially hides the hybrid fault management interfaces supported by the various network elements. Managers that register to receive traps from the SNMP Service Agent can receive trap representations of robust alarm status changes. Managers also receive, through the Service Agent, traps from network elements supporting native traps.
Therefore, the complete fault management interface presented to users of the Service Agent includes:
•
traps from Cisco MGX 8220 Cisco MGX 8230, Cisco MGX 8250, Cisco MGX 8830, Cisco MGX 8850 PXM1-based, Cisco MGX 8850 PXM1E-based, and Cisco MGX 8850 PXM45-based.
•
trap representations of robust alarms from Cisco IGX, Cisco BPX 8600.
•
trap representations of textual events from Cisco IGX, Cisco BPX 8600, Cisco MGX 8230, Cisco MGX 8250, Cisco MGX 8830, Cisco MGX 8850 PXM1-based, Cisco MGX 8850 PXM1E-based, and Cisco MGX 8850 PXM45-based.
•
traps from CWM
The traps that are delivered by the Service Agent are standard SNMP-Version 1 traps. These traps are delivered on behalf of the network devices and CWM.
The Service Agent delivers traps in Robust Trap Mechanism (RTM) mode. In this mode lost traps can be detected and recovered by the SNMP managers.
To receive these RTM traps from the Service Agent, an SNMP manager must register itself with the Service Agent.
The Service Agent delivers the following categories of traps:
•
Traps that are generated in CWM that reflect the CWM health.
•
Traps that are generated in CWM based on the proprietary robust messages received from the
Cisco BPX 8600 and Cisco IGX 8400 devices.
•
Traps that are generated at Cisco MGX 8220, Cisco MGX 8230, Cisco MGX 8250,
Cisco MGX 8850 PXM1-based, and Cisco MGX 8850 PXM45-based platforms. These traps are forwarded to the manager untouched.
Trap definitions for this category are not available in SV+Network.mib. They are available in the corresponding Cisco MGX MIBs.
•
User connection traps that are generated in CWM based on the traps and robust messages received for individual segments from Cisco MGX, Cisco BPX, and Cisco IGX platforms.
For traps that are generated in CWM, see the "Traps" section of "Network MIB."
Access Methods
The RtmProxy does not validate the community string for the objects in the aagTraps Group or the trapConfigTable.
To obtain the number of Managers registered for traps, issue an SNMP GET on the following variable of the trapConfigTable:
•
OID: 1.3.6.1.4.1.351.120.1.2.0
•
Name: managerNumOfValidEntries
•
Type: Integer
•
Community: public (ignored)
>snmpGET -p 8161 -c public nm20fst7 managerNumOfValidEntries.0
stratacom.rtm.trapsConfig.managerNumOfValidEntries.0 : INTEGER: 2
To obtain the sequence number of the last trap generated on the CWM SNMP Service Agent, issue an SNMP GET on the following variable:
•
OID: 1.3.6.1.4.1.351.120.1.3.0
•
Name: lastSequenceNumber
•
Type: Integer
•
Community: public (ignored)
>snmpGET -p 8161 -c public nm20fst7 lastSequenceNumber.0
stratacom.rtm.trapsConfig.lastSequenceNumber.0 : INTEGER: 833
Process for Fault Management
This section contains the process for implementing fault management capabilities based on the Service Agent trap stream. The overall process for fault management includes:
1.
Manager Registration with the Service Agent (via the trapConfigTable) to receive traps. This registration process results in the Service Agent storing some context information about the Manager, such as its IP address and its trap retrieval status.
2.
Trap Processing. For each trap, check the sequence number when you are concerned about missing traps.
3.
Trap Recovery. The Manager can synchronously retrieve missing traps one at a time. The Manager has control over where to start trap retrieval by setting the trap sequence number to be retrieved. Successive missing traps are obtained via repetitive trap retrieval requests.
Manager Registration
To receive the RTM traps from the Service Agent, an SNMP manager must register itself with the Service Agent.
As part of the registration process, the Manager can specify a specific port as a destination for all traps. When a port is not specified, the agent sends all traps to port 162 on the Manager.
The Service Agent can support up to 16 external SNMP Managers (with one reserved for CWM). The managerNumOfValidEntries MIB variable stores the number of subscribed Managers in the RTM table.
To receive the RTM traps from the Service Agent, an SNMP manager must register itself with the Service Agent.
To register an SNMP manager with the Service Agent, enter a SET request on the trapConfigTable with the following mandatory objects:
•
managerRowStatus.<managerIPaddress> = addRow
•
readingTrapFlag.<managerIPaddress> = false
•
trapRedeliverFlag.<managerIPaddress> = false
A maximum of 16 managers can be registered with the Service Agent.
Because only 16 slots are available for manager registration, aging (keep alive) is implemented in the Service Agent so that the registration of inactive managers is automatically cancelled.
Each registered manager is required to poll the Service Agent by doing a GET on the manager entry to keep the entry alive. If a manager fails becomes inactive, the manager is automatically unregistered, and no further traps are sent.
Example of Registering with RTM Service Agent
To register with the RTM Service Agent, send a SET request on the following two variables:
•
OID: 1.3.6.1.4.1.351.120.1.1.1.2.<IPADDR>
where <IPADDR> is the IP address of the Manager in dotted decimal notation.
–
Name: managerPortNumber
–
Type: Integer
–
Community: public (ignored)
–
Value: <ManagerPortNumber>
•
OID: 1.3.6.1.4.1.351.120.1.1.1.3.<IPADDR>
where <IPADDR> is the IP address of the Manager in dotted decimal notation.
–
Name: managerRowStatus
–
Type: Integer
–
Community: public (ignored)
–
Value: 1
This example uses Manager IP address: 192.99.88.101 and Port number: 162.
>snmpSET -p 8161 -c private nmclearc managerPortNumber.192.99.88.101 integer 162
managerRowStatus.192.99.88.101 integer 1
stratacom.rtm.trapsConfig.trapConfigTable.trapConfigEntry.managerPortNumber.192.99.88.101
: INTEGER: 162
stratacom.rtm.trapsConfig.trapConfigTable.trapConfigEntry.managerRowStatus.192.99.88.101 :
INTEGER: addRow
Note
To keep the registration active, the SNMP manager must send a keep-alive with the Service Agent once every 60 minutes.
Example of Unregistering with the RTM Service Agent
To unregister with the RtmProxy, send a SET request on the following variable:
•
OID: 1.3.6.1.4.1.351.120.1.1.1.3.<IPADDR>
where, <IPADDR> is the IP address of the Manager in dotted decimal notation.
•
Name: managerRowStatus
•
Type: Integer
•
Community: public (ignored)
•
Value: 2
This example uses Manager IP address: 192.99.88.101
> snmpSET -p 8161 -c private nmclearc managerRowStatus.192.99.88.101 integer 2
stratacom.rtm.trapsConfig.trapConfigTable.trapConfigEntry.managerRowStatus.192.99.88.101 :
INTEGER: delRow
Extending the Expiration Time
To extend the expiration time, enter a GET request with the following object:
managerRowStatus.<managerIPaddress>
A successful GET response indicates that the registration is extended. Otherwise, the registration is already expired.
The default expiration time is 60 minutes. It is recommended that the keep alive be done at least once every 55 minutes.
The default expiration time can be changed in the /usr/users/svplus/config/process.conf file. Use the command line option -k for starting the RtmProxy process.
For example, to change the expiration time to 7200 seconds or 2 hours execute the following command:
RtmProxy on off . RtmProxy -v -k 7200
Note
To keep the registration active, the SNMP manager must send a keep-alive with the Service Agent once every 60 minutes.
Manager Entry Configuration
Because the Service Agent delivers all of the traps from the entire network, the SNMP manager can filter out unwanted traps.
The SNMP manager can configure the Service Agent to deliver the traps that are of interest to the manager by defining trap groups and numbers in the /usr/users/svplus/config/trap_filter.conf file This file is used by the Service Agent to filter out the traps.
The trap_filter.conf file contains predefined trap groups and traps. You can modify this file.
To control the trap groups, enter a SET with the following object in the trapConfigTable:
trapFilterRegisterCategory.<managerIPaddress> = <Bit Mask, 1 bit per Trap Group>
You can SET this object as part of the registration or at a later time.
If you leave the parameter at the default value, all trap groups are delivered.
Trap Group Limitations
You can define up to 32 trap groups in trap_filter.conf file. Only one file can be defined per CWM workstation. This file is used for all of the SNMP managers. An SNMP manager can control which trap groups to receive.
Register Agents When Using HPOV
You can use HP OpenView (HPOV) to view traps. HPOV does not have to be installed on the same machine as CWM.
If you want to receive CWM traps using HPOV that is installed on a separate machine than CWM, you must register with the RtmProxy that is running on the CWM machine.
To register with the RtmProxy, complete the following steps:
Step 1
Make sure all CWM MIBs are integrated with HPOV on the machine on which HPOV is installed.
If the MIBs are not integrated, you see OIDs in event browser.
Step 2
Make sure the RtmProxy is running on the CWM workstation.
Step 3
Issue the following SNMP commands on the CWM workstation.
/opt/OV/bin/snmpset -p 8161 -c private <cwm_machine>
managerPortNumber.<ipaddress_of_HPOV m/c> integer 162
managerRowStatus.<ipaddress_of_HPOV_m/c> integer 1
The following example shows the command syntax with the CWM machine as cwmnectra2 and the machine for viewing traps has an IP address of 10.77.240.67.
/opt/OV/bin/snmpset -p 8161 -c "private" cwmnectra2
managerPortNumber.10.77.240.67 integer 162
managerRowStatus.10.77.240.67 integer 1
Step 4
Open the Event browser on machine with HPOV installed, and see if the browser is receiving traps.
Step 5
If you do not want to view the traps in HPOV (unregister), issue the following command on CWM workstation.
/opt/OV/bin/snmpset -p 8161 -c private <cwm_machine>
managerRowStatus.<ipaddress_of_HPOV_m/c> integer 2
The following example shows unregistering with the RtmProxy using the same variables as above.
/opt/OV/bin/snmpset -p 8161 -c "private" cwmnectra2 managerRowStatus.10.77.240.67 integer
Trap Processing
The Robust Trap Mechanism (RTM) discovers and recovers the lost traps. This mechanism requires the SNMP managers to provide registration and some follow-up interface to the Service Agent.
The following steps summarize the RTM process:
•
The Service Agent includes a sequence number in each of the traps.
•
The Service Agent saves the last 10000 traps in case they need to be delivered again.
•
The SNMP manager checks for the continuity of the sequence number on the received trap.
•
On detecting an out-of-sequence number, the SNMP manager informs the Service Agent about the missing traps.
•
The Service Agent resends the missing traps to the manager.
Trap Discovery Process
All the traps that are delivered by the Service Agent to the manager contain a common MIB object called lastSequenceNumber. This lastSequenceNumber is incremented by 1 for each trap that is delivered.
Because of trap filtering, the traps are not broadcast to all of the managers. Instead, they are sent on an individual basis depending on the categories of traps that a client registers with the RTM by using the trapFilterRegisterCategory object.
The lastSequenceNumber is maintained by each registered manager. This number can be different for different managers.
The manager uses the following process for discovering lost traps:
1.
The manager saves the lastSequenceNumber value from the first trap.
2.
For each subsequent trap, the manager compares the lastSequenceNumber value with the sequence number that was stored from the previous trap.
3.
If the values are in sequence, the manager saves the new lastSequenceNumber and repeats the process. If the values are out of sequence, the manager uses the recovery process.
Note
The lost traps could be any number.
Trap Recovery
Whenever the SNMP manager discovers an out-of-sequence lastSequenceNumber in the trap, the manager must follow the predefined protocol of RTM mode for recovering lost traps.
Trap Recovery Process
The manager uses the following process for recovering lost traps:
1.
The manager discards any traps coming from the Service Agent and enters RTM mode.
2.
The manager sends a SET request with the following objects to the Service Agent:
•
readingTrapFlag.<managerIPaddress> = true
•
trapRedeliverFlag.<managerIPaddress> = false
•
nextTrapSeqNum.<managerIPaddress> = ExpectedSequenceNumber
3.
The Service Agent sends the SET response and stops sending any more traps to the manager.
4.
The manager receives the SET response and sends a GET request with the following objects to recover the first missing trap:
•
trapSequenceNum.<managerIPaddress>
•
trapPduString.<managerIPaddress>
•
endofQueueFlag.<managerIPaddress>
5.
The Service Agent sends the GET response to the manager with the following objects:
•
trapSequenceNum.<managerIPaddress> = ExpectedSequenceNumber + X
The value of X is 0 if the trap requested is still available in the circular buffer. Otherwise, the ExpectedSequenceNumber + X represents the oldest trap available in the buffer.
•
trapPduString.<managerIPaddress> = TrapPDU
•
endofQueueFlag.<managerIPaddress> = <false or true>
This object is false if more traps are to be recovered by the manager. This object is true if this trap is the only one missing.
6.
The manager receives the GET response, and verifies the values of the following objects:
•
trapSequenceNum
If this value is not equal to the ExpectedSequenceNumber, some traps are over-written.
•
endofQueueFlag
If this value is true, no more traps are to be recovered.
7.
The manager sends a SET request with the following object:
trapRedeliverFlag.<managerIPaddress>=true
8.
The Agent sends the SET response.
If the value of the endofQueueFlag is already set to true, the manager initializes the readingTrapFlag and trapRedeliverFlag to false and gets into the real-time mode. No traps are sent from the circular queue.
If the value of the endofQueueFlag is false, the Agent starts sending the saved traps from ExpectedSequenceNumber+1.
9.
The manager receives the SET response and starts receiving the traps.
The manager is in real-time mode and should start checking the lastSequenceNumber again.
10.
After the Service Agent sends all of the lost traps to the manager (when the Service Agent catches up with the lastSequenceNumber), the Service Agent switches to real-time mode and initializes readingTrapFlag and trapRedeliverFlag to false.
Special Case of Manager Registration
When an SNMP manager registers with the Service Agent, the manager can request the traps that were generated prior to the registration.
The manager retrieves the prior-registration traps by using the following process:
•
The manager invokes special registration with the Service Agent.
•
The manager recovers the traps in special RTM mode.
Set Special Registration
For special registration the manager performs a SET on the following objects:
•
managerRowStatus.<managerIPaddress> = addRow
•
readingTrapFlag.<managerIPaddress> = true
•
nextTrapSeqNum.<managerIPaddress> = -<X> (X = Number Of Previous TRAPs To Retrieve)
•
trapRedeliverFlag.<managerIPaddress> = false
•
trapFilterRegisterCategory.<managerIPaddress> = <trap filter> (Optional)
After the registration is successful, the lastSequenceNumber is initialized to X, instead of the usual value=0. Because this registration involves some special processing in the Service Agent, the processing time is a few seconds. During that time the Service Agent sends the SET response back to the manager.
Trigger RTM Protocol
After the SET response is received, the manager can trigger the special RTM protocol:
1.
The manager is in regular mode and sends a SET request with the following objects:
•
nextTrapSeqNum.<managerIPaddress> = 0
•
trapRedeliverFlag.<managerIPaddress> = true
The manager intends to recover traps from sequence=0 to X and is ready to receive the traps.
2.
The Service Agent sends the SET response and starts sending the saved traps from lastSequenceNumber=0.
3.
The manager receives the SET response and starts receiving the traps at the same time.
The manager starts checking the lastSequenceNumber.
4.
After the Service Agent sends all of the saved traps to the manager (when the Agent catches up with lastSequenceNumber), the Agent switches to real-time mode and initializes readingTrapFlag and trapRedeliverFlag to false.