Table Of Contents
Managing Faults
9.1 What Is Fault Management?
9.1.1 What the NE Provides
9.1.2 Fault Notification and Maintenance
9.1.3 Root Cause Analysis
9.2 Where Can I Get Information on Affected Services and Customers?
9.2.1 Viewing the Alarm Browser
9.2.2 Filtering Data in the Alarm Browser
9.2.3 Adding Alarm Notes
9.2.4 Viewing the Alarm Log
9.2.5 Filtering Data in the Alarm Log
9.2.6 Viewing the Syslog Viewer
9.2.7 Filtering Data in the Syslog Viewer
9.2.8 Diagnostic Center—MGX Voice Gateway Devices
9.3 What Fault Information Can I See?
9.3.1 How Are Alarms Displayed?
9.3.2 Suppressing Alarms
9.4 Is the Service Working?
9.4.1 Locating Alarms
9.4.2 Setting Up and Viewing Alarm Configuration Parameters
9.4.3 Creating Alarm Profiles—CTC-Based NEs
9.4.4 Applying Alarm Profiles—CTC-Based NEs
9.4.5 Managing Alarm Profiles—CTC-Based NEs
9.4.6 Using the Recovery Properties Pane
9.4.7 Using the Statistics Panel—MGX Voice Gateway Cards
9.4.8 Displaying the Status of All Nodes in the Network—MGX Voice Gateway Cards
9.4.9 Displaying the Status of a Node—MGX Voice Gateway Cards
9.4.10 Displaying the Status of MGX Voice Gateway Cards
9.4.11 Displaying the Status for Lines or Ports—MGX Voice Gateway Cards
9.4.12 Displaying the Status for Lines in a Loopback—MGX Voice Gateway Cards
9.4.13 Displaying the Paths in Loopback for the VXSM-OC3 Card
9.4.14 Displaying the Status for a Bit Error Rate Test—VXSM and VISM-PR Cards
9.4.15 Monitoring and Diagnosing MGC Communication Protocols on VXSM Cards
9.4.16 Monitoring and Diagnosing the Status for VISM-PR Features
9.4.17 Managing Diagnostics for DPNSS and DLC—VXSM Cards
9.4.18 Managing Diagnostics for SS7 Signaling Components—VXSM Cards
9.4.19 Managing Diagnostics for VXSM Card Resource Overload and Utilization
9.4.20 Collecting Real-Time Performance and Health Statistics—PXM45 and PXM1E Cards
9.4.21 Monitoring SNMPv3 Real-Time Counters
9.4.22 Diagnosing Connections—MGX Voice Gateway Cards
9.4.23 Bit Error Rate Test—MGX Voice Gateway Cards
9.5 Where Is the Fault?
9.5.1 Sources of Information
9.5.2 Identifying and Monitoring Alarms
9.5.3 Using Visual and Audible Alarm Notifications
9.5.4 Sources of Events
9.5.5 Enabling Fault Synchronization—CTC-Based NEs
9.6 How Can I Find the Root Cause of the Fault?
9.6.1 Setting Up Error Logs
9.6.2 Viewing the Error Log
9.6.3 Filtering Data in the Error Log
9.6.4 Managing the NE Audit Trail—CTC-Based NEs
9.6.5 Setting Debug Options
9.7 Who Is Responsible for Managing the Fault?
9.7.1 Acknowledging and Unacknowledging Alarms
9.7.2 Configuring Alarm Acknowledgement and Alarm Notes
9.8 How Can the Fault Be Fixed?
9.8.1 Clearing a Security Violation Alarm—ONS 15305 R3.0, ONS 15327, ONS 15454 SONET, ONS 15454 SDH
9.8.2 Performing a System Reset—ONS 15600 SONET and ONS 15600 SDH
9.8.3 Restarting an ONS 15800, ONS 15801, or ONS 15808 Laser
9.8.4 Exporting Alarms and Events to a Text File
Managing Faults
This chapter describes the process of fault management (FM), and details the options available in CTM to locate, diagnose, and report network problems. This chapter includes the following information:
•
What Is Fault Management?
•Where Can I Get Information on Affected Services and Customers?
•What Fault Information Can I See?
•Is the Service Working?
•Where Is the Fault?
•How Can I Find the Root Cause of the Fault?
•Who Is Responsible for Managing the Fault?
•How Can the Fault Be Fixed?
9.1 What Is Fault Management?
Fault management is the process of locating, diagnosing, and reporting network problems. This is important for increasing network reliability and effectiveness, and for increasing the productivity of network users. Fault management is more than just handling emergencies. It provides functions for managing problems with services and handling customer-facing service problems.
Efficient fault management can:
•Save repair costs through efficient fault detection, location, and correction
•Improve customer care through efficient trouble administration
•Improve service availability and equipment reliability through proactive maintenance and through measurement, review, and corrective action
One responsibility of fault management is to detect faults. A piece of equipment, a transmission medium, a software module, or a database is said to be in a fault state if it cannot perform its intended function and meet all of the requirements placed on that function. The onset of a fault is called a failure event and is usually signaled by one or more alarm reports. The termination of a fault state is called a clear event.
Fault management is responsible for determining, from a variety of information sources, the root cause of a fault, and for its repair. In certain cases, the root cause of a fault might be in a connecting network. In such cases, fault management is responsible for reporting the problem through appropriate channels.
Service assurance is the overall process of ensuring that the purchased level of service is delivered. The Element Management System (EMS) plays a key role in maintaining the health of both NEs and transmission facilities. This is done in conjunction with other systems, typically at the network management layer and service management layer. The EMS can be the primary repository of detailed history of NE-specific faults and events, technician action, and performance data.
The steps for successful fault management are:
1. Identify a problem by gathering data about the state of the network (polling and trap generation).
2. Restore any services that have been lost.
3. Isolate the cause, and decide if the fault should be managed.
4. Correct the fault if possible.
9.1.1 What the NE Provides
Currently deployed, intelligent NEs provide the management system with the following, which are required for effective fault management:
•Detection of the four main types of failure:
–Equipment failure—Detected through failure detection mechanisms built into the hardware, and through routine exercises and diagnostics.
–Software failure—Detected through failure of software checks, and through routine audits.
–Communication failure—Detected through defects in the incoming signal or outgoing signal characteristics. Defects include line coding errors, framing bit errors, parity errors, cyclic redundancy check errors, and addressing errors. Signal characteristics include optical or electrical power, analog signal-to-noise ratio, and deviation from required voltage or wavelength.
–Environmental failure.
•Notification of failure—NEs notify a management system when a failure occurs by generating an alarm report. The NE can also report a summary of current fault states, or replay its log of historical failures and clears.
•Notification of changes in operational state of the NE components—If a component of the NE is in a fault state, a management system should not receive further alarms, alerts, or scheduled performance data from that component.
Note CTM forwards northbound information and integrates with other third-party management systems to give options that are not directly available in CTM.
9.1.2 Fault Notification and Maintenance
Fault notification and maintenance can be proactive or reactive:
•Proactive notification—Where X contacts Y to query Y on potential problems in Y's domain.
•Reactive maintenance—Where Y notifies X of a problem regarding a service delivered from Y to X.
9.1.2.1 Proactive Maintenance
Automated detection tests and surveillance software enable rapid initiation of the repair process, sometimes even before customers have noticed a problem. This is called proactive maintenance and promotes customer satisfaction.
Proactive maintenance consists of functions and processes associated with the detection, analysis, isolation, and resolution of problems by means that are independent of customer trouble reports. The problems might be faults or degradations in equipment or transmission media.
The goals of proactive maintenance are to:
•Detect and fix service quality problems before the customer calls to establish a trouble report, or at least to start the repair process before the customer calls, thereby minimizing the time, as perceived by the customer, before service is restored.
•Maintain the transport network at a high level of quality by identifying the facilities that perform relatively poorly and rehabilitating them.
9.1.2.2 Reactive Maintenance
Reactive maintenance is required when a failure occurs. This type of problem can be time-consuming and costly. It requires accurate administration of trouble reports, rapid analysis and repair of service-affecting faults, and notifications to the customer of restoration of service, all of which also promote customer satisfaction.
9.1.3 Root Cause Analysis
The root cause is the most basic reason for an undesirable condition or problem that, if eliminated or corrected, would have prevented the problem from occurring. The outcome of the root cause analysis is not a restatement of the most obvious symptom, but is the result of a methodical analysis of the problem situation, leading to the most basic cause.
Root cause analysis captures additional information about defects for the purpose of identifying preventive actions. CTM includes advanced debugging features that capture additional information about defects.
9.2 Where Can I Get Information on Affected Services and Customers?
The first thing to do in fault management is to identify what services and which customers are affected by the fault. CTM provides a number of options for viewing this information. The following table describes where to obtain information on affected services and customers.
Table 9-1 Obtaining Information on Affected Services and Customers
Where
|
Description
|
For More Information, See
|
Dashboard
|
Shows useful alarm and NE information in one easily accessible location.
|
1.3.1 Dashboard, page 1-6
|
Tooltips
|
Visible when you position the cursor over a managed object (domain, group, subnetwork, NE, board, link, and so on). The tooltip displays additional information about the selected object.
|
—
|
Domain Explorer
|
The Domain Explorer is the home window and provides a logical view of the network plus alarm, connectivity, and operational status.
|
1.3.2 Domain Explorer, page 1-7
|
Subnetwork Explorer
|
Similar in appearance and function to the Domain Explorer. A key difference is that the Subnetwork Explorer provides a single-level grouping of NEs based on network partitions and subnetworks.
|
1.3.3 Subnetwork Explorer, page 1-18
|
Network Map
|
Displays a geographical layout of the network.
|
1.3.7 Network Map, page 1-25
|
Alarm Browser
|
Displays standing alarms and conditions in the managed domain that are assigned a severity level of critical, major, minor, or warning. It also shows cleared alarms that are not acknowledged.
|
Viewing the Alarm Browser
|
Alarm Log
|
The Alarm Log contains alarms that have transitioned from the Alarm Browser.
|
Viewing the Alarm Log
|
Syslog Viewer
|
The Syslog Viewer contains important faults and events in the form of syslog messages for the Cisco 7600 NE.
|
Viewing the Syslog Viewer
|
9.2.1 Viewing the Alarm Browser
The Alarm Browser displays standing alarms and conditions in the managed domain that are assigned a severity level of critical, major, minor, or warning. It also shows cleared alarms that are not acknowledged. The Alarm Browser and Alarm Log views provide a robust listing of all current and historical alarms and events. See Viewing the Alarm Log for information about the Alarm Log.
To display the Alarm Browser, select an NE, group, subnetwork, or domain node from the Domain Explorer, Subnetwork Explorer, Network Map, or NE Explorer; then, choose Fault > Alarm Browser (or click the Open Alarm Browser tool from the Dashboard).
Note•No alarms or events are generated in the Alarm Browser if Oracle shuts down.
•Refer to the appropriate NE documentation for a list of alarms supported on each NE. See 1.7.2 Related Cisco NE Documentation, page 1-48.
•For a list of MGX 8830, 8850, and 8880 alarms, see Appendix A of the Cisco Transport Manager Release 8.0 GateWay/CORBA User Guide and Programmer Manual.
•Use the toolbar icons to manage the alarm display. See Appendix A, "Icons and Menus Displayed in CTM" for an explanation of each toolbar icon.
The following table describes the fields in the Alarm Browser.
Table 9-2 Field Descriptions for the Alarm Browser Window
Field
|
Description
|
Alarm ID
|
Unique number that the system uses to identify a particular alarm.
|
Perceived Severity
|
Perceived severity of the selected alarm (critical, major, minor, or warning). The background color also indicates the severity, where:
•Red = Critical
•Orange = Major
•Yellow = Minor
•Blue = Warning
•Green = Cleared
Note For the ONS 15501, ONS 15530, and ONS 15540, alarms that are shown in the command-line interface (CLI) as informational are shown in CTM as warnings.
Note For the CRS-1 and XR 12000, alarms that are shown in the CLI as informational and notifications are shown in CTM as warnings.
|
Acknowledged
|
Whether the selected alarm has been acknowledged by the user. Values are Yes and No.
|
Note
|
Any notes that were entered for the selected alarm. If you choose Fault > Show Alarm Note (or click the Show Alarm Note tool), you can see the login name of the user who entered the note and the time when the note was entered.
|
Alias ID
|
Alias name of the NE.
|
Probable Cause
|
Probable cause of the selected alarm. Some possible values include:
•Not Applicable/Unknown—If no additional information is available
•Mismatch of equipment and attributes (MEA) alarm—For misconfigured pluggable port modules (PPMs)
•Link Layer Keep-Alive Failure—When the keepalive frame on the POS port is disabled and the port is shut down
•Bad Packet Count Exceeds Threshold—When the packets through the front port have CRC errored frames
•Auto-Negotiation Remote Failure Indication—When a remote Gigabit Ethernet port is shut down from a local port
|
Affected Object
|
Name of the object where the selected alarm occurred.
|
Module Name
|
Name of the module where the selected alarm occurred.
|
Physical Location
|
Physical location of the equipment where the selected alarm occurred, such as chassis, rack, subrack (shelf), slot, and port numbers.
|
CTM Alarm Time Stamp
|
Date and time when the alarm occurred on the server.
|
NE Alarm Time Stamp
|
Date and time when the alarm occurred on the NE.
|
Service Affecting
|
Whether the alarm or event is service affecting (SA). Values are:
•Yes—The alarm is service affecting
•No—The alarm is not service affecting
•N/A—No information is provided by the NE
|
CTM Clear Time Stamp
|
Date and time when the alarm was cleared on the server.
|
NE Clear Time Stamp
|
Date and time when the alarm was cleared on the NE.
|
Acknowledged Time Stamp
|
Date and time when the user acknowledged the selected alarm.
|
Acknowledged Username
|
Login name of the user who acknowledged the selected alarm.
|
Alarm Status
|
Status of the selected alarm (active or cleared).
|
Description
|
Additional information about the selected alarm. If there is no additional information, this field is blank.
|
Source ID
|
Name of the NE where the selected alarm occurred.
|
TL1 Direction
|
TL1 direction for RTRV-ALM-ALL and RTRV-COND-ALL TL1 commands and REPT^ALM/COND autonomous messages. Values are Receive or Transmit.
This field is blank for non-TL1 alarms.
|
TL1 Location
|
TL1 location for RTRV-ALM-ALL and RTRV-COND-ALL TL1 commands and REPT^ALM/COND autonomous messages. Values are Near End or Far End.
This field is blank for non-TL1 alarms.
|
9.2.2 Filtering Data in the Alarm Browser
Step 1 In the Alarm Browser window, choose File > Filter (or click the Filter Data tool). The Filter dialog box opens.
Step 2 Specify the filter parameters described in the following table.
Step 3 After making your selections, click OK to run the filter.
Table 9-3 Field Descriptions for the Alarm Browser Filter Dialog Box
Tab
|
Description
|
CTM Alarm Time (time zone)
|
Allows you to filter alarm data for a specified time period, ranging from the past hour to the past 6 months. Additionally, you can click the User Specified radio button to specify an exact filter starting and ending time by month, day, year, and hour. The time zone can be Greenwich Mean Time (GMT), a user-defined offset from GMT, or local time, depending on what is specified in the User Preferences dialog box. If you want to filter alarms and the time period is not important, click No Time Specified. Click From Now Onward to set the filter time to start immediately and continue until you change filter parameters.
|
Source ID
|
Allows you to move NEs back and forth between the list of available source IDs and selected source IDs and then run the alarm filter. If you have the appropriate user permissions, you can filter CTM EMS alarms by selecting CTM and adding it to the Selected Source ID list.
If more than 100 NEs are selected, the Source ID tab dims and all devices are included in the filter criteria you specify.
Note Use the scroll bars at the bottom and right side of the lists to display all options in the lists.
|
Module Name
|
Allows you to specify which module types you want to include in the alarm filter. The modules displayed depend on the NE selection in the Domain Explorer tree when the Alarm Browser is opened. Use the Add and Remove buttons to filter the display to specific modules. The Alarm Browser displays alarms for modules listed under Selected Module Name.
Note Use the scroll bars at the bottom and right side of the lists to display all options in the lists.
|
Affected Object
|
Allows you to specify which objects you want to include in the alarm filter. All objects are displayed regardless of the NE selected in the Domain Explorer tree when the Alarm Browser is opened. Use the Add and Remove buttons to filter the display to specific objects. The Alarm Browser displays alarms for objects listed under Selected Affected Object. To filter NE-specific EMS alarms, include CTM in the Selected Affected Object list.
Below the Available Affected Object list is a text field where you can enter characters to search quickly for available objects. The text field accepts an asterisk (*) as a wildcard character. Alarms that do not match the search criteria are not displayed.
Note Use the scroll bars at the bottom and right side of the lists to display all options in the lists.
|
PS
|
Allows you to filter alarm data based on perceived severity (Critical, Major, Minor, and Warning) and alarm status (Active and Cleared). You can filter on acknowledged alarms, unacknowledged alarms, or both. You can also filter service-affecting alarms, nonservice-affecting alarms, and/or alarms where the service-affecting status is not known.
|
Physical Location
|
Allows you to filter alarm data based on the physical location of an NE or its components. To view the tab, an NE must be selected in the Domain Explorer tree. The filters that are available depend on the NE that is selected when the Alarm Browser is opened. For example, if you select an ONS 15800, you can filter alarms by rack, subrack (shelf), and slot. If you select a CTC-based NE, you can filter data by shelf, slot, and port.
Note The Physical Location tab is not supported for MGX NEs.
|
ID
|
Allows you to filter alarm data based on a specific starting and ending alarm ID. Check the Disregard All Other Filter Criteria check box to base the filter on only the starting and ending alarm ID.
|
NE Alarm Time (time zone)
|
Allows you to filter alarm data for a specified time period, ranging from the past hour to the past 6 months. Additionally, you can click the User Specified radio button to specify an exact filter starting and ending time by month, day, year, and hour. The time zone can be GMT, a user-defined offset from GMT, or local time, depending on what is specified in the User Preferences dialog box. If you want to filter alarms and the time period is not important, click No Time Specified. Click From Now Onward to set the filter time to start immediately and continue until you change filter parameters.
|
9.2.3 Adding Alarm Notes
Use the Alarm Note dialog box to read any comments that have been entered for the selected alarm and to enter additional comments. In the Alarm Browser window, select an alarm and choose Fault > Show Alarm Note (or click the Show Alarm Note tool). The following table provides descriptions.
Table 9-4 Field Descriptions for the Alarm Note Dialog Box
Field
|
Description
|
Note
|
Provides space to type comments about the selected alarm. To add your comments to the previous comments, click the Append radio button. To overwrite the previous comments, click Replace. To delete the comments, click Delete.
Note You can enable and disable the ability to overwrite or delete alarm notes in the Control Panel > UI Properties pane.
|
History
|
Comments that were entered by previous users. This field also shows the login name of the user who entered the note and the time when the note was entered.
|
9.2.4 Viewing the Alarm Log
The Alarm Log contains alarms that have transitioned from the Alarm Browser. Cleared alarms are transitioned when you acknowledge them or when automatic acknowledgment has been enabled (in the Control Panel > UI Properties pane). In addition, the Alarm Log shows a history of cleared and acknowledged alarms and all transient conditions (also known as events or autonomous nonalarmed messages). Events are placed directly into the Alarm Log; they do not appear in the Alarm Browser. By default, the Alarm Log shows alarm and event information that occurred during the last 4 hours.
To view the Alarm Log, select a node in the Domain Explorer tree and choose Fault > Alarm Log (or click the Open Alarm Log tool). The following table provides descriptions.
Table 9-5 Field Descriptions for the Alarm Log Window
Field
|
Description
|
ID
|
Unique number that the system uses to identify a particular alarm or event.
|
Alias ID
|
Alias name of the NE.
|
Affected Object
|
Name of the object where the selected alarm or event occurred. For NE-specific alarms, the affected object field displays "CTM." For the non-NE specific alarms:
•Maximum number of login attempts exceeded alarm—The affected object field displays the user ID associated with the alarm.
•CTM self-monitor alarm—The affected object field displays the threshold parameter associated with the alarm.
|
Module Name
|
Name of the module where the selected alarm or event occurred.
|
Physical Location
|
Physical location of the equipment where the selected alarm or event occurred, such as rack, subrack (shelf), slot, and port numbers.
|
Probable Cause
|
Probable cause of the selected alarm or event.
|
Perceived Severity
|
Severity of the alarm before it was cleared. Perceived Severity is listed as:
•Critical (CR)
•Major (MJ)
•Minor (MN)
•Warning (WR)
•Indeterminate (IN)
The background color of the column indicates the alarm status, where:
•Green = Cleared alarms
•Purple = Indeterminate events
Note Indeterminate events are transient events that do not have a severity indicated by the source NE. Indeterminate events do not have a cleared condition.
|
Service Affecting
|
Whether the alarm or event is service affecting. Values are:
•Yes if the alarm is service affecting
•No if the alarm is not service affecting
•N/A if no information is provided by the NE
|
CTM Time Stamp (time zone)
|
Date and time when the alarm or event occurred on the CTM server.
|
CTM Clear Time (time zone)
|
Date and time when the alarm was cleared on the CTM server.
|
CTM Duration
|
Amount of time required to clear an alarm (CTM clear time - CTM time) in dddd:hh:mm:ss format.
|
NE Time Stamp (time zone)
|
Date and time when the alarm or event occurred on the NE.
|
NE Clear Time (time zone)
|
Date and time when the alarm or event was cleared on the NE.
|
Description
|
Brief description of the selected alarm or event. If no description is entered, this field is blank.
|
Acknowledged Username
|
Login name of the user who acknowledged the alarm or event.
Note If the alarm acknowledgement is set to Automatic and you can manually acknowledge an alarm, the Acknowledged Username is not overwritten when the alarm clears.
|
Acknowledged Time
|
Date and time when the alarm or event was acknowledged.
Note If the alarm acknowledgement is set to Automatic and you can manually acknowledge an alarm, the Acknowledged Time is not overwritten when the alarm clears.
|
Note
|
Any notes that were entered for the selected alarm or event. This field also shows the login name of the user who entered the note and the time stamp when the note was entered.
|
Source ID
|
Name of the NE or EMS where the selected alarm or event occurred.
|
TL1 Direction
|
TL1 direction for RTRV-ALM-ALL and RTRV-COND-ALL TL1 commands and REPT^ALM/COND autonomous messages. Values are Receive or Transmit.
This field is blank for non-TL1 alarms.
|
TL1 Location
|
TL1 location for RTRV-ALM-ALL and RTRV-COND-ALL TL1 commands and REPT^ALM/COND autonomous messages. Values are Near End or Far End.
This field is blank for non-TL1 alarms.
|
9.2.5 Filtering Data in the Alarm Log
By default, the Alarm Log shows alarm and event information that occurred during the last 4 hours. Use the drop-down menu to the right of the time-based Filter Data tool to filter event data for various time periods.
Step 1 In the Alarm Log window, choose File > Filter (or click the Filter Data tool). The Filter dialog box opens.
Step 2 Specify the filter parameters described in the following table.
Step 3 After making your selections, click OK to run the filter.
Table 9-6 Field Descriptions for the Alarm Log Filter Dialog Box
Tab
|
Description
|
CTM Time Stamp (time zone)
|
Allows you to filter alarm and event data for a specified time period, ranging from the past hour to the past 6 months. Additionally, you can click the User Specified radio button to specify an exact filter starting and ending time by month, day, year, and hour. The time zone can be GMT, a user-defined offset from GMT, or local time, depending on what is specified in the User Preferences dialog box. If you want to filter alarms and events and the time period is not important, click No Time Specified. Click From Now Onward to set the filter time to start immediately and continue until you change filter parameters.
|
Source ID
|
Allows you to move NEs back and forth between the list of available source IDs and selected source IDs and then run the filter. If you have the appropriate user permission, you can filter CTM EMS alarms and events by selecting CTM and adding it to the Selected Source ID list.
If more than 100 NEs are selected, the Source ID tab dims and all devices are included in the filter criteria you specify.
Note Use the scroll bars at the bottom and right side of the lists to display all options in the lists.
|
Module Name
|
Allows you to specify which modules you want to include in the filter. The modules displayed depend on the NE selection in the Domain Explorer tree when the Alarm Log is opened. Use the Add and Remove buttons to filter the display to specific modules. The Alarm Log displays events for modules listed under Selected Module Name.
Note Use the scroll bars at the bottom and right side of the lists to display all options in the lists.
|
Affected Object
|
Allows you to specify which objects you want to include in the filter. The objects displayed depend on the NE selection in the Domain Explorer tree when the Alarm Log is opened. Use the Add and Remove buttons to filter the display to specific objects. The Alarm Log displays events for entities listed under Selected Affected Object.
To filter NE-specific EMS alarms, select CTM and add it to the Selected Affected Object list.
Below the Available Affected Object list is a text field where you can enter characters to search quickly for available objects. The text field accepts an asterisk (*) as a wildcard character. Alarms that do not match the search criteria are not displayed.
Note Use the scroll bars at the bottom and right side of the lists to display all options in the lists.
|
PS
|
Allows you to filter data based on the perceived severity (PS) of the alarm or event. Additionally, you can filter service-affecting alarms and events, nonservice-affecting alarms and events, and/or alarms and events where the service-affecting status is not known.
|
Physical Location
|
Allows you to filter data based on the physical location of an NE or its components. To view the tab, an NE must be selected in the Domain Explorer tree. The filters that are available depend on the NE selected. For example, if you select an ONS 15800, you can filter events by rack, subrack (shelf), and slot. If you select a CTC-based NE, you can filter data by shelf, slot, and port.
|
ID
|
Allows you to filter data based on alarm or event ID. Enter a starting ID and an ending ID; then, run the filter to see only the alarms or events that occurred within the specified range of IDs. Check the Disregard All The Other Filter Criteria check box to ignore all other filter specifications. In addition, you can filter the view to only alarms, only events, or both alarms and events.
|
NE Alarm Time (time zone)
|
Allows you to filter alarm data for a specified time period, ranging from the past hour to the past 6 months. Additionally, you can click the User Specified radio button to specify an exact filter starting and ending time by month, day, year, and hour. The time zone can be GMT, a user-defined offset from GMT, or local time, depending on what is specified in the User Preferences dialog box. If you want to filter alarms and the time period is not important, click No Time Specified. Click From Now Onward to set the filter time to start immediately and continue until you change filter parameters.
|
9.2.6 Viewing the Syslog Viewer
Cisco 7600 NEs report important faults and events in the form of syslog messages. CTM provides services to collect, process, and store all the syslog messages sent by devices managed by CTM. CTM also informs applications when devices send a syslog message to the application.
Note The Syslog Viewer displays syslog messages for the Cisco 7600 NE only.
To open the syslog viewer, choose Fault > Syslog Viewer from the Domain Explorer.
Table 9-7 Syslog Viewer Fields and Descriptions
Field
|
Description
|
ID
|
Unique number that the system uses to identify a particular syslog message.
|
Source ID
|
Name of the NE where the selected syslog message occurred.
|
Facility
|
Facility of the error or event (daemon, kernel, VSHD, or other facility).
|
Severity
|
Severity of the selected alarm (critical, major, minor, warning, or intermediate). The background color also indicates the severity, where:
•Red = Critical
•Orange = Major
•Yellow = Minor
•Blue = Warning
•Green = Intermediate
|
Mnemonic
|
Text string that uniquely describes the system message.
|
Message
|
Text of the syslog message.
|
Time Stamp
|
Date and time when the syslog message occurred on the server.
|
9.2.7 Filtering Data in the Syslog Viewer
Step 1 In the Syslog Viewer window, choose File > Filter (or click the Filter Data tool). The Filter dialog box opens.
Step 2 Specify the filter parameters described in the following table.
Step 3 After making your selections, click OK to run the filter.
Table 9-8 Field Descriptions For the Syslog Viewer Filter Dialog Box
Tab
|
Description
|
Time Stamp
|
Allows you to filter alarm data for a specified time period, ranging from the past hour to the past 6 months. Additionally, you can click the User Specified radio button to specify an exact filter starting and ending time by month, day, year, and hour. The time zone can be GMT, a user-defined offset from GMT, or local time, depending on what is specified in the User Preferences dialog box. If you want to filter alarms and the time period is not important, click No Time Specified. Click From Now Onward to set the filter time to start immediately and continue until you change filter parameters.
|
Source ID
|
Allows you to move NEs back and forth between the list of available source IDs and selected source IDs and then run the syslog filter. If you have the appropriate user permissions, you can filter CTM EMS syslog messages by selecting CTM and adding it to the Selected Source ID list.
If more than 100 NEs are selected, the Source ID tab dims and all devices are included in the filter criteria you specify.
|
Sev
|
Allows you to view syslog message data based on the syslog message severity.
|
ID
|
Allows you to filter syslog message data based on a specific starting and ending message ID. Check the Disregard All Other Filter Criteria check box to base the filter on only the starting and ending message ID.
|
9.2.8 Diagnostic Center—MGX Voice Gateway Devices
The Diagnostic Center provides a hierarchical representation of the network's MGX nodes, cards, lines, and ports. Associated information about a selected NE is displayed in table format in the right pane of the Diagnostic Center window. See Figure 9-1.
Each NE managed by CTM has its own attributes and fits into the network's physical or logical hierarchy. The Diagnostic Center displays the NEs in a hierarchical format based on either a physical or logical relationship among the various NEs. Networks are displayed at the root level of the component tree; nodes are displayed beneath the networks in a parent/child relationship.
The Diagnostic Center also displays informational messages in a multiline text display; other types of messages are displayed in response to user actions.
You can launch the Diagnostic Center for an NE, group, or the domain. Select an MGX node in the Domain Explorer window and choose Fault > MGX Voice Gateway > Diagnostic Center.
The Diagnostic Center allows you to:
•Monitor real-time counters and statistics.
•Extend support for the Node Resync process to allow for two different levels.
•Configure bit error rate test (BERT) to test the integrity of physical lines, ports, and logical paths.
•Conduct line and path loopback tests.
•Conduct node manageability tests.
•Verify that all the nodes in the network are managed correctly by CTM.
•Identify general network problems.
•Provide success rates, failure rates, and throughput of CTM to protocols such as FTP and SNMP.
•Create a trouble ticket that extracts all the information from the History panel.
•Access fault management capabilities in the form of diagnostic tests for connections.
•Perform general test operations such as up and down connections, connection loopback, and round-trip delay.
Figure 9-1 Diagnostic Center
9.3 What Fault Information Can I See?
An alarm is represented by a notification from a managed NE that a certain condition has just occurred. These alarms usually represent error conditions on NEs. Each alarm is associated with the NE for which it provides notification, and an NE can have a number of alarms related to itself at any time.
Each NE shown in the Domain Explorer tree has a corresponding alarm icon that indicates the highest severity alarm that affects the NE. Management domain nodes and group nodes have alarm icons that reflect the highest alarm condition of the NEs contained in the domain or group.
The user-defined Domain Explorer views have "bubble-up" alarm severity propagation and drill-down capabilities to isolate fault conditions and identify service-delivery impact.
9.3.1 How Are Alarms Displayed?
In the Diagnostic Center, Configuration Center, Chassis View, and Statistics Report, you can display the color legend for alarms by choosing Help > Color Legend.
The following table lists the color legend for MGX alarms in the Diagnostic Center, Configuration Center, Chassis View, and Statistics Report.
Table 9-9 Color Legend for MGX Alarms
Color
|
Alarm State
|
Green
|
Clear
|
Yellow
|
Minor
|
Orange
|
Major
|
Red
|
Critical
|
Gray
|
Unmanaged
|
Blue
|
Unknown
|
Purple
|
LineHasLoopback
|
You can set the Alarm Browser or Alarm Log to display full background color for the entire selected row. The color corresponds to the alarm status and severity. In the Domain Explorer window, choose Edit > User Preferences. The User Preferences dialog box opens. On the FM Preferences tab, check the Color Entire Row in Table View check box.
In the Dashboard, Map Viewer, Alarm Browser, and Domain Explorer, the color of the border surrounding a component, or the background color, indicates the operational status of the component. When status changes, the border or the background color changes as indicated in the following table.
Table 9-10 Alarm Severity Colors for the Dashboard, Map Viewer, Alarm Browser, and Domain Explorer
Color
|
Severity
|
Meaning
|
Description
|
Green
|
Cleared
|
Component is active.
|
The component is operating normally.
|
Yellow
|
Minor
|
Minor failure.
|
The component is down; both administrative and operational values are down. This does not necessarily indicate a fault condition; the component might be disabled.
|
Orange
|
Major
|
Component is down.
|
Administrative status is up and operational value is down.
|
Red
|
Critical
|
Component failed.
|
Physical hardware failure.
|
Cyan (blue-green)
|
Warning
|
Interface is dormant.
|
The interface cannot pass packets, but is in a pending state waiting for some external event to place it in the up state. The interface might have packets to transmit before establishing a connection to a remote system, or a remote system might be establishing a connection to the interface.
When the pending event occurs, the interface changes to the up state.
|
9.3.1.1 Understanding How CTM Displays the Affected Object Field
CTM displays the Affected Object field in the Alarm Browser and Alarm Log windows in the same way as for the Interface field in PM tables. See 10.4.9 Understanding How CTM Displays the Interface Field, page 10-42.
Note the following exceptions:
•For the ONS 1580x, the Affected Object field is set to Equipment.
•For the ONS 155xx, the Affected Object field can be:
–wavepatch—The alarm is related to the optical client side of the card.
–wave—The alarm is related to the optical connection with the OADM (filter) card.
–The actual interface object, which changes depending on the card. For example, WaveEthernetPhy for a 10 Gb card, EsconPhy for an ESCON card, and so on.
9.3.2 Suppressing Alarms
Alarm suppression is useful when the NE is under maintenance.
Caution If multiple CTC or TL1 sessions are open, alarms in all other open sessions are also suppressed.
9.3.2.1 Suppressing Alarms at the Card Level—CTC-Based NEs
Step 1 In the Domain Explorer window, select a CTC-based NE and choose Configuration > NE Explorer.
Step 2 In the NE Explorer tree, click the specific card.
Step 3 In the card slot property sheet of the NE Explorer window, click the Identification tab.
Step 4 Check the Suppress Alarms check box.
Step 5 Click Apply.
Note For the ONS 15800, ONS 15801, and ONS 15808 NEs, you can suppress alarms at the card level by tagging the card as Out of Service. See 5.13.3 Tagging an ONS 15800, ONS 15801, or ONS 15808 Module as Out of Service, page 5-311.
9.3.2.2 Suppressing Alarms at the Node Level—CTC-Based NEs
Step 1 In the Domain Explorer window, select a CTC-based NE and choose Configuration > NE Explorer.
Step 2 In the node property sheet of the NE Explorer window, click the Alarm tab.
Step 3 Click the Alarm Behavior subtab.
Step 4 Check the Suppress Alarms check box.
Step 5 Click Apply.
9.4 Is the Service Working?
Network devices report symptoms of problems by generating events. An event in this context is a message indicating that a device or application in your network has discovered something of note. The network devices generate many types of events automatically. In addition, you can use thresholds to define or modify the conditions under which events are generated. A threshold is a trigger, set up on a continuous data stream, that is a point of interest that generates events when that point is satisfied.
The events generated need to be analyzed to determine whether they represent a fault condition or a problem in your network.
It is important to generate events when there is a problem. It is also important to limit the number of events generated to prevent an excessive load on the network. CTM performs a number of self-monitoring tasks where threshold limits can be set. The threshold limits are set in the Self Monitor table. (See 10.3.13 Using the Self Monitor Table, page 10-22.) If a threshold is crossed, an EMS alarm is generated.
You can obtain information regarding how the system is performing and how long certain tasks are taking to complete by selecting Administration > Control Panel, then Alarm Configuration > Threshold EMS Alarms or Alarm Configuration > Nonthreshold EMS Alarms. (See Setting Up and Viewing Alarm Configuration Parameters.) By monitoring this data, you can identify potential system problems before they become critical in the operation of the EMS. Associated with each parameter that is monitored are three alarm thresholds. The administrator can set a minor, major, and critical threshold value for each parameter. If any of these thresholds are crossed, then an alarm will be raised to provide notification of the situation.
Threshold alarms are raised when their limit exceeds the value set for critical, major, minor, or warning thresholds. For example, you can set threshold alarms for disk usage for 90%, 80%, 70%, and 60%, meaning a warning alarm is raised when the disk becomes 61% full and a critical alarm is raised when the disk becomes 91% full. The server checks these parameters at every polling interval that is set in the Poll Frequency field.
Nonthreshold alarms do not have an alarm threshold. Instead, nonthreshold alarms occur when a condition occurs, such as loss of connectivity to an NE. Use the Nonthreshold EMS Alarms tab to set the severity level (critical, major, minor, or warning) for which a nonthreshold alarm should be raised when that condition occurs.
Caution Changing the EMS alarm severities can affect the alarm status seen by listeners on the EMS's OSS interfaces.
The following sections provide information on NEs:
•Locating Alarms
•Setting Up and Viewing Alarm Configuration Parameters
These tasks allow you to manage the alarm profiles features:
•Creating Alarm Profiles—CTC-Based NEs
•Applying Alarm Profiles—CTC-Based NEs
•Managing Alarm Profiles—CTC-Based NEs
This task allows you to mark a service as critical for process monitoring purposes:
•Using the Recovery Properties Pane
This task allows you to poll counters and plot the polling results.
•Using the Statistics Panel—MGX Voice Gateway Cards
These tasks are used to diagnose NEs with the Diagnostic Center:
•Displaying the Status of All Nodes in the Network—MGX Voice Gateway Cards
•Displaying the Status of a Node—MGX Voice Gateway Cards
•Displaying the Status of MGX Voice Gateway Cards
•Displaying the Status for Lines or Ports—MGX Voice Gateway Cards
•Displaying the Status for Lines in a Loopback—MGX Voice Gateway Cards
These tasks are used to diagnose the voice cards for both Voice Service Module (VXSM) and Voice Interworking Service Module (VISM-PR):
•Displaying the Paths in Loopback for the VXSM-OC3 Card
•Displaying the Status for a Bit Error Rate Test—VXSM and VISM-PR Cards
•Monitoring and Diagnosing MGC Communication Protocols on VXSM Cards
•Monitoring and Diagnosing the Status for VISM-PR Features
These tasks are used to manage diagnostics:
•Managing Diagnostics for DPNSS and DLC—VXSM Cards
•Managing Diagnostics for SS7 Signaling Components—VXSM Cards
•Managing Diagnostics for VXSM Card Resource Overload and Utilization
These tasks are used to monitor real-time statistics:
•Collecting Real-Time Performance and Health Statistics—PXM45 and PXM1E Cards
•Monitoring SNMPv3 Real-Time Counters
This task is used to diagnose connections:
•Diagnosing Connections—MGX Voice Gateway Cards
These tasks describe the BERT used to verify the integrity of an NE:
•Configuring a Bit Error Rate Test
•Stopping a Bit Error Rate Test
•Modifying a Bit Error Rate Test
•Displaying a Bit Error Rate Test
9.4.1 Locating Alarms
The Alarm Browser has a specific selection context, which means that it displays alarm information that corresponds to the view where it was launched. If you launch the Alarm Browser from the management domain node, the browser shows all NE alarms and all EMS alarms (if you have permission to see EMS alarms). If you launch the Alarm Browser from a group or NE node, the browser shows only NE alarms for that group or NE node. If you launch the Alarm Browser from the Dashboard, the browser shows all NE alarms for the domain.
You can locate the equipment for an existing alarm from the Alarm Browser.
Step 1 In the Domain Explorer, choose Fault > Alarm Browser (or click the Open Alarm Browser tool).
Step 2 In the Alarm Browser, click an alarm condition and choose Fault > Locate Alarm/Event (or click the Locate Alarm/Event Through NE Explorer tool). The NE Explorer opens and displays the property sheet of the alarmed equipment.
Note Refer to the appropriate NE documentation for a list of alarms supported on each NE. See 1.7.2 Related Cisco NE Documentation, page 1-48.
9.4.2 Setting Up and Viewing Alarm Configuration Parameters
Use the Alarm Configuration pane to configure and view alarm severities for system parameters.
Caution Changing the EMS alarm severities can affect the alarm status seen by users on the EMS's OSS interfaces.
Step 1 In the Domain Explorer window, choose Administration > Control Panel.
Step 2 Click Alarm Configuration to open the Alarm Configuration pane. Table 9-11 provides descriptions.
Step 3 In the Nonthreshold EMS Alarms tab, you can select the severity level that will be assigned to the nonthreshold alarm parameter.
Note Nonthreshold alarms do not have an alarm threshold. These types of alarms occur when an error condition occurs, such as loss of connectivity to an NE. Use the Nonthreshold EMS Alarms tab to set the severity level for which a nonthreshold alarm should be raised when that condition occurs.
Step 4 After making your selections, click Save.
Note•If an alarm is outstanding when you disable it, the system clears the alarm.
•If an alarm has been manually cleared in the Alarm Browser and a clear alarm is sent, the request will still be processed but it will not have any impact on the alarm.
•To recover from an alarm condition, see Appendix K, "Troubleshooting" for information.
•All alarms on an NE are cleared when the NE is marked as Out of Service except for the following MGX NE alarms:
–Sync-Up has not Started yet
–Currently in Sync-Up
–Partial Sync-Up
–Sync-Up Failed
–Server in Partial Sync-Up
–Server Sync-Up Failed
–FTP Transfer failure
Table 9-11 Field Descriptions for the Alarm Configuration Pane
Field
|
Description
|
Threshold EMS Alarms Tab
|
Poll Frequency
|
Threshold alarms are raised when their limit exceeds the value set for critical, major, minor, or warning thresholds. For example, you can set threshold alarms for disk usage for 90%, 80%, 70%, and 60%, meaning a warning alarm is raised when the disk becomes 61% full and a critical alarm is raised when the disk becomes 91% full. The server checks these parameters at every polling interval that is set in the Poll Frequency field.
|
Parameter Name
|
Name of the parameter:
•Base Circuit Creation Time (seconds)—Time it takes to create a base circuit.
•CPU Usage (%)—Percentage of CPU time used for executing user, system, and I/O tasks.
•Circuit Creation Time Per Hop (seconds)—Time it takes for CTM to create an end-to-end circuit.
•Config Resynch Time (seconds)—Time it takes for CTM to collect alarm and inventory information from the NE.
•Disk Usage (%)—Percentage of disk space used in a particular partition. CTM database and partitions are monitored separately.
•Memory Usage RAM (%)—Percentage of RAM memory used for all system processes.
•Memory Usage SWAP (%)—Percentage of SWAP memory used for all system processes.
•NE Synch Time (seconds)—Time it takes to synchronize the CTM server with the NEs.
•PM Collection Time (seconds)—Time it takes to collect PM data for an NE. It is the sum of the time it takes to read the PM data and the time it takes to update the database.
•Prune Time 15 min PM (seconds)—Time it takes to prune 15-minute PM data.
•Prune Time 1 day PM (seconds)—Time it takes to prune 1-day PM data.
•Prune Time Audit Log (seconds)—Time it takes to prune Audit Log data.
•Prune Time Audit Trail Log (seconds)—Time it takes to prune Audit Trail Log data.
•Prune Time Error Log (seconds)—Time it takes to prune Error Log data.
•Prune Time FM (seconds)—Time it takes to prune FM data.
•Prune Time Job Monitor (seconds)—Time it takes to prune job monitor data.
•Prune Time Purge NE (seconds)—Time it takes to prune NE purge data.
•Prune Time Server Monitor (seconds)—Time it takes to prune server monitor data.
|
Enable
|
Whether or not the corresponding parameter in the Parameter Name column is enabled (checked) or disabled (unchecked). When checked, it enables monitoring for the selected parameter. If an EMS threshold alarm is outstanding when you disable monitoring, CTM clears the alarm.
|
Critical
|
Amount of time, in minutes, that must elapse before triggering a critical alarm.
|
Major
|
Amount of time, in minutes, that must elapse before triggering a major alarm.
|
Minor
|
Amount of time, in minutes, that must elapse before triggering a minor alarm.
|
Nonthreshold EMS Alarms Tab
|
Parameter Name1
|
Name of the parameter:
•A critical process is hanging; CTM will be shut down in 5 minutes.
•A CTM process is hanging or terminated.
•Alarm resync unsuccessful—The alarm resynchronization could not be completed on the node.
•Backoff failed—The backoff failed for connection endpoints. This alarm is generated when CTM is managing MGX NEs.
•Card lost trap—A controller card or service card is lost. This alarm triggers a card resynchronization for the service card or a node resynchronization for the controller card. This alarm is generated when CTM is managing MGX NEs.
•Communication through secondary IP address.
•Config resync unsuccessful—CTM could not synchronize with the node.
•Communication error in CORBA—CORBA communication failed. This alarm is generated when CTM is managing MGX NEs.
•Communication error in ILOG—An interprocess communication failure occurred. This alarm is generated when CTM is managing MGX NEs.
•Communication error in shared memory—A shared memory read/write error occurred. This alarm is generated when CTM is managing MGX NEs.
•Configuration error—A process detects a configuration error. This alarm is generated when CTM is managing MGX NEs.
•CTM failed authentication by NE—This is a major alarm. An incorrect username or password was provided for login. This alarm applies to CTC-based NEs and is cleared when you provide the correct username or password and mark the NE as Out of Service, then as In Service.
•Currently in sync-up—The node is currently synchronizing. This alarm is generated when CTM is managing MGX NEs. This alarm is cleared automatically after the NE synchronization has been completed. If NE synchronization fails, a Partial Sync-Up or Sync-Up Failed alarm is generated. No alarm is generated if the NE synchronization succeeds. For MGX NEs, this alarm is not cleared when the NE is marked as Out of Service.
•Database error, DELETE failed—The DELETE database operation failed. This alarm is generated when CTM is managing MGX NEs.
•Database error, SELECT failed—The SELECT database operation failed. This alarm is generated when CTM is managing MGX NEs.
|
Parameter Name (continued)
|
•FTP file size mismatch—The FTP file size does not match. This alarm is generated when CTM is managing MGX NEs.
•FTP retry exceeded—The FTP operation is for configuration upload. If the number of FTP retries exceeds the maximum number of retries allowed, the configuration upload file is not loaded. This failure causes the node resynchronization to fail or to complete only partially. This alarm is generated when CTM is managing MGX NEs.
•FTP session timeout—A node timed out on an FTP session request. This alarm is generated when CTM is managing MGX NEs.
•FTP transfer failed—The FTP file transfer failed. This alarm is generated when CTM is managing MGX NEs.
•FTP transfer failure—The FTP file transfer failed. This alarm is generated when CTM is managing MGX NEs. For MGX NEs, this alarm is not cleared when the NE is marked as Out of Service.
•FTP transfer timeout—A node timed out on an FTP request. This alarm is generated when CTM is managing MGX NEs.
•Initialization error—A process cannot initialize itself or one of its subsystems. This alarm is generated when CTM is managing MGX NEs.
•Loss of communication—This is a critical alarm. CTM cannot communicate with the node, possibly because the node was disconnected from the network. This alarm applies to all NEs and is cleared when CTM regains connectivity to the NE.
•Maximum login attempts exceeded—CTM raises this alarm when you try to connect to the CTM server several times with the wrong username and password. The number of failed retries is a parameter in the Control Panel. To avoid this alarm, change the value of the Max Retries field in the Control Panel > Security Properties > CTM Security tab.
•Memory auto or manual backup failure—This is a minor alarm. Memory backup on the node failed. This alarm applies to CTC-based NEs and is cleared when a subsequent memory backup succeeds.
•Minus 2 trap—When the node generates many traps, the EMS fails to receive all of them. A Minus 2 trap is received and triggers a node resynchronization to upload the latest configuration. This alarm is generated when CTM is managing MGX NEs.
•NTS registration failed—Trap registration failed. This alarm is generated when CTM is managing MGX NEs.
•NTS trap loss—Traps have been lost. This alarm is generated when CTM is managing MGX NEs.
•Partial sync-up—The node is partially synchronized. This alarm is generated when CTM is managing MGX NEs. The alarm is cleared automatically when an NE synchronization is initiated. For MGX NEs, this alarm is not cleared when the NE is marked as Out of Service.
•PM error—This alarm is generated when CTM is managing MGX NEs.
•PM fail EMS alarm—This is a warning alarm. CTM cannot retrieve performance statistics from the node even though robust PM data collection is enabled and the node has PM buckets that were not retrieved. This alarm applies to CTC-based NEs and is cleared when a subsequent PM collection cycle succeeds. If PM collection is unsuccessful after subsequent retries, the FAIL alarm is cleared and the PM lost EMS alarm is generated.
|
Parameter Name (continued)
|
•PM lost EMS alarm—This is a major alarm. CTM cannot retrieve performance statistics from the node when robust PM data collection is not enabled or when robust PM data collection is enabled but the node has overwritten the PM buckets that were not retrieved. This alarm applies to CTC-based NEs and is cleared when the user clears it manually from the Alarm Browser.
•PoS port shut down. L2 topology in wrapped state—This alarm is generated when one or more PoS ports related to an L2 topology are shut down. This alarm is cleared when all the PoS ports related to the topology are enabled.
•Process restarted —This alarm is generated when CTM is managing MGX NEs.
•Server in partial sync-up—The server is partially synchronized.
•SNMP retry exceeded—The number of SNMP retries exceeds the maximum number of retries allowed. The operation fails. If the SNMP operation is for configuration upload, this failure causes the node resynchronization to fail or to complete only partially. This alarm is generated when CTM is managing MGX NEs.
•SNMP throttle error—Throttling limits have been reached for GET or SET requests on a node. This alarm is generated when CTM is managing MGX NEs.
•SNMP timeout—A node timed out on an SNMP request. This alarm is generated when CTM is managing MGX NEs.
•Server sync-up failed—Server synchronization failed. This alarm is generated when CTM is managing MGX NEs. The alarm is cleared automatically when server synchronization succeeds. For MGX NEs, this alarm is not cleared when the NE is marked as Out of Service.
•Sync-up has not started yet—Synchronization has not started. This alarm is generated when CTM is managing MGX NEs. The Currently in Sync-Up alarm clears this alarm automatically. For MGX NEs, this alarm is not cleared when the NE is marked as Out of Service.
•Stats file error—The stats file contains an error. The error could be a checksum error, a file open error, or a file format error. This alarm is generated when CTM is managing MGX NEs.
•Stats file transfer error—The stats file cannot be collected from the switch because of problems with reachability, timeout, or resynchronization. This alarm is generated when CTM is managing MGX NEs.
•Sync-up failed—Node synchronization failed. This alarm is generated when CTM is managing MGX NEs. This alarm is also generated when the EMS fails to synchronize with a card or node. The alarm is cleared automatically when an NE synchronization is initiated. For MGX NEs, this alarm is not cleared when the NE is marked as Out of Service.
•SysObjectID of the NE does not match that of the Catalyst 6509.
•SysObjectID of the NE does not match that of the Cisco 7600.
•Template manager event.
•TFTP transfer timeout—A TFTP operation timed out. This alarm is generated when CTM is managing MGX NEs.
•The router is low on memory.
•Unknown error—A process experienced a critical internal error during resynchronization. This alarm is generated when CTM is managing MGX NEs.
•Upload file error—The configuration upload file contains an error. The error could be a checksum error or a file open error. This alarm is generated when CTM is managing MGX NEs.
|
Enable
|
Whether or not the corresponding parameter in the Parameter Name column is enabled (checked) or disabled (unchecked). When checked, it enables monitoring for the selected parameter. If an EMS nonthreshold alarm is outstanding when you disable monitoring, CTM clears the alarm.
|
Severity
|
Click the appropriate cell and select the alarm severity level from the available options (Critical, Major, Minor, or Warning) for each of the parameters listed in the Parameter Name column.
Note If an EMS alarm is outstanding when you change its severity level, the outstanding alarm's severity level remains the same and the new severity level takes effect the next time the alarm is raised.
|
9.4.3 Creating Alarm Profiles—CTC-Based NEs
Use the Create Alarm Profile dialog box to create new alarm profiles for CTC-based NEs.
Step 1 In the Domain Explorer tree, select a CTC-based NE and choose Configuration > NE Explorer.
Step 2 In the node properties sheet, click the Alarm tab.
Step 3 In the Profile subtab, click the Create button. The Create Alarm Profile dialog box opens. The following table provides descriptions.
Step 4 After making your selections, click OK.
Table 9-12 Field Descriptions for the Create Alarm Profile Dialog Box
Field
|
Description
|
Enter the Profile Name
|
Enter the name of the new alarm profile.
|
Condition
|
Condition of the alarm.
|
Severity
|
Select a severity for the new alarm from the list. Alarm severities include:
•Not Reported (NR)—A raise or clear of the condition is not sent to clients, but is tracked on the NE. You can retrieve a complete list of all raised conditions, including Not Reported as well as Not Alarmed, Critical, Major, and Minor, by using the RTRV COND TL1 command, or its equivalent.
•Not Alarmed (NA)—A raise or clear of the condition is sent to clients as a nonalarmed TL1 message (REPT EVT). The message has no severity and no service affecting flag.
•Minor (MN)—The alarm is a minor alarm.
•Major (MJ)—The alarm is a major alarm.
•Critical (CR)—The alarm is a critical, traffic-affecting alarm.
Note For critical, major, and minor alarms, a raise or clear of the condition is sent to clients as an alarmed message (REPT ALM). This message includes a service affecting flag, which is On or Off. If a normally service affecting condition is raised in a nonservice affecting situation (for example, the nontraffic-bearing side of a protect pair), the condition is sent as minor even if the profile says major or critical.
•UNSET—The value of this alarm corresponds to the value of the default alarm profile on the node. UNSET is useful when transferring alarm profiles between different versions of software.
•Transparent—The alarm behaves according to its parent object:
–If you set an alarm as Transparent on a port, it uses the card profile.
–If you set an alarm as Transparent on a card, it uses the node profile.
–If you sent an alarm as Transparent on a node, it uses the default profile.
|
9.4.4 Applying Alarm Profiles—CTC-Based NEs
The alarm profiles feature allows you to change default alarm severities by creating unique alarm profiles for individual CTC-based NEs. A profile can be applied to any node on the network. Alarm profiles must be stored on a node before they can be applied to a node, card, or port.
The two reserved profiles include the Default profile, which sets severities to standard Telcordia GR-253 settings, and the Inherited profile, which sets all alarm severities to transparent (TR). If an alarm has an Inherited profile, it inherits (copies) its severity from the same alarm at the next level. For example, a card with an Inherited alarm profile copies the severities used by the node that contains the card. The Inherited profile is not available at the node level.
In the card view, the Alarm Behavior subtab displays the alarm profiles of the selected card. In the node view, the Alarm Behavior subtab displays the alarm profiles for the node. Alarms form a hierarchy. A node-level alarm profile applies to all cards in the node, except those that have their own profiles. A card-level alarm profile applies to all ports on the card, except those that have their own profiles.
At the node level, apply profile changes on a card-by-card basis or set a profile for the entire node. At the card level, apply profile changes on a port-by-port basis or set the profiles for all ports on that card simultaneously.
Step 1 In the Domain Explorer window, click a CTC-based NE and choose Configuration > NE Explorer.
Step 2 In the node properties sheet, click the Alarm tab > Alarm Behavior subtab. In the card properties sheet, click the Alarm Behavior tab.
Step 3 Click Update to view the latest Alarm Profile list.
Step 4 To apply an alarm profile at the card view, click the appropriate row in the Alarm Profile column for the port desired. To apply an alarm at the node view, click the appropriate row in the Profile column for the card desired.
Step 5 Choose the appropriate alarm profile from the drop-down list.
Step 6 Repeat these steps for each port that is being assigned a profile.
Step 7 Click Apply.
Note In the Alarm Behavior tab > Alarm Profile drop-down list, you can select any profile and click Force to all ports to apply this profile to all ports. But when the profile is saved, the Alarm Profile value returns to Inherited. This is expected behavior, because the drop-down list does not represent a value on the NE. After setting the alarm profile value on the ports, the Alarm Profile field returns to the first selectable value in the drop-down list.
9.4.5 Managing Alarm Profiles—CTC-Based NEs
Use the Alarm Profiles Management wizard to download an alarm profile from either an NE or from a local file.
Note This feature is available for CTC-based NEs with software release 3.4 or later.
Step 1 In the Domain Explorer window, choose Configuration > CTC-Based SONET NEs or CTC-Based SDH NEs > Alarm Profiles Management. The Alarm Profiles Management dialog box opens. The following table provides descriptions.
Step 2 Select an available profile from the NE by clicking the From NE radio button or from a file by clicking the From File radio button.
Step 3 Complete one of the following options:
•If you clicked the From NE radio button, select the NE and alarm profile from the lists.
•If you clicked the From File radio button, specify the file from a local drive by clicking the Local radio button, or from a server by clicking the Server radio button.
Step 4 Click Next.
Step 5 Edit the alarm severity for each alarm condition by clicking a row in the Alarm Severity column.
Step 6 Click Next.
Step 7 Enter the profile name and save the new alarm profile to the NE by clicking the Save to NE(s) radio button, or to a file by clicking the Save to File radio button.
Step 8 Complete one of the following options:
•If you clicked Save to NE(s), select the NE(s) on which to apply the new alarm profile. Check the Apply to Selected NE(s) check box if you want to set the alarm profile as the current profile.
•If you clicked Save to File, you can either save the file locally by clicking the Local radio button and specifying the directory path, or save it to a server by clicking the Server radio button and specifying the server name.
Step 9 Click Finish.
CTM schedules a job for this action. The alarm profile is downloaded to each selected NE and set, if so selected. This is tracked as a separate task in the Job Monitor table.
Table 9-13 Field Descriptions for the Alarm Profiles Management Wizard
Field
|
Description
|
Select a Profile
|
From NE
|
Choose From NE if the alarm profile is on an NE. If you select From NE, the From File options are not accessible.
|
Select NE
|
Select the NE where the alarm profile exists.
|
Select Profile
|
Select an alarm profile from the list.
|
From File
|
Choose From File if the alarm profile file is on your PC or a server. If you select From File, the From NE options are not accessible.
|
Local
|
Choose Local if the alarm profile file is on your PC. Enter the path for the file, or click Browse to search for it.
|
Server
|
Choose Server if the alarm profile file is located on a server. Use the drop-down list to select a server.
|
Edit the Profile
|
Alarm Condition
|
List of alarm conditions for the selected alarm profile.
|
Alarm Severity
|
Click the field to select a new alarm severity for the alarm condition. Severities are:
•Not Reported (NR)—A raise or clear of the condition is not sent to clients, but is tracked on the NE. You can retrieve a complete list of all raised conditions, including Not Reported as well as Not Alarmed, Critical, Major, and Minor, by using the RTRV COND TL1 command, or its equivalent.
•Not Alarmed (NA)—A raise or clear of the condition is sent to clients as a nonalarmed TL1 message (REPT EVT). The message has no severity and no service affecting flag.
•Minor (MN)—The alarm is a minor alarm.
•Major (MJ)—The alarm is a major alarm.
•Critical (CR)—The alarm is a critical, traffic-affecting alarm.
Note For critical, major, and minor alarms, a raise or clear of the condition is sent to clients as an alarmed message (REPT ALM). This message includes a service affecting flag, which is On or Off. If a normally service affecting condition is raised in a nonservice affecting situation (for example, the nontraffic-bearing side of a protect pair), the condition is sent as minor even if the profile says major or critical.
•Unset—The value of this alarm corresponds to the value of the default alarm profile on the node. Unset is useful when transferring alarm profiles between different versions of software.
•Transparent—The alarm behaves according to its parent object:
–If you set an alarm as Transparent on a port, it uses the card profile.
–If you set an alarm as Transparent on a card, it uses the node profile.
–If you sent an alarm as Transparent on a node, it uses the default profile.
|
Save the Profile
|
|
Profile Name
|
Name of the selected alarm profile.
|
Save to NE(s)
|
To save the alarm profile to one or more NEs, choose Save to NE(s). If you choose Save to NE(s), the Save to File options are not accessible.
|
Available NE(s)
|
Select one or more NEs in the Available NE(s) list and click Add to move them to the Selected NE(s) list.
|
Selected NE(s)
|
Select one or more NEs in the Selected NE(s) list and click Remove to move them to the Available NE(s) list.
|
Apply to Selected NE(s)
|
When checked, applies the alarm profile as the node-level alarm profile.
|
Overwrite the Profile
|
If a profile of the same name already exists, check this check box to overwrite the profile with the new profile.
|
Save to File
|
To save the defaults to a file on your PC or a server, choose Save to File. If you choose Save to File, the Save to NE(s) options are not accessible.
|
Local
|
Choose Local to save the file on your PC. Enter the path for the file, or click Browse to search for it.
|
Server
|
Choose Server to save the file on a server. Enter a server path in the field.
|
9.4.6 Using the Recovery Properties Pane
You can use the Recovery Properties pane in the Control Panel window to mark a service as critical for process monitoring purposes. If a critical process stops running or fails to poll monitoring services for a long time, the server shuts down and the client generates an alarm.
You can also use the Recovery Properties pane to list the servers that clients will log into if access to the primary server is disrupted.
Step 1 In the Domain Explorer window, choose Administration > Control Panel.
Step 2 Click Recovery Properties to open the Recovery Properties pane. The following table provides descriptions.
Step 3 Complete the following substeps to set a process as critical:
a. Click the Process Monitoring tab.
b. Check the Critical check box beside each service to indicate that the service is critical.
c. Click Save.
Step 4 Complete the following substeps to set up session recovery:
a. Click the Session Recovery tab.
b. In the Session Recovery area, fill in the Wait Time and No. of Retries fields.
c. In the Server List area, specify the servers that CTM clients will log into if access to the primary server is disrupted.
d. Click Save.
Table 9-14 Field Descriptions for the Recovery Properties Pane
Field
|
Description
|
Session Recovery Tab
|
Wait Time
|
If disruption to the primary CTM server occurs, the Wait Time field provides the number of seconds the client waits before attempting to log into the backup server.
|
No. of Retries
|
If the first backup CTM server is not available, this field provides the number of times the client should retry logging in before moving to the next server on the list.
|
Server List
|
Lists the CTM servers that CTM clients will log into if access to the primary CTM server is disrupted. The Add button adds a CTM server to the CTM server list. The Remove button removes a CTM server from the list.
If access to the primary CTM server is disrupted, the CTM client tries to log back into the primary server by default. If the CTM client cannot log back into the primary server, and if the server list contains server names, the client tries to log into each backup server in the order listed until the last server in the list is reached, or until the client successfully logs into a server.
|
Process Monitoring Tab
|
Service Name
|
Displays the process monitoring service name.
|
Critical
|
If checked, the selected service is designated as critical for process monitoring.
Note OS Agent, Oracle Service, and Service Manager are permanently critical. You cannot uncheck the Critical check boxes for these services.
|
9.4.7 Using the Statistics Panel—MGX Voice Gateway Cards
You can use the Statistics Panel in the Diagnostic Center to poll counters and plot the polling results.
9.4.7.1 Polling
This procedure covers the selection of counters to be polled and polling intervals.
Note After you start polling for an element, polling will continue as configured until you stop polling for that element. Polling of multiple counters can occur simultaneously. You may want to limit the level of simultaneous polling activity for performance reasons. You can verify that polling is in progress for at least one element by looking for the Polling icon at the bottom of the Diagnostic Center window.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the line or port from the Hierarchy pane to the right pane to display the diagnostics at the line or port level. See Figure 9-2.
Step 2 Click the Delta Mode radio button, Absolute Mode radio button, or From Start radio button. See the following table for a description of these radio buttons.
Step 3 In the Poll Interval drop-down list, choose the polling interval time.
Step 4 Check the Poll counter check box for each individual counter you want to poll.
Step 5 Click Start Polling to begin polling.
Polling begins. The Start Polling button changes to the Stop Polling button. The Polling icon appears at the bottom of the Diagnostic Center window.
Table 9-15 Diagnostic Center—Absolute Mode, From Start Mode, and Delta Mode Radio Buttons
Radio Button
|
Description
|
Absolute Mode
|
(Default) Provides the raw data shelved at the current counter state for the switch.
|
From Start Mode
|
Starts counters the moment polling starts, when the Start Poll button is clicked.
From Start mode restarts the timer and takes the current values of the counters as base values as soon as the Start Poll button is clicked. The from start report always contains the base values and the timer, which indicates the time lapsed since entering From Start mode.
|
Delta Mode
|
Takes the current values of the counters as base values and subtracts the polled values to identify the change that occurred in the values between polling periods.
|
9.4.7.2 Plotting
This procedure controls the selection of counters to be plotted.
Note When polling is stopped, the graph of polling activity remains available on the Graph tab. When polling is started again, a new graph is created.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the line or port from the Hierarchy pane to the right pane to display the diagnostics at the line or port level. See Figure 9-2.
Step 2 Click the Delta Mode radio button, Absolute Mode radio button, or From Start radio button. See Table 9-15 for a description of these radio buttons.
Step 3 From the Poll Interval drop-down list, choose the polling interval time.
Step 4 From the Statistics Panel, click the Table tab.
Step 5 Check the Poll counter check box for each individual counter you want to poll.
Step 6 Check the Plot counter check box for each individual counter you want to plot. Each counter to be plotted must have both the Poll and Plot check boxes checked.
Step 7 Click Start Polling to begin polling.
Polling begins. The Start Polling button changes to the Stop Polling button. The Polling icon appears at the bottom of the Diagnostic Center window.
Step 8 Click the Graph tab in the Statistics panel to view the polling graph.
9.4.8 Displaying the Status of All Nodes in the Network—MGX Voice Gateway Cards
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the network to the right pane to display the diagnostics at the network level. By default, the Diagnostics tab is selected in the right pane.
Step 2 The following fields at the network level are displayed:
•Node ID
•Node Name
•Node Sync Up Status
•Total Nodes Partially Synced
•Total Nodes Failed to Sync
9.4.9 Displaying the Status of a Node—MGX Voice Gateway Cards
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the node to the right pane to display the diagnostics at the node level. By default, the Diagnostics tab is selected in the right pane.
Step 2 Verify the following at the node level:
•All out-of-sync cards in the node
•SNMP and FTP communication with the node protocol status check (Manageability tab)
9.4.10 Displaying the Status of MGX Voice Gateway Cards
Depending on the type of card, you can view its status using one or all of the options listed in the following table.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the card from the Hierarchy pane to the right pane to display the diagnostics at the card level.
Step 2 Verify the status of the attributes at the card level.
Step 3 Click Refresh to update the Card Diagnostics window.
9.4.11 Displaying the Status for Lines or Ports—MGX Voice Gateway Cards
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the line or port from the Hierarchy pane to the right pane to display the diagnostics at the line or port level. See Figure 9-2.
Depending on the card type, you can also configure BERT. For more information, see Configuring a Bit Error Rate Test.
You can also see if a line is in loopback. See Displaying the Status for Lines in a Loopback—MGX Voice Gateway Cards.
Step 2 Click the Delta Mode radio button, Absolute Mode radio button, or From Start radio button. See Table 9-15 for a description of these radio buttons.
Step 3 From the Poll Interval drop-down list, choose the polling interval time.
Step 4 Verify the statistics name and statistics value for the real-time counters.
Step 5 Check the Poll counter check box for each individual counter you want to poll.
Step 6 Check the Plot counter check box for each individual counter you want to plot.
Step 7 Click Start Polling to begin polling.
Figure 9-2 Diagnostic Center—Line Status
9.4.12 Displaying the Status for Lines in a Loopback—MGX Voice Gateway Cards
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the card from the Hierarchy pane to the right pane to display the line in the loopback.
Step 2 Click the Lines In Loopback tab. See Figure 9-3.
Step 3 Verify the following parameters for the line:
•Line Number
•Loopback Type
Step 4 Click Refresh to update the settings for the Line Diagnostics window.
Figure 9-3 Diagnostic Center—Lines in Loopback
9.4.13 Displaying the Paths in Loopback for the VXSM-OC3 Card
Note This procedure applies to both SONET and SDH lines.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM-OC3 card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-4.
Step 2 Click the Paths In Loopback tab.
Step 3 Verify the following parameters for the path:
•Path ID
•Path Type
•Loopback Type
Step 4 Click Refresh to update the settings for the paths in loopback for the VXSM-OC3 card for both SONET and SDH lines.
Figure 9-4 Diagnostic Center—Paths in Loopback
9.4.14 Displaying the Status for a Bit Error Rate Test—VXSM and VISM-PR Cards
You can set up BERT options on the loopback connection and use the available test patterns displayed in the BERT Configuration window. See Configuring a Bit Error Rate Test.
Note This procedure applies to VXSM card for a T1/E1 line.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM-48T1/E1 card or VISM-PR card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-5.
Step 2 Click the Running BERTs tab.
Step 3 Verify the following parameters for the BERT test:
•Entity ID
•Entity Type
•Operation Status
•Bits Received
•Bits Received (Wrap Count)
•Bits Received In Error
•Bits Received In Error (Wrap Count)
Step 4 Click Refresh to update the settings for the lines in loopback for VXSM.
Figure 9-5 Diagnostic Center—Bit Error Rate Test
9.4.15 Monitoring and Diagnosing MGC Communication Protocols on VXSM Cards
These tasks are used to manage the status for the VXSM features:
•Monitoring and Diagnosing Media Gateway Links for the Voice Cards
•Monitoring and Diagnosing Reliable User Datagram Protocol for VXSM Cards
•Monitoring and Diagnosing Xternal Gateway Control Protocol for VXSM Cards
9.4.15.1 Monitoring and Diagnosing Media Gateway Links for the Voice Cards
Note•A session of the type of object must be available before it can be diagnosed.
•This procedure applies to voice cards with any interface for SONET, T1/E1, and T3 lines.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level.
Step 2 Click the MGC Communication tab and click the Media Gateway Links tab.
Step 3 The top view of the diagnostics panel displays a table of all available MGC links. Click the Refresh button to update the table contents. Verify the following MGC link parameters:
•Gateway Link ID
•Gateway IP Address
•Gateway Port
•Control Protocol
•Gateway Link Operational State
Step 4 To poll real-time counters for a particular MGC link, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
9.4.15.2 Monitoring and Diagnosing Reliable User Datagram Protocol for VXSM Cards
Note•A session of the type of object must be available before it can be diagnosed.
•This procedure applies to VXSM cards for both SONET and T1/E1 lines.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-6.
Step 2 Click the MGC Communication tab and click the RUDP Sessions tab.
Step 3 The top view of the diagnostics panel displays a table of all available RUDP sessions. Click the Refresh button to update the table contents. Verify the following RUDP session parameters:
•RUDP Session Number—Specifies the session group where the group belongs. One group has a maximum of four sessions. The range is from 1 to 64.
•State—Specifies the following RUDP session states:
–oos—Out of Service. When an RUDP session is created, the session state is oos.
–Is—In Service. The state changes to Is after the VXSM sends a start message to the Media Gateway Controller (MGC). The message is sent after a channel is created between the VXSM and the MGC.
–unknown
•Local IP—Specifies the IP address of the local VXSM.
•Remote IP—Specifies the IP address of the remote VXSM.
Step 4 To poll real-time counters for a particular RUDP session, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
Figure 9-6 Diagnostic Center—Reliable User Datagram Protocol for Voice Cards
9.4.15.3 Monitoring and Diagnosing Xternal Gateway Control Protocol for VXSM Cards
Note•A session of the type of object must be available before it can be diagnosed.
•This procedure applies to VXSM cards for both SONET and T1/E1 lines.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-6.
Step 2 Click the MGC Communication tab.
Step 3 Click the XGCP MGC tab.
Step 4 The top view of the diagnostics panel displays a table of all available XGCP parameters. Click the Refresh button to update the table contents. Verify the following XGCP peer group parameters:
•MGC Group Number—Specifies the MGC number. The range is from 1 to 8.
•Gateway IP Address—The Gateway IP Address Index parameter defines the media gateway address that relates to the PVC control type. The range is from 1 to 16.
•Protocol Index—Specifies the media gateway protocol number. The range is from 1 to 3.
•Gateway UDP Port—Specifies the gateway User Datagram Protocol (UDP) port.
Step 5 To poll real-time counters for a particular XGCP parameter, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
9.4.16 Monitoring and Diagnosing the Status for VISM-PR Features
The following task is used to manage the status for the VISM-PR features.
9.4.16.1 Monitoring and Diagnosing the Status of Simple Resource Coordination Protocol Peers
To display the status for the Simple Resource Coordination Protocol (SRCP) peer group for VISM-PR cards:
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VISM-PR card from the Hierarchy pane to the right pane to display the diagnostics at the card level.
Step 2 Click the VISM Features tab.
Step 3 Click the SRCP Peers tab.
Step 4 The top view of the diagnostics panel displays a table of all available SRCP peer groups. Click the Refresh button to update the table contents. Verify the following SRCP peer group parameters:
•SRCP Peer Number—Specifies the MGC number. The range is from 1 to 8.
•SRCP Peer Name—Specifies the name of the SRCP peer. The range is from 1 to 64 characters.
Step 5 To poll real-time counters for a particular SRCP peer group, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
9.4.17 Managing Diagnostics for DPNSS and DLC—VXSM Cards
The following tasks are used to manage the status for the DPNSS and DLC features.
9.4.17.1 Monitoring and Diagnosing Digital Private Network Signaling System Links
Note You must be using VXSM 2.5 or later to view the DPNSS/DASS tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-7.
Step 2 Click the DPNSS/DASS tab.
Step 3 Click the DPNSS Stats tab.
Step 4 The top view of the diagnostics panel displays a table of all available DPNSS peer groups. Click the Refresh button to update the table contents. Verify the following DPNSS peer group parameters:
•Bay—Specifies the bay number.
•Line—Specifies the channelized line number.
•D-Channel—Specifies the channelized line bit map.
•Signaling Type—Specifies the application type for the corresponding DPNSS D-channel.
•PBX Side—Specifies whether the DPNSS side is applicable.
•DPNSS Application Server—Specifies the DPNSS application server name.
Step 5 To poll real-time counters for a particular DPNSS peer group, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
9.4.17.2 Monitoring and Diagnosing Data Link Control Connections
Note You must be using VXSM 2.5 or later to view the DPNSS/DASS tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-7.
Step 2 Click the DPNSS/DASS tab.
Step 3 Click the DLC Stats tab.
Step 4 The top view of the diagnostics panel displays a table of all available DLC connections. Click the Refresh button to update the table contents. Verify the following DLC connection parameters:
•Bay—Specifies the bay number.
•Line—Specifies the channelized line number.
•D-Channel—Specifies the channelized line bit map.
•DLC Index—Specifies the DLC Channel Index value.
Step 5 To poll real-time counters for a particular DLC connection, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
Figure 9-7 Diagnostic Center—DPNSS and DLC
9.4.18 Managing Diagnostics for SS7 Signaling Components—VXSM Cards
These tasks are used to manage the status for the SS7 signaling components:
•Monitoring and Diagnosing GSP Links
•Monitoring and Diagnosing XUA Application Server Routes
•Monitoring and Diagnosing the XUA Application Server Process
•Monitoring and Diagnosing Signaling Connection Control Part Global Title Translation Mated Application Protocol
•Monitoring and Diagnosing Signaling Connection Control Part Global Title Translation Statistics
9.4.18.1 Monitoring and Diagnosing GSP Links
Note You must be using VXSM 2.5 or later to view the SS7 Signaling tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-8.
Step 2 Click the SS7 Signaling tab.
Step 3 Click the GSP Links tab.
Step 4 The top view of the diagnostics panel displays a table of all available GSP links. Click the Refresh button to update the table contents. Verify the following GSP link parameters:
•Link Set ID
•Link Signaling Code
•Link Interface
•Link Type
•Link State
Step 5 To poll real-time counters for a particular GSP link, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
9.4.18.2 Monitoring and Diagnosing XUA Application Server Routes
Note You must be using VXSM 2.5 or later to view the SS7 Signaling tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-8.
Step 2 Click the SS7 Signaling tab.
Step 3 Click the XUA AS Routes tab.
Step 4 The top view of the diagnostics panel displays a table of all available XUA AS routes. Click the Refresh button to update the table contents. Verify the following XUA AS route parameters:
•AS Route Number
•AS Route Name
•Destination Point Code
Step 5 To poll real-time counters for a particular route, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
9.4.18.3 Monitoring and Diagnosing the XUA Application Server Process
Note You must be using VXSM 2.5 or later to view the SS7 Signaling tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-8.
Step 2 Click the SS7 Signaling tab.
Step 3 Click the XUA ASP Stats tab.
Step 4 The top view of the diagnostics panel displays a table of all available XUA ASPs. Click the Refresh button to update the table contents. Verify the following XUA ASP parameters:
•ASP Number
•ASP Name
•Remote Port
Step 5 To poll real-time counters for a particular server process, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
9.4.18.4 Monitoring and Diagnosing Signaling Connection Control Part Global Title Translation Mated Application Protocol
Note You must be using VXSM 2.5 or later to view the SS7 Signaling tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-8.
Step 2 Click the SS7 Signaling tab.
Step 3 Click the SCCP GTT MAP tab.
Step 4 The top view of the diagnostics panel displays a table of all available SCCP global title translation (GTT) Mated Application Protocol (MAP) parameters. Click the Refresh button to update the table contents. Verify the following SCCP GTT MAP parameters:
•MAP Index
•Primary Point Code
•MAP Type
•MAP PC Status
Step 5 To poll real-time counters for a particular MAP, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
9.4.18.5 Monitoring and Diagnosing Signaling Connection Control Part Global Title Translation Statistics
Note You must be using VXSM 2.5 or later to view the SS7 Signaling tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-8.
Step 2 Click the SS7 Signaling tab.
Step 3 Click the SCCP GTT Stats tab.
Step 4 The top view of the diagnostics panel displays a table of all available SCCP GTT statistics. Click the Refresh button to update the table contents. Verify the following SCCP GTT statistics parameters:
•GSP Instance Number
•GSP Instance Name
•GSP Network Name
Step 5 To poll real-time counters for a particular statistic, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
Figure 9-8 Diagnostic Center—SS7 Signaling Components
9.4.19 Managing Diagnostics for VXSM Card Resource Overload and Utilization
The following tasks are used to manage the status for the resource overload and utilization features.
9.4.19.1 Monitoring and Diagnosing Resource Overload
Note You must be using VXSM 2.5 or later to view the Resource Control tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-9.
Step 2 Click the Resource Control tab.
Step 3 Click the Resource Overload tab.
Step 4 The top view of the diagnostics panel displays a table of all available resources. Click the Refresh button to update the table contents. Verify the following resource parameters:
•Resource Index
•Resource Type
•High Threshold
•Medium Threshold
•Low Threshold
Step 5 To poll real-time counters for a particular resource, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
9.4.19.2 Monitoring and Diagnosing Resource Utilization
Note You must be using VXSM 2.5 or later to view the Resource Control tab in this procedure.
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the VXSM card from the Hierarchy pane to the right pane to display the diagnostics at the card level. See Figure 9-9.
Step 2 Click the Resource Control tab.
Step 3 Click the Resource Overload tab.
Step 4 The top view of the diagnostics panel displays a table of all available resources. Click the Refresh button to update the table contents. Verify the following resource parameters:
•Resource Name
•Resource Scalar Units
•Initial Bucket
•Bucket Interval
Step 5 To poll real-time counters for a particular resource, select the corresponding row in the table. Follow the procedure in Using the Statistics Panel—MGX Voice Gateway Cards to perform statistics polling.
Appendix H, "Supported Real-Time Counters for MGX Voice Gateway Cards" gives details of the real-time counters displayed in the Diagnostic Center.
Figure 9-9 Diagnostic Center—VXSM Card Resource Overload and Utilization
9.4.20 Collecting Real-Time Performance and Health Statistics—PXM45 and PXM1E Cards
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the PXM45 or PXM1E card from the Hierarchy pane to the right pane to display the diagnostics at the card level.
Step 2 Select the Health Statistics tab to display and monitor a table of various node health indicators. Each row in the table displays the following indicator parameters:
•Description
•Value
•Sensor Type
•Sensor Scale
•Sensor State
Step 3 Click Refresh to display the latest health statistics of the card. You can monitor the following real-time statistics in the Table tab:
•CPU utilization
•Static memory utilization
•Dynamic memory utilization
•SNMP memory utilization
•Stats memory utilization (available for PXM1E cards only)
•IPC-360 buffer utilization
•IPC-1K buffer utilization
•IPC-4k buffer utilization
•IPC-8k buffer utilization
See Using the Statistics Panel—MGX Voice Gateway Cards for information on how to poll counters and plot polling results.
9.4.21 Monitoring SNMPv3 Real-Time Counters
Step 1 In the Diagnostic Center, click the Elements tab, and double-click or drag the MGX 8830 or MGX 8850 node from the Hierarchy pane to the right pane to display the diagnostics at the node level.
Step 2 Select the Manageability tab; then, select the Real Time Statistics subtab. You can monitor the following SNMPv3 real-time counters:
Note The Real Time Statistics tab and the SNMPv3 real-time counters are available in the Diagnostic Center only if the node is configured with SNMPv3.
•Packets dropped due to unsupported security level
•Packets dropped due to appearance outside of authoritative window
•Packets dropped due to unknown username
•Packets dropped due to unknown engine ID
•Packets dropped due to wrong digest value
•Packets dropped due to decryption error
See Using the Statistics Panel—MGX Voice Gateway Cards for information on how to poll counters and plot polling results.
9.4.22 Diagnosing Connections—MGX Voice Gateway Cards
The Diagnostic Center is used to diagnose connections. The key functions are the ability to test status, delay, and integrity of SPVC connections.
You can test the integrity of any existing connection that is nondisruptive to user traffic. The operation is similar to the tstcon command. For information about the tstcon command, see the Cisco WAN Switching Command Reference, Release 9.3.30.
9.4.22.1 Polling a Connection for Diagnostics
Step 1 In the Domain Explorer, select the MGX node and choose Fault > MGX Voice Gateway > Diagnostic Center.
Step 2 Click the Connections tab.
Step 3 Drag the node from the Hierarchy pane to the right pane to display the connection details.
Step 4 Click More Filters to display the Filter Settings dialog box.
Step 5 Choose the filter settings from the Filter Settings dialog box. For example, check both the Status check box and the OK check box if you want to retrieve only active connections.
Step 6 Click OK to apply the filter settings.
Step 7 Enter the number of connections that you want to retrieve in the Connection count to be retrieved field. To retrieve all the connections, enter an asterisk (*).
Step 8 Click Get to retrieve the connections. For example, you can retrieve both local and remote endpoints.
Step 9 The Connections table shows the number of connections matching the filter criteria and a list of connections that have matching local or remote endpoints.
Step 10 You now test the connection. Click the table row for the connection that you want to diagnose; then, click Diagnose. The Connection Diagnostic panel opens.
Step 11 In the Connection Diagnostic panel, click the Test Connection tab. The tab contains the following radio buttons:
•Test Conn—Initiates an end-to-end integrity test of the selected connection.
•Test Delay—Initiates an end-to-end measurement of round-trip delay.
Step 12 In the Connection Diagnostic Panel, click the Miscellaneous Operation tab. The tab contains the following radio buttons:
•Up Connection then Start Operation—Brings up the connection.
•Down Connection then Start Operation—Brings down the connection.
•Loopback—Initiates a connection endpoint loopback operation.
Step 13 If you clicked the Loopback radio button in the Miscellaneous Operation tab, the Loopback Operation panel opens with the following check boxes:
•Endpoint Type—Choose Local for the local endpoint or Remote for the remote endpoint.
•Loopback Type—Choose Destructive to set a connection loopback. Choose None to remove the connection loopback.
•Loopback Direction—Choose External or Internal.
Step 14 Click the Start Operation button to apply the loopback settings. You can monitor the results of the operation in the History pane.
9.4.23 Bit Error Rate Test—MGX Voice Gateway Cards
Bit error rate test is used to test the integrity of physical lines, ports, and logical paths. BERT is used on DS-3, DS-1, and DS-0/DS-0 bundle interfaces.
BERT generates a known data sequence into a transmission device, and examines the received sequence at the same device or a remote device for errors. Tests are run on a full T1/E1 line or a fractional T1/E1; for example, a single DS-0 or a group of DS-0s. BERT tests the quality of links by directly comparing a pseudo-random or repetitive test pattern with an identical, local-generated test pattern.
Note SRME, SRME/B, VXSM, VISM-PR, and AXSME card types support BERT.
BERT includes the following functions:
•Configures a local loopback on a line or port, or specifies that a remote loopback be used instead.
•Defines the bit pattern.
•Specifies a duration for the BERT session after the session terminates automatically.
•Specifies BERT can start on a designated line or port.
•Lists BERT tests that are initiated from CTM.
•Refreshes the display to update the output for the current BERT.
•Stops a BERT started from CTM.
9.4.23.1 Configuring a Bit Error Rate Test
You can set up BERT options on the loopback connection and use the available test patterns displayed in the BERT Configuration window. Before you configure the BERT options in the BERT Configuration window, make sure a line or port is selected from a BERT-supported service module.
In CTM, BERTs are applicable only on:
•AXSME 32 T1/E1 (on the DS-1 line)
•VXSM 48 T1/E1 (on the DS-1 line)
•VXSM OC3 (on DS-1 paths)
•VISM PR 8 T1/E1 (accompanied by an SRM card on the DS-1 line)
Note Selectable BERT parameters vary depending on the specific card, line, port, and path configuration.
Step 1 In the Diagnostic Center, double-click or drag the line or port that is supported by BERT from the Hierarchy pane to the right pane to display the Line and Port Configuration window.
Step 2 Click the BERT tab to display the BERT Configuration window. See Figure 9-10.
Note Not all the BERT configuration parameters are applicable for all card types.
Step 3 From the Test Pattern drop-down list, choose the test pattern options. The default option is allZeros.
Step 4 From the Invert Rx Test Pattern drop-down list, choose one of the following invert receive test options:
•Not Inverted
•Inverted
Step 5 From the Invert Tx Test Pattern drop-down list, choose one of the following invert transmit test options:
•Not Inverted
•Inverted
Step 6 From the Device to Loop drop-down list, ensure that the type of end device and loopback device option is correct.
Step 7 From the Error Insertion drop-down list, choose the error insertion option.
The insert constant-rate error option verifies that the transmitted pattern is being properly received and errors are properly detected. Once set to send continuous errors, errors are inserted at the configured rate until set to the no Error option.
If you set the value to insert errors while the test is not running, the test is not affected. However, when the test is started again, errors are inserted at the configured rate.
Step 8 Click Start BERT to initiate the BERT session. Note that once BERT is started, you can restart it with the modified options.
Figure 9-10 Diagnostic Center—BERT Options
9.4.23.2 Stopping a Bit Error Rate Test
While the BERT session is running, you can stop the session from the BERT Configuration windowthat is applicable to your selected line or port. From the BERT Configuration window, click Stop Bert.
9.4.23.3 Modifying a Bit Error Rate Test
After you stop a BERT session, you can modify the session.
Step 1 From the Insert Single Bit Error drop-down list, choose one of the following single-bit error options:
•No Error
•Insert Error
Step 2 Click Modify Bert to modify the BERT line by inserting a single-bit error in the transmitted BERT pattern.
9.4.23.4 Displaying a Bit Error Rate Test
After initiating a BERT session, you can view the status of all initiated BERT sessions at once. To display BERT status:
Step 1 In the Diagnostic Center, double-click or drag the line or port that is supported by BERT from the Hierarchy pane to the right pane to display the Line and Port Configuration window.
Step 2 Click the Monitored BERTs tab to display the Monitored BERTs window. See Figure 9-10.
The following table describes the parameters for the BERT statistics. Select the BERT entry to update the statistics, and click Refresh.
Table 9-17 Bit Error Rate Test Statistics Parameters
Name
|
Definition
|
Start Date and Time
|
Specifies the start date and time.
|
Operation Status
|
Specifies the current operation status of BERT.
|
Failed Reason
|
Defines the reason for the BERT failure. If the Operation Status parameter is set to BERT Failed, more information is available.
|
Bits Transmitted
|
Specifies the total number of bits.
|
Bits Received In Error
|
Specifies the total number of bit errors.
|
Error Inject Count
|
Specifies the total number of injected bit errors.
|
9.5 Where Is the Fault?
You need to be able to quickly troubleshoot problems in the network, identify when network capacity is being reached, and provide information to management on the number and types of devices in use. If the network goes down, one of the first things you will need to know is what devices are running on the network. You will want to know the names, addresses, and interfaces associated with each device in order to begin troubleshooting the problem. The more information you have in one central place about all of the devices, the easier it is to locate the necessary information, resolve problems quickly, and provide detailed information to interested parties.
9.5.1 Sources of Information
Fault management receives and processes information from the following sources:
•Autonomous reports of failures from NEs
•Trouble reports from customers and peer systems
•Results of diagnostics, exercises, and audits from NEs
•Impairment indications from performance management
•Network configuration data from configuration management
CTM constantly updates the alarm status of the network based on the alarm and event notifications sent by the monitored NEs and generated by the EMS itself. It performs alarm synchronization with the NE each time the connection to the NE is established or re-established and the NE is in service.
9.5.2 Identifying and Monitoring Alarms
To identify and monitor alarms on groups of NEs:
Step 1 In the Domain Explorer tree, select the management domain node or a group node. If it shows a critical, major, minor, or warning alarm icon, it means that one or more NEs within the management domain or group are experiencing an alarm.
Step 2 Select the management domain node or group node and choose Fault > Alarm Browser (or right-click the node and choose Alarm Browser in the popup menu). This opens the Alarm Browser window, which shows all the NEs in the management domain or group that are experiencing an alarm.
To identify and monitor alarms on a specific NE:
Step 1 Select an in-service NE in the Domain Explorer tree that shows an alarm icon.
Step 2 Choose Fault > Alarm Browser (or right-click the NE and choose Alarm Browser in the popup menu).
Step 3 For the CRS-1, XR 12000, ONS 15216, ONS 1580x, and CTC-based NEs, you can also choose Configuration > NE Explorer to view alarms on the NE or on specific modules. The Module View tab displays a graphic of the module that is installed in the slot. The number of critical, major, minor, and warning alarms for the module is displayed under Alarm Status. (Alarms also display when you move the mouse pointer over the graphic.)
9.5.3 Using Visual and Audible Alarm Notifications
Step 1 In the Domain Explorer window, choose Edit > User Preferences. The User Preferences dialog box opens.
Step 2 In the Event Notification tab, in the Show Notification Dialog For area, select whether or not an alert popup opens when a specific alarm or informational event occurs on NEs in the management domain or in the application.
The Event Notification dialog box opens whenever a new alarm or event occurs. According to your User Preferences selection, you will receive popup notification about alarms by severity and information on events from the NE or from CTM. The Event Notification popup remains open until one of the following occurs:
•You click OK to close the dialog box.
•It is replaced by an Event Notification dialog box with a higher severity.
•You click Disable on the popup window itself to disable additional popups.
The following table describes the fields in the Event Notification dialog box.
Step 3 In the Play Audible Notification For area, select whether or not an audible alert is sounded when a specific alarm or informational event occurs on the NE or in the application. You can also select whether or not a continuous audible alert is sounded when there is an update in the Dashboard. Check the Continuous Alarm for Dashboard Notifications check box.
Note To stop the continuous audible alert, choose Fault > Stop Continuous Beep in the Domain Explorer.
Step 4 Check the Save current settings check box and click OK.
Table 9-18 Field Descriptions for the Event Notification Dialog Box
Field
|
Description
|
Source
|
Name of the source where the alarm or event originated.
|
Time
|
Date and time that you received the Event Notification popup.
|
Category
|
Category of alarm or event. Alarm categories include Critical, Major, Minor, or Cleared. Event categories include NE event (if the event occurred on an NE) or EMS event (if the event occurred on CTM).
|
Probable Cause
|
Probable cause of the alarm or event.
|
Affected Object
|
Object that is affected by the alarm or event.
|
Description
|
Description of the alarm or event.
|
Service Affecting
|
Whether the alarm or event affects service.
|
9.5.4 Sources of Events
There are two sources of events: EMS-generated alarms and OSS (SNMP, CORBA GateWay, TL1). See Chapter 12, "Managing Southbound and Northbound Interfaces" for information about OSS events.
9.5.4.1 EMS-Generated Alarms
In addition to reporting NE-generated alarms, the EMS monitors and reports alarms and events on the EMS itself; for example, loss of connectivity to NE, and so on.
The EMS monitors and reports the NE-specific alarms and events (see Table 9-19) and non-NE-specific alarms and events (see Table 9-20).
Note NE-specific alarms and events can be viewed and accessed by users who are assigned to the particular NE.
Table 9-19 NE-Specific Alarms and Events
NE-Specific Alarms and Events
|
Description
|
Loss of communication to an NE
|
When the system detects loss of connectivity to an NE, an EMS alarm is generated in the Alarm Browser. This EMS alarm is cleared when the system re-establishes connectivity to the NE or when the NE is marked as Out of Service.
|
Automatic or manual memory backup failure
|
If an automatic or manual memory backup job fails, an EMS alarm is generated in the Alarm Browser. An individual EMS alarm is generated for each memory backup failure that occurs. All instances of the backup-related EMS alarms are cleared (for that particular NE) when the memory backup succeeds or when the NE is marked as Out of Service.
|
CTM-to-NE authentication failure
|
If the system attempts to log into an NE and fails, an alarm is generated. This alarm indicates that the username and password are no longer valid.
|
Switching IP addresses
|
For the ONS 1580x NEs, when an active IP address switches from the primary to the secondary IP address, an alarm is generated. It is cleared when the address is switched back to the primary IP address or when communication to the ONS 1580x NE is lost.
|
Failed PM data retrieval
|
For CTC-based NEs, an alarm is generated for every PM data retrieval failure. PM 15-minute retrieval fail alarms are generated if the system has not retrieved 15-minute PM data after the number of times to retrieve PM data has been reached. These alarms can be cleared manually or cleared automatically if a PM lost alarm is generated or if PM data is retrieved (PM collection should be set to 15 Min Robust).
|
Lost PM data
|
For CTC-based NEs, an alarm is generated for all PM lost data. A lost PM alarm is generated when:
•The EMS cannot collect PM data for 15 minutes or 1 day and the NE's PM collection is set to either 15 Min or 1 Day.
•The EMS cannot collect 15-minute PM data after 8 hours or 1-day PM data after 2 days and the NE's PM collection is set to 15 Min Robust or 1 Day Robust. If there are outstanding PM retrieval fail alarms, these alarms are cleared and the PM lost alarm is generated.
These alarms can be cleared manually.
|
Table 9-20 Non-NE-Specific Alarms and Events
Non-NE-Specific Alarms and Events
|
Description
|
Maximum number of login attempts exceeded
|
By default, users have a maximum of five login attempts. The user account is locked after the fifth unsuccessful login attempt and an EMS alarm is generated in the Alarm Browser. The alarm is cleared once the user account is unlocked or the account is deleted.
|
System self-monitor alarm
|
Threshold parameters such as CPU usage, memory usage, disk usage, circuit creation time, PM collection time, and resynchronization time are collected and evaluated to monitor the server performance. An alarm is generated if any of these parameters cross their threshold values with their corresponding severity level. The alarms are cleared only after the corresponding parameter value falls below the minor threshold. Subsequent threshold crossings for the same parameter do not generate additional alarms. Only the severity level is changed to indicate the current severity level for the specific parameter.
Note Alarms associated with circuit creation, configuration resynchronization, NE synchronization, and PM data collection indicate that the load on the system is high. Reduce the load on the system before proceeding. Alarms associated with pruning times also indicate that the load on the system is high. Reschedule pruning at a time when the system has less activity.
|
9.5.5 Enabling Fault Synchronization—CTC-Based NEs
Fault synchronization allows the system to collect the alarm history of an NE that has been marked as Out of Service for some time and is marked as In Service again. Alarm history is displayed in the Alarm Browser.
Step 1 In the Domain Explorer window, choose Administration > Control Panel and expand NE Service.
Step 2 Choose CTC-Based SONET NEs or CTC-Based SDH NEs.
Step 3 In the Robust Fault Synchronization area, check the appropriate check box:
•Enable ONS 15310 CL
•Enable ONS 15310 MA
•Enable ONS 15327
•Enable ONS 15454
•Enable ONS 15600
•Enable ONS 15454 SDH
•Enable ONS 15600 SDH
Step 4 Click Save.
9.6 How Can I Find the Root Cause of the Fault?
Root cause analysis captures additional information about defects for the purpose of identifying preventive actions.
In some cases, the alarm report or set of alarm reports generated by a fault are sufficient to indicate the root cause. But often, the information in the alarm messages must be supplemented or confirmed by information from customer trouble reports, diagnostics and exercises of equipment, audits of software and databases, and testing of circuits. Tests of equipment are called diagnostics, which are designed to identify the root cause of a fault; exercises, which isolate a unit or subsystem and verify that it can perform its intended function; and audits, which verify the integrity of software.
The EMS should correlate events and determine the faults that exist in the network. To correlate events means to look for relationships between them.
9.6.1 Setting Up Error Logs
The Error Log tables display server error information that is useful for debugging CTM processes. In most cases, the Error Log is requested by service personnel for debugging a problem on the CTM server. The Error Log captures abnormal and significant events based on severity level. Critical, major, minor, and informational errors are logged to the database; trace and debug information is logged to a log file and does not appear in the Error Log.
Caution CTM performance will degrade significantly if the trace or debug option is left on. All operations will slow down and you might lose alarm and event notifications. Use trace or debug only when troubleshooting with a customer support engineer.
There are two types of error logs available in CTM:
•Error logs for the Diagnostic Center, Configuration Center, Statistics Report, and Chassis View. These MGX-specific logs are accessed directly from the log directory on the server at /opt/root/log.
•Error logs for the other applications in CTM. Choose Administration > Error Log to view these logs.
The Logging Properties pane allows you to control the volume of messages that is created by the server. To reduce the amount of information logged to the database, turn off entire components (CTM, CTM GateWay/SNMP, and CTM GateWay/TL1).
Step 1 In the Domain Explorer window, choose Administration > Control Panel.
Step 2 Click Logging Properties to open the Logging Properties pane. The Logging Properties pane contains the following tabs: General, SNMP Trap Service Debug, SM Service Debug, and SysLog Service Debug. The following table provides descriptions.
Step 3 In the Error Log Properties area of the General tab, choose the error level that will be included in the Error Log for each component.
Step 4 Click Save. All changes take effect immediately and do not require restart of the server.
Table 9-21 Field Descriptions for the Logging Properties Pane
Field
|
Description
|
General Tab
|
Error Log Properties
|
CTM Server
|
Choose the error level to include in the Error Log for all services (such as NE services, PM services, and so on).
|
Preferences
|
Log File Directory
|
Choose the directory where the log files are saved. The default directory is /opt/CiscoTransportManagerServer/log.
|
Max Debug Log File Size
|
Select the maximum size of the Debug Log file, in megabytes (MB). You can select a default log size for each log file.
|
Upload Log Files button
|
Opens the Upload Log Files dialog box, which allows you to view a list of Archive Log files stored on the server and upload them to a specified directory on the client. See Uploading Log Files.
|
Compress File
|
Compresses the log file, optimizing space.
|
Enable Initialization Log
|
Enables the log file that pertains to initialization of a service. With initialization logging, debugging begins well before any service starts.
|
Archiving
|
Enable Log File Archiving
|
If checked, allows you to Archive Log files to facilitate longer term collection of logging information. When a log file is about to wrap, you can save the regular log file to a separate archive file.
|
Archive Directory
|
Specify the directory where Archive Log files are saved. The name of the archived file incorporates a time stamp (when the file is created) to ensure a unique filename.
|
Max No. Archived Log Files
|
Specify the maximum number of log files to archive on the server. The range is from 1 to 25 files.
|
Upload Archive button
|
Opens the Upload Archive dialog box, which allows you to view a list of Archive Log files stored on the server and upload them to a specified directory on the client. See Uploading Archive Log Files.
|
Levels of Errors Logged to the Database
|
Fatal
|
This database log level corresponds to the CTMDebug.FATAL level in the Debug Log. The Debug Log is used for service-side debugging.
|
Error
|
This database log level corresponds to the CTMDebug.ERROR level in the Debug Log.
|
Warning
|
This database log level corresponds to the CTMDebug.WARN level in the Debug Log.
|
Debug
|
This database log level corresponds to the CTMDebug.DEBUG level in the Debug Log.
Caution CTM performance will degrade significantly if the Debug option is left on. All operations will slow down and you might lose alarm and event notifications. Use Debug only when troubleshooting with a Cisco customer support engineer.
|
SNMP Trap Service Debug Tab
|
Overall Logging
|
Enable
|
Select this radio button to enable overall debugging and to select debug modules for the SNMP trap service.
|
Disable
|
Select this radio button to disable overall debugging.
|
Debug Modules
|
Available
|
Lists the modules that can be used for debugging. Select a module from the list; then, click the Add button to add the module to the Selected list.
|
Selected
|
Lists the modules that will be used for debugging. Select a module from the list; then, click the Remove button to remove the module from the Selected list.
|
SM Service Debug Tab
|
Overall Logging
|
Enable
|
Select this radio button to enable overall debugging and to select debug modules for the SM service.
|
Disable
|
Select this radio button to disable overall debugging.
|
Debug Modules
|
Available
|
Lists the modules that can be used for debugging. Select a module from the list; then, click the Add button to add the module to the Selected list.
|
Selected
|
Lists the modules that will be used for debugging. Select a module from the list; then, click the Remove button to remove the module from the Selected list.
|
SysLog Service Debug Tab
|
Overall Logging
|
Enable
|
Select this radio button to enable overall debugging and to select debug modules for the syslog service.
|
Disable
|
Select this radio button to disable overall debugging.
|
Debug Modules
|
Available
|
Lists the modules that can be used for debugging. Select a module from the list; then, click the Add button to add the module to the Selected list.
|
Selected
|
Lists the modules that will be used for debugging. Select a module from the list; then, click the Remove button to remove the module from the Selected list.
|
9.6.1.1 Uploading Archive Log Files
Use the Upload Archive dialog box to view a list of archive log files stored on the server and upload them to a specified directory on the client.
Step 1 In the Domain Explorer window, choose Administration > Control Panel.
Step 2 Click Logging Properties to open the Logging Properties pane.
Step 3 In the General tab > Archiving area, click the Upload Archive button. The Upload Archive dialog box opens. The following table provides descriptions.
Step 4 In the Files area, select log files from the list. To select multiple files, hold down the Ctrl key on your keyboard while using your mouse to click files. Click Select All to select all files in the list.
Step 5 In the Upload Location text field, specify where you want to save the log files. Click Browse to choose a client location different from the default.
Step 6 Click Upload to upload the selected log files to the specified directory.
Table 9-22 Field Descriptions for the Upload Archive Dialog Box
Field
|
Description
|
Files
|
Lists the archive files available on the server location.
|
Upload Location
|
Allows you to specify where you want to save the archive files. The default location is <CTM_client_installation_directory>\archive\ or <CTM_client_installation_directory>/archive/. Click Browse to choose a different location.
|
Upload
|
Uploads the selected log files to the specified location on the client.
|
Select All
|
Selects all of the log files in the list.
|
Cancel
|
Replaces any changes to user-defined fields with the previous values and closes the dialog box.
|
Help
|
Launches the online help for the Upload Archive dialog box.
|
9.6.1.2 Uploading Log Files
Use the Upload Log Files dialog box to view a list of log files stored on the server and upload them to a specified directory on the client.
Step 1 In the Domain Explorer window, choose Administration > Control Panel.
Step 2 Click Logging Properties to open the Logging Properties pane.
Step 3 In the General tab > Preferences area, click the Upload Log Files button. The Upload Log Files dialog box opens. The following table provides descriptions.
Step 4 In the Files area, select log files from the list. To select multiple files, hold down the Ctrl key on your keyboard while using your mouse to click files. Click Select All to select all files in the list.
Step 5 In the Upload Location text field, specify where you want to save the log files. Click Browse to choose a client location different from the default.
Step 6 Click Upload to upload the selected log files to the specified directory.
Table 9-23 Field Descriptions for the Upload Log Files Dialog Box
Field
|
Description
|
Files
|
Lists the log files available on the server location.
|
Upload Location
|
Allows you to specify where you want to save the log files. The default location is <CTM_client_installation_directory>\log\ or <CTM_client_installation_directory>/log/. Click Browse to choose a different location.
|
Upload
|
Uploads the selected log files to the specified location on the client.
|
Select All
|
Selects all of the log files in the list.
|
Cancel
|
Replaces any changes to user-defined fields with the previous values and closes the dialog box.
|
Help
|
Launches the online help for the Upload Log Files dialog box.
|
9.6.2 Viewing the Error Log
The Error Log shows CTM server error information that is useful for debugging purposes. In most cases, the Error Log is requested by service personnel for debugging a problem on the CTM server. The Error Log captures abnormal and significant events based on severity level.
As the default, the Error Log displays information about significant events that occurred during the last four hours. You can change the default time period in the User Preferences dialog box.
To open the Error Log, choose Administration > Error Log in the Domain Explorer. The following table describes the fields in the Error Log.
Table 9-24 Field Descriptions for the Error Log
Column Name
|
Description
|
CTM Time Stamp
|
Date and time when the error occurred on the CTM server.
|
Module
|
Name of the module where the error occurred.
|
Severity
|
Severity level of the error:
•Critical, Major, Minor, or Informational—When set to any of these severity levels, all messages corresponding to critical, major, and minor severity levels are logged to the database and all informational messages are stored in the log file.
•Debug or Trace—When set to debug or trace, all informational and higher messages are logged to the database. All debug and trace messages are logged to the log files.
|
Submodule
|
Name of the submodule where the error occurred.
|
Filename
|
Name of the file where the error occurred. Cisco technical support engineers use this information for troubleshooting.
|
Line
|
Exact line where the error occurred. Cisco technical support engineers use this information for troubleshooting.
|
Message
|
Text of the error message.
|
By default, all messages are logged to the following files in the /opt/CiscoTransportManagerServer/log directory:
•CTMTL1FWDerror.log
•CTMServerError.log
•CTMerror.log
•HFRNEService-<number>-<time_stamp>.log
•HFRPMService-<number>-<time_stamp>.log
•CAT65xxNEService-<number>-<time_stamp>.log
•CAT65xxNEServiceError-<number>-<time_stamp>.log
•MGX88xxNEService-<number>-<time_stamp>.log
•MGX88xxPMService-<number>-<time_stamp>.log
•ONS15216NEService-<number>-<time_stamp>.log
•ONS15302NEService-<number>-<time_stamp>.log
•ONS15305NEService-<number>-<time_stamp>.log
•ONS1530xPMService-<number>-<time_stamp>.log
•ONS15454NEService-<number>-<time_stamp>.log
•ONS15454SDHNEService-<number>-<time_stamp>.log
•ONS15454SDHPMService-<number>-<time_stamp>.log
•ONS15454PMService-<number>-<time_stamp>.log
•ONS155xxNEService-<number>-<time_stamp>.log
•ONS155xxPMService-<number>-<time_stamp>.log
•ONS15600SDHPMService-<number>-<time_stamp>.log
•ONS15600PMService-<number>-<time_stamp>.log
•ONS15800NEService-<number>-<time_stamp>.log
•ONS15801NEService-<number>-<time_stamp>.log
•ONS15808NEService-<number>-<time_stamp>.log
•ONS158xxPMService-<number>-<time_stamp>.log
•UnmanagedNEService-<number>-<time_stamp>.log
•SMService-0-<time_stamp>.log
•SnmpTrapService-2-<time_stamp>.log
•CORBAGWService-1-<time_stamp>.log
By default, all Syslog Service messages are logged to the SyslogService.log file in the /opt/CiscoTransportManagerServer/log directory.
Note The default directory /opt/CiscoTransportManagerServer might have been changed during installation of the CTM server.
After resetting the Error Log level to Critical, Major, Minor, or Informational, remove the log files to free disk space. Each time a new log file is started, a backup of the previous file is kept in the log-file.bak file. Remove the backup file at any time.
9.6.3 Filtering Data in the Error Log
Step 1 In the Error Log, choose File > Filter (or click the Filter Data tool). The Filter dialog box opens.
Step 2 Specify the filter parameters described in the following table.
Step 3 After making your selections, click OK to run the filter.
Table 9-25 Field Descriptions for the Error Log Filter Dialog Box
Tab
|
Description
|
CTM Time Stamp (time zone)
|
Allows you to filter Error Log data for a specified time period, ranging from the past hour to the past 6 months. Click the User Specified radio button to specify an exact filter starting and ending time by month, day, year, and hour. The time zone can be GMT, a user-defined offset from GMT, or local time, depending on what is specified in the User Preferences dialog box. If you want to filter Error Log data and the time period is not important, click No Time Specified.
|
Modules
|
Allows you to filter Error Log data by CTM module.
|
Submodules
|
Allows you to select CTM server submodules to filter Error Log data.
|
Severity
|
Allows you to filter Error Log data based on severity level: Critical, major, minor, and informational.
|
9.6.4 Managing the NE Audit Trail—CTC-Based NEs
The Audit Trail table is a security tool used to investigate unauthorized activities after they occur so that proper remedial action can be taken. It displays audit trail information for CTC-based NEs.
Audit trail entries might be missing from the Audit Trail table because of the following reasons:
•There is heavy provisioning activity on the NE that the local audit trail log on the NE might wrap between polling intervals, and records will be missed for collection.
•The server is down or communications to the NE are unavailable for an extended period. There is a greater likelihood that the log on the NE will wrap before the system can collect the records.
Note•The audit trail collection interval can be set in the applicable NE Service pane in the Control Panel. See Changing the Audit Trail Collection Interval.
•To collect the latest and most reliable audit trail information for a specific NE, the time must be synchronized with the SNTP server. Do not manually set the clock on CTC-based NEs.
9.6.4.1 Viewing the Audit Trail Table
To view the Audit Trail table, choose Administration > CTC-Based NEs > Audit Trail Table. The following table provides descriptions.
Table 9-26 Field Descriptions for the Audit Trail Table
Field
|
Description
|
Alias ID
|
Alias name of the NE.
|
Sequence Number
|
NE-generated record ID.
|
NE Username
|
NE user ID.
|
Time Stamp
|
Date and time.
|
Description of Operation
|
Description of the audit trail operation.
|
Status of Operation on NE
|
Status of the audit trail operation. Statuses are Passed, Failed, and Aborted.
|
NE ID
|
ID of the selected NE.
|
9.6.4.2 Filtering Data in the Audit Trail Table
Step 1 In the Audit Trail table, choose File > Filter (or click the Filter Data tool). The Filter dialog box opens.
Step 2 Specify the filter parameters described in the following table.
Step 3 After making your selections, click OK to run the filter.
Table 9-27 Field Descriptions for the Audit Trail Table Filter Dialog Box
Tab
|
Description
|
Time Stamp (time zone)
|
Allows you to filter audit trail data for a specified time period, ranging from the past hour to the past 6 months. Additionally, you can click the User Specified radio button to specify an exact filter starting and ending time by month, day, year, and hour. The time zone can be GMT, a user-defined offset from GMT, or local time, depending on what is specified in the User Preferences dialog box. Click No Time Specified if you want to filter audit trail data and the time period is not important.
|
NE ID
|
Allows you to move NEs back and forth between the list of available NEs and selected NEs. The filter runs on the NEs in the Selected NE ID list.
|
Username
|
Allows you to move users back and forth between the list of available users and selected users. The filter runs on the users in the Selected Users list.
|
Sequence Number
|
Allows you to enter a starting and ending sequence number for filtering. Check the Disregard All Other Filter Criteria check box to base the filter on only the starting and ending sequence number.
Note If the sequence number reaches 59999, CTM collects audit records starting with 1 in the next collection interval.
|
Operation Status
|
Select an operation status for filtering. Operation statuses are Passed, Failed, and Aborted.
|
9.6.4.3 Enabling or Disabling Audit Trail Collection
Step 1 In the Domain Explorer tree, select a CTC-Based NE.
Step 2 In the Network Element Properties pane > Status tab > Audit Trail State field, choose Enabled or Disabled from the drop-down list.
Note By default, the Audit Trail State field is set to Disabled.
Step 3 Click Save.
9.6.4.4 Changing the Audit Trail Collection Interval
Step 1 In the Domain Explorer, choose Administration > Control Panel.
Step 2 In the Control Panel, expand NE Service and choose CTC-Based SONET NEs or CTC-Based SDH NEs.
Step 3 In the Status tab > Audit Trail Collection Interval field, enter the collection interval time. The Cisco default is 30 minutes.
Step 4 Click Save.
9.6.5 Setting Debug Options
In CTM, the debug option gives you information to investigate, diagnose, and fix a problem.
Note If required, set the debug levels for specific MGX backend processes by updating the debug parameter in the configuration file for that process in the /opt/svplus/config directory. See the following table. After the changes are made, stop and restart the CTM server.
Table 9-28 CTM Processes and Configuration Files
Process
|
Config File
|
Parameter
|
Range
|
Default
|
AuditLogger
|
AuditLogger.conf
|
DB_LEVEL
|
[1..5]
|
5
|
cmgrd
|
-none-
|
-none-
|
—
|
—
|
cmsvr
|
cmsvr.conf
|
LOG_LEVEL
|
[1..7]
|
7
|
configserver
|
configserver.conf
|
LOG_LEVEL
|
[1..7]
|
7
|
cwmftpd
|
cwmftpd.conf
|
LOG_LEVEL
|
[1..7]
|
2
|
cwmsmap
|
-none-
|
-none-
|
—
|
—
|
DCServer
|
DCServer.conf
|
LOG_LEVEL
|
[1..7]
|
7
|
dmd
|
dmd.conf
|
LOG_LEVEL
|
[1..7]
|
2
|
—
|
—
|
SYNCUP_LOG_LEVEL
|
[1..7]
|
2
|
—
|
—
|
MSG_TRACING
|
enable/disable
|
disable
|
—
|
—
|
MAX_NUM_OF_OLD_LOGS
|
—
|
5
|
eventd
|
-none-
|
-none-
|
—
|
—
|
ILMITopoc
|
ILMITopoc.conf
|
Debug Level
|
[1..5]
|
2
|
NMServer
|
NMServer.conf
|
LOG_LEVEL
|
[1..7]
|
2
|
—
|
—
|
STARTUP_LOG_LEVEL
|
[1..7]
|
2
|
—
|
—
|
MAX_NUM_OF_OLD_LOGS
|
—
|
5
|
—
|
—
|
MSG_TRACING
|
enable/disable
|
disable
|
nts
|
nts.conf
|
DEBUG_LEVEL
|
[1..5]
|
2
|
—
|
—
|
LOG_KEEP
|
—
|
5
|
ooemc
|
emd.conf
|
OODebug Level
|
[1..7]
|
2
|
—
|
—
|
OOKeep
|
—
|
5
|
pmcollector
|
pmcollector.conf
|
LOG_LEVEL
|
[1..7]
|
7
|
RtmProxy
|
SNMPProxy.conf
|
-none-
|
—
|
—
|
sdbroker
|
sdbroker.conf
|
LOG_LEVEL
|
[1..7]
|
2
|
—
|
—
|
SYNCUP_LOG_LEVEL
|
[1..7]
|
2
|
—
|
—
|
MSG_TRACING
|
enable/disable
|
disable
|
—
|
—
|
MAX_NUM_OF_OLD_LOGS
|
—
|
5
|
snmpcomm
|
snmpcomm.conf
|
DEBUG_LEVEL
|
[1..5]
|
2
|
—
|
—
|
LOG_KEEP
|
—
|
5
|
srtserver
|
srtserver.conf
|
LOG_LEVEL
|
[1..7]
|
7
|
statsparser
|
statsparser.conf
|
LOG_LEVEL
|
[1..7]
|
7
|
topod
|
Topod.conf
|
Debug Level
|
[1..5]
|
2
|
Specifying debug options allows you to choose parameters to display in the Debug Log.
Step 1 In the Domain Explorer, choose File > Debug Options.
Step 2 Specify the debug options. The following table provides descriptions.
Step 3 After making your selections, click Apply.
Table 9-29 Field Descriptions for the Debug Options Dialog Box
Field
|
Description
|
Modules
|
Available, Selected
|
Select modules that will display debug messages. Use the Add and Remove buttons to move modules to the Selected list or remove modules from the list.
|
Debug Level
|
Fatal
|
Instructs the Debug Log to display messages with a severity level of at least Fatal.
|
Warning
|
Instructs the Debug Log to display messages with a severity level of at least Warning.
|
Info
|
Instructs the Debug Log to display messages with a severity level of at least Info.
|
Debug
|
Instructs the Debug Log to display messages with a severity level of at least Debug.
Caution CTM performance will degrade significantly if the Debug option is left on. All operations will slow down and you might lose alarm and event notifications. Use Debug only when troubleshooting with a Cisco customer support engineer.
|
Trace
|
Instructs the Debug Log to display messages with a severity level of Trace.
Caution CTM performance will degrade significantly if the Trace option is left on. All operations will slow down and you might lose alarm and event notifications. Use Trace only when troubleshooting with a Cisco customer support engineer.
|
Display Options
|
File
|
Check the File check box to write the Debug Log to a specific file. You can click Browse to browse for a local client directory for the Debug Log. After you specify the filename, the log is stored at <filename>0.log and <filename>1.log when <filename>0.log is filled to its maximum size.
By default, the Debug Log is saved at C:\Cisco\TransportManagerClient<version_number>\debug\CTMC-debug0.log or /opt/CiscoTransportManagerClient<version_number>/debug/CTMC-debug0.log. The dialog box shows the filename without the number 0 or 1, which is appended by default by the Java debugging APIs.
Note <version_number> is replaced by the version number of the installed CTM client.
|
Max File Size
|
Enter the maximum file size for the Debug Log, in bytes.
|
Telnet
|
Check the Telnet check box to write the Debug Log to a Telnet port.
Note Telnet to the IP address of the CTM client workstation, not the CTM server.
|
Console
|
Check the Console check box to write the Debug Log to the console.
|
9.7 Who Is Responsible for Managing the Fault?
To manage faults effectively, you must know who is taking responsibility for managing each case. CTM offers the following options:
•Acknowledging and Unacknowledging Alarms
•Configuring Alarm Acknowledgement and Alarm Notes
9.7.1 Acknowledging and Unacknowledging Alarms
The alarm acknowledgement feature acknowledges selected alarms or all alarms with a single click.
Step 1 In the Domain Explorer window, select an NE and choose Fault > Alarm Browser. This opens the Alarm Browser window for the selected NE.
The Alarm Browser window lists critical, major, minor, and warning alarms that have not been cleared or cleared alarms that have not been acknowledged.
Step 2 Select the alarms to be acknowledged and choose Fault > Acknowledge Alarms (or click the Acknowledge Selected Alarm(s) tool). Click Yes in the confirmation dialog box. Click the Refresh Data tool to see the changes. A check mark icon provides a visual indication of acknowledged alarms.
Step 3 Click the Acknowledge Selected Alarm(s) tool again to unacknowledge the selected alarms. Click Yes in the confirmation dialog box. Click Refresh Data to see the changes. The check mark is removed, indicating that the alarm has been unacknowledged.
Note Alarm unacknowledgement is disabled by default. Make sure to enable the alarm unacknowledgement feature in the Control Panel before unacknowledging an alarm. See Configuring Alarm Acknowledgement and Alarm Notes for more information.
Step 4 To acknowledge all alarms in the view, choose Fault > Acknowledge All Alarms (or click the Acknowledge All Alarms tool). Click Yes in the confirmation dialog box.
9.7.2 Configuring Alarm Acknowledgement and Alarm Notes
Use the UI Properties pane to configure alarm acknowledgment and enable or disable the alarm note feature.
Step 1 In the Domain Explorer window, choose Administration > Control Panel and click UI Properties.
Step 2 In the Fault Management area, select either Manual or Automatic for alarm acknowledgement.
•If you choose Manual, alarms must be acknowledged manually. Cleared alarms move from the Alarm Browser to the Alarm Log once they are acknowledged.
•If you choose Automatic, the server automatically acknowledges alarms when they are cleared and moves them from the Alarm Browser to the Alarm Log.
Note Active alarms are not automatically acknowledged.
If the alarms are initially set to Manual Alarm Acknowledgement, and then you switch to Automatic Alarm Acknowledgement, all the alarms in the Alarm Browser will be cleared and acknowledged automatically. This might take a while, depending on the number of alarms in the database that have not been acknowledged manually.
You can still acknowledge alarms manually even if Automatic Alarm Acknowledgment is set.
Step 3 Use the Overwrite Alarm Notes option to enable or disable the ability to overwrite alarm notes created by another user.
Step 4 Choose either Enable or Disable for alarm unacknowledgement.
•If you choose Enable, you can unacknowledge alarms in the Alarm Browser.
•If you choose Disable, alarms cannot be unacknowledged in the Alarm Browser.
Step 5 Click Save.
9.8 How Can the Fault Be Fixed?
After receiving an alarm, the EMS or the user must take some sort of action on each of the faults. This can include logging the fault, delivering it to an appropriate tracking application, alerting key personnel to a critical fault, or implementing a repair.
9.8.1 Clearing a Security Violation Alarm—ONS 15305 R3.0, ONS 15327, ONS 15454 SONET, ONS 15454 SDH
Step 1 In the Domain Explorer tree, select an ONS 15305 R3.0, ONS 15327, ONS 15454 SONET, or ONS 15454 SDH NE and choose Configuration > NE Explorer (or click the Open NE Explorer tool).
Step 2 In the NE Explorer window, choose Fault > Clear Security Violation Alarm. A message appears if there are no security violations or intrusion alarms on the NE or if the operation has been accepted.
Step 3 Click OK in the message box.
9.8.2 Performing a System Reset—ONS 15600 SONET and ONS 15600 SDH
Step 1 In the Domain Explorer tree, select the ONS 15600 SONET or ONS 15600 SDH NE for which to view the configuration.
Step 2 Choose Configuration > NE Explorer (or click the Open NE Explorer tool).
Step 3 In the NE Explorer, choose File > System Reset.
9.8.3 Restarting an ONS 15800, ONS 15801, or ONS 15808 Laser
The ONS 1580x NEs include an automatic laser shutdown feature where the laser is automatically shut down to a safe power level in the event of a fiber break. You can manually restart an ONS 15800, ONS 15801, or ONS 15808 laser.
Note This procedure applies only to the following amplifiers:
•Infrared-band Booster Amplifier (ONS 15800, ONS 15801)
•Infrared-band Booster Amplifier-10 Gbps (ONS 15800, ONS 15801)
•Transmit Power Amplifier-Infrared Band (ONS 15800, ONS 15801)
•Pre-Line Amplifier-Infrared Band (ONS 15800, ONS 15801)
•Optical Booster Amplifier (ONS 15808)
•Optical Extra Pump (ONS 15808)
•Optical External Pump (ONS 15808)
Step 1 Select an ONS 15800, ONS 15801, or ONS 15808 NE and choose Configuration > NE Explorer (or click the Open NE Explorer tool).
Step 2 In the tree view of the NE Explorer window, select the card.
Step 3 In the property sheet of the NE Explorer window, click the Action tab.
Step 4 In the Laser Off Control section, click Enable.
Step 5 Click Execute.
9.8.4 Exporting Alarms and Events to a Text File
Use the Event Export Manager to export alarms and events to a text file as they occur. In addition, you can use the Event Export Manager to set various export parameters to refine the export. See 1.5.8 Exporting Alarms and Events, page 1-41.
