Table Of Contents
NME-APA Troubleshooting Concepts
Access to Application Performance Assurance for Troubleshooting
Accessing the NME-APA Module for Troubleshooting
Accessing the NME-APA Console for Troubleshooting
Starting and Stopping APADC Services
APA Logging Systems
NME-APA Logging
Copying the User Log
Viewing the User Log
Clearing the User Log
Viewing the User Log Counters
Generating a File for Technical Support
APADC Logging
Operational Status
Signature, Protocol, and Service IDs
Debug Flags for Protocol Libraries
Debug Flags for SML Components
NME-APA Troubleshooting Concepts
This module includes background information that can help operators to troubleshoot issues when using the Cisco Network Enhanced Module for Application Performance Assurance (NME-APA) and the Cisco Application Performance Assurance Device Console (APADC).
•
Access to Application Performance Assurance for Troubleshooting
•Starting and Stopping APADC Services
•APA Logging Systems
•Operational Status
•Signature, Protocol, and Service IDs
•Debug Flags for Protocol Libraries
•Debug Flags for SML Components
Access to Application Performance Assurance for Troubleshooting
Accessing the NME-APA Module for Troubleshooting
The NME-APA CLI includes many debug commands specifically to assist in troubleshooting the APA.
To access the NME-APA module:
Step 1 Connect to the NME-APA module via Telnet, using the IP address configured from the host ISR router.
Step 2 Login at the CLI login prompt.
Step 3 Execute CLI commands based on the level (5, 10, or 15) being enabled.
Note CLI debug commands can only be executed from level 15 (ROOT level).
Accessing the NME-APA Console for Troubleshooting
Use the Engineering commands to troubleshoot from the NME-APA console.
To access the NME-APA console through the ISR router, use the following command from the host-router CLI:
service-module integrated-service-engine slot/0 session
To execute engineering commands:
•Access ShellCon by opening a Telnet session to port 2301.
All engineering commands are used for debugging purposes. They should be used with care so as not to interfere with live traffic.
Starting and Stopping APADC Services
APADC is composed of two services running on the Windows platform.
•Cisco APADC MySQL
•Cisco APADC Tomcat
Both of these services are installed as Windows services and can be managed using the "Services" control panel component.
To start, stop, or restart the APADC services:
Step 1 Access the Windows "Services" panel by doing either of the following:
•Go to Control Panel > Administrative Tools> Services
•At the Windows prompt, type "services.msc"
Step 2 Locate the APADC services in the "Services" panel.
The status of a service can be verified under the "status" column of the service row.
Step 3 Execute the appropriate command.
APA Logging Systems
The APA system generates log files containing records of system events, including startup, shutdown and errors. Each log file is an ASCII file that can be viewed in any editor.
There are two separate logging components that generate log files:
•The logging system of the NME-APA blade.
•The logging system of the APADC web application.
NME-APA Logging
The NME-APA log is accessed using the NME-APA blade CLI commands.
Basic operations include:
•Copying the User Log to an external source
•Viewing the User Log
•Clearing the User Log
•Viewing and clearing the User Log counters
Copying the User Log
You can view the log file by copying it to an external source or to disk. This command copies both log files to the local NME-APA platform disk or any external host running a FTP server.
To copy the user log to an external source, use the following command:
•From the NME-APA# prompt, type logger get user-log file-name ftp://username:password@ipaddress/path and press Enter.
To copy the user log to an internal location, use the following command:
•From the NME-APA# prompt, type logger get user-log file-name target-filename and press Enter.
Viewing the User Log
Note This command is not recommended when the user log is large. Copy a large log to a file to view it.
To view the user log:
•From the prompt, type more user-log and press Enter.
The user log appears.
Clearing the User Log
You can clear the contents of the user log at any time. The user log contains important information regarding the functioning of the system. It is recommended that a copy be made before the log is cleared.
To clear the user log:
Step 1 From the prompt, type clear logger device user-file-log and press Enter.
Step 2 The system asks Are you sure?
Step 3 Type Y and press Enter.
Viewing the User Log Counters
Note This not applicable to APADC logs.
There are two types of log counters:
•User log counters—Count the number of system events logged from the SCE platform since the last reboot
•Non-volatile counters—Are not cleared at boot time
To view the user log counters for the current session, use the following command:
•From the prompt, type show logger device user-file-log counters and press Enter.
The logger lines information appears, followed by the prompt.
To view the non-volatile logger counters for both the User log file and the debug log file, use the following command:
•From the prompt, type show logger nv-counters and press Enter.
The non-volatile log counter information appears, followed by the prompt.
To view the non-volatile counter for the user-file-log only, use the following command:
•From the prompt, type show logger device user-file-log nv-counters and press Enter.
The user-file-log non-volatile log counter information appears, followed by the prompt.
Generating a File for Technical Support
In order for technical support to be most effective, the user should provide them with the information contained in the system logs. Use the logger get support-file command to generate a support file for the use of Cisco technical support staff.
To generate a log file for technical support:
•From the prompt, type logger get support-file filename and press Enter.
The support information file is created using the specified filename, and the prompt appears. This operation may take some time.
APADC Logging
APADC uses the Java Log4J software to generate the web application log. The generated log file is located at:
<installation-drive>:\log\SCAtEWebApp.log
The log will grow to a maximum size of 2.5MB, then will roll over to a new file in the same directory.
Additional log files with more detailed information can be found in the Apache Tomcat application server directory at:
<installation-drive>:\APADC\apache-tomcat-5.5.20\logs
Three different log files are available:
•jakarta_service_YYYYMMDD.log —The Tomcat web server
•stderr_ YYYYMMDD.log —The APADC application error log
•stdout_ YYYYMMDD.log —The APADC application trace/informative log
where YYYYMMDD is the date of installation of the application.
Operational Status
The operational status of the NME-APA blade can be displayed using the CLI command show system operation-status. The following table lists the operational states of the NME-APA.
Table 1-1 Operational States of the NME-APA
Operational Status
|
Description
|
Booting
|
Initial state after reset
|
Operational
|
The system becomes operational after completing the following process:
•Boot is completed
•Power self-tests are completed without failure
•Platform configuration is applied
|
Warning
|
The system is fully operational (as above) but one of the following occurred:
•Line ports (FE ports) to the link are down
•The management port link is down
•Temperature raised above threshold
•Insufficient space on the disk
Note If the condition that caused the platform to be in Warning state is resolved (for example, link is up) the platform reverts to Operational state.
|
Failure
|
The system is in Failure state after Boot due to one of the following conditions:
•Power-on test failure
•Three abnormal reboots in less than 20 minutes
•The platform is configured to enter Failure mode subsequent to a failure-induced reboot (this is configurable using a CLI command)
Note Depending on the cause of failure, the management interface and the platform configuration may or may not be active/available.
|
The APADC application maintains three operational states of managed NME-APA. The following table lists these operational states and explains possible related troubleshooting procedures.
Table 1-2 Operational States of the Managed NME-APA
Operational Status
|
Description
|
Troubleshooting
|
Offline
|
The device is unreachable by ICMP ping.
|
Check if the device can be pinged from the server running the APADC application.
If the device is reachable from the server, check the application logs to see why the device failed to become "Available".
|
Available
|
The device is reachable but has not been authenticated.
|
If APADC fails to "Connect" to the managed NME-APA, do the following:
1. Go to Admin >Admin User Management.
2. Verify that the NME-APA you are trying to connect is included under the "Configured Devices" for the APADC user trying to connect to the NME-APA.
3. Click on the device name under "Configured Devices" for the APADC user.
4. Verify that the device username on APADC also exists on the NME-APA through Telnet.
5. Verify that the passwords for the device user on APADC and NME-APA are identical.
6. Check the application logs for any error messages.
|
Connected
|
The device is reachable and authenticated.
|
Only one managed device at a time can be in this state.
Check the application logs if the user is unable to "Disconnect" from NME-APA. If the problem persists, restart APADC windows services as described in Starting and Stopping APADC Services. Restart the server if the problem still persists.
|
Signature, Protocol, and Service IDs
For protocol classification troubleshooting, the first thing is to make sure that appropriate signature, protocol, and service ID values are correct in the RDR report. The following table contains the values for the new protocols introduced in the Galaxy project for reference.
Table 1-3 Signature, Service and Protocol IDs
Protocol / Traffic Type
|
Signature ID (defined by Galaxy)
|
Protocol ID (defined by APADC)
|
Service ID (defined by APADC)
|
| |
|
|
|
PROTOCOL_TYPE_BLACKBOARD
|
0x200D0000
|
4011
|
44
|
| |
|
|
|
PROTOCOL_TYPE_FIX
|
0x200C0000
|
4010
|
43
|
PROTOCOL_TYPE_DICOM
|
0x200B0000
|
|
42
|
PROTOCOL_TYPE_HL7
|
0x200A0000
|
4008
|
41
|
PROTOCOL_TYPE_HL7_EMERGENCY
|
0x200A0100
|
4008
|
41
|
PROTOCOL_TYPE_HL7_PATIENT
|
0x200A0200
|
4008
|
41
|
PROTOCOL_TYPE_HL7_OTHERS
|
0x200A0300
|
4008
|
41
|
| |
|
|
|
PROTOCOL_TYPE_ORACLE
|
0x20010000
|
4000
|
400
|
| |
|
|
|
PROTOCOL_TYPE_CITRIX
|
0x20020000
|
4001
|
401
|
PROTOCOL_TYPE_CITRIX_ICA
|
0x20020100
|
4001
|
401
|
PROTOCOL_TYPE_CITRIX_CGP
|
0x20020200
|
4001
|
401
|
PROTOCOL_TYPE_CITRIX_IMA
|
0x20020300
|
4001
|
401
|
PROTOCOL_TYPE_CITRIX_SBM
|
0x20020400
|
4001
|
401
|
| |
|
|
|
PROTOCOL_TYPE_SAP
|
0x20030000
|
4002
|
402
|
| |
|
|
|
PROTOCOL_TYPE_AIM
|
0x20040000
|
714
|
28
|
PROTOCOL_TYPE_AIM_TEXT
|
0x20040100
|
714
|
28
|
PROTOCOL_TYPE_AIM_VOICE
|
0x20040200
|
714
|
28
|
PROTOCOL_TYPE_AIM_VIDEO
|
0x20040300
|
714
|
28
|
PROTOCOL_TYPE_AIM_MEDIA_CHAT
|
0x20040400
|
714
|
28
|
PROTOCOL_TYPE_AIM_FILEXFER
|
0x20040500
|
714
|
28
|
PROTOCOL_TYPE_AIM_PICSHARE
|
0x20040600
|
714
|
28
|
PROTOCOL_TYPE_AIM_GAME
|
0x20040700
|
714
|
28
|
| |
|
|
|
PROTOCOL_TYPE_MS_EXCHANGE
|
0x20050000
|
4004
|
4
|
| |
|
|
|
PROTOCOL_TYPE_GOOGLE_TALK
|
0x20060000
|
1030
|
28
|
PROTOCOL_TYPE_GOOGLE_TALK_VOICE
|
0x20060100
|
1030
|
28
|
PROTOCOL_TYPE_GOOGLE_TALK_FILEXFER
|
0x20060200
|
1030
|
28
|
| |
|
|
|
PROTOCOL_TYPE_VOICE_YAHOO
|
0x050B0000
|
45
|
37
|
PROTOCOL_TYPE_CHAT_YAHOO_MESSENGER
|
0x0B020000
|
40
|
28
|
| |
|
|
|
PROTOCOL_TYPE_MSSQL
|
0x20070000
|
4006
|
403
|
| |
|
|
|
PROTOCOL_TYPE_ACTIVEX
|
0x20080000
|
4007
|
404
|
| |
|
|
|
PROTOCOL_TYPE_MSN_MESSENGER
|
0x20090000
|
883
|
28
|
PROTOCOL_TYPE_MSN_MESSENGER_TEXT
|
0x20090100
|
883
|
28
|
PROTOCOL_TYPE_MSN_MESSENGER_VOICE
|
0x20090200
|
883
|
28
|
PROTOCOL_TYPE_MSN_MESSENGER_VIDEO
|
0x20090300
|
883
|
28
|
PROTOCOL_TYPE_MSN_MESSENGER_EMAIL
|
0x20090400
|
883
|
28
|
PROTOCOL_TYPE_MSN_MESSENGER_FILEXFER
|
0x20090500
|
883
|
28
|
PROTOCOL_TYPE_MSN_MESSENGER_GAME
|
0x20090600
|
883
|
28
|
PROTOCOL_TYPE_MSN_MESSENGER_
WHITEBOARD
|
0x20090700
|
883
|
28
|
Debug Flags for Protocol Libraries
The following flags can be used to collect debug traces for a specific protocol library.
Note Debug flags can be used only if a special SML debug image is loaded into the NME-APA. The release image is not compatible with debug flags.
To apply the PL debug flag:
NME-APA(config)#>interface linecard 0
NME-APA(config if)#>Tunnable PL_PT_ShowDebugReportForModule[Flag Value] value TRUE
The flag values for Protocol Libraries are listed in the following table.
Table 1-4 Debug Flags for Protocol Libraries
Debug Flag
|
Value
|
public const uint16 PL_ALL_MODULES
|
0
|
public const uint16 PL_UNIFICATION
|
1
|
public const uint16 PL_COMMON
|
2
|
public const uint16 PL_VOICE_SKYPE
|
3
|
public const uint16 PL_MAIL_SMTP
|
4
|
public const uint16 PL_BEHAVIORAL
|
5
|
public const uint16 PL_YAHOO
|
6
|
public const uint16 PL_VOICE_RTP
|
7
|
public const uint16 PL_VOICE_DINGOTEL
|
8
|
public const uint16 PL_P2P
|
9
|
public const uint16 PL_VOICE_SIP
|
10
|
public const uint16 PL_RTSP
|
11
|
public const uint16 PL_FTP
|
12
|
public const uint16 PL_HTTP
|
13
|
public const uint16 PL_MMS
|
14
|
public const uint16 PL_TFTP
|
15
|
public const uint16 PL_MAIL_IMAP
|
16
|
public const uint16 PL_MAIL_MIME
|
17
|
public const uint16 PL_NNTP
|
18
|
public const uint16 PL_MAIL_POP3
|
19
|
public const uint16 PL_SSL
|
20
|
public const uint16 PL_DHCP
|
21
|
public const uint16 PL_RADIUS
|
22
|
public const uint16 PL_H323
|
23
|
public const uint16 PL_MGCP
|
24
|
public const uint16 PL_PTT
|
25
|
public const uint16 PL_RTCP
|
26
|
public const uint16 PL_SDP
|
27
|
public const uint16 PL_SKINNY
|
28
|
public const uint16 PL_SMPP
|
29
|
public const uint16 PL_WAP
|
30
|
public const uint16 PL_DNS
|
31
|
public const uint16 PL_SSDP
|
32
|
public const uint16 PL_AGG_AGING
|
33
|
public const uint16 PL_BASIC
|
34
|
public const uint16 PL_HITLESS_UPGRADE
|
35
|
public const uint16 PL_CUWORLD
|
36
|
public const uint16 PL_ICQ
|
37
|
public const uint16 PL_JABBER
|
38
|
public const uint16 PL_STUN
|
39
|
public const uint16 PL_CITRIX
|
40
|
public const uint16 PL_AIM
|
41
|
public const uint16 PL_MS_EXCHANGE
|
42
|
public const uint16 PL_MSSQL
|
43
|
public const uint16 PL_GOOGLETALK
|
44
|
public const uint16 PL_ACTIVEX
|
45
|
public const uint16 PL_MSN_MESSENGER
|
46
|
public const uint16 PL_CITRIX
|
47
|
public const uint16 PL_AIM
|
48
|
public const uint16 PL_MS_EXCHANGE
|
49
|
public const uint16 PL_MSSQL
|
50
|
public const uint16 PL_GOOGLETALK
|
51
|
public const uint16 PL_ACTIVEX
|
52
|
public const uint16 PL_MSN_MESSENGER
|
53
|
public const uint16 PL_BLACKBOARD
|
54
|
public const uint16 PL_HL7
|
55
|
Note PL_HITLESS_UPGRADE (35) will be used for the new features added in hitless upgrade.
Debug Flags for SML Components
The following flags can be used to collect debug traces for a software component in the Galaxy SML application.
Note Debug flags can be used only if a special SML debug image is loaded into the NME-APA. The release image is not compatible with debug flags.
To apply the SML debug flag:
NME-APA(config)#>interface linecard 0
NME-APA(config if)#>Tunnable APP_PT_ShowDebugReportForModule[Flag Value] value TRUE
The flag values for SML components are listed in the following table.
Table 1-5 Debug Flags for SML Components
Debug Flag
|
Value
|
public const uint16 APP_ALL_MODULES
|
0
|
public const uint16 APP_MAIN
|
1
|
public const uint16 APP_UNIFICATION
|
2
|
public const uint16 APP_CLASSIFY
|
3
|
public const uint16 APP_EXTERN_CALLBACKS
|
4
|
public const uint16 APP_LISTNERS
|
5
|
public const uint16 APP_PARTY
|
6
|
public const uint16 APP_HANDLERS
|
7
|
public const uint16 APP_REPORTS
|
8
|
public const uint16 APP_MAX_MODULE
|
9
|
