Troubleshooting CBT 3.3


Revised: June 28, 2007, OL-13930-01

This section contains the following procedures for verifying and troubleshooting CBT 3.3:

Troubleshooting Continuous Sweep Spectrum Operation in CBT 3.3

Changing Server Ports in XML Script

Saving System Message Logs for Troubleshooting

Verifying Installation Status on the CBT 3.3 Server

Verifying the CBT 3.3 License

Troubleshooting the Poller

Troubleshooting Poller Problems

Troubleshooting Continuous Sweep Spectrum Operation in CBT 3.3

A non-standard behavior has been observed in which the Continuous Spectrum Operation of CBT 3.3 fails, even in circumstances in which the Single Sweep Spectrum Operation remains functioning. One example would be the proper Trace Window single sweep operation, but with failed Continuous Sweep behavior in the Trace Window.

CBT 3.3 requires that the following processes and tasks be used for Continuous Sweep Operation.


Step 1 Verify that the TCP ports used by CBT 3.3 are as follows:

Port 8105: JVM server port

Port 9080: Non-SSL HTTP port

Port 9443: SSL HTTP port

Port 9082: AJP 1.3 Connector

Port 2640: For Sybase database connectivity

Port 8020: For Poller operation

Step 2 Implement the following TCP port ranges for spectrum operations. Spectrum events such as Trace Window, Spectrogram, CNR Trending, and the Generic Query use a specified range of port numbers for operation.

For both Trace Window events and the Spectrogram events, the port numbers range from 2100 to 2500.

For the Spectrum tools and CNR Trending, port numbers range from 3100 to 3500.

For the Diagnostics tools and Generic Query, the port numbers range from 5100 to 5500.

The usage of these port numbers should in increments of 1. That is, for the first Trace Window launched, the port number is 2100, and the next Trace Window port 2101. For the next Spectrogram window, the port number is 2102 and so forth up to port 2500.


Changing Server Ports in XML Script

Cisco IOS Release 12.3(9a)BC changes the Tomcat server port in server XML script. The script is set to port 8105. This prevents possible port conflict when multiple Tomcat Web servers are running on the same workstation.

If required, perform the following steps to change the Tomcat server port in XML script to port 8105 and check for additional port conflicts:


Step 1 Edit the server.xml script in the following location:

/opt/CSCOcbt/httpServer/conf

Step 2 Check for all ports numbers that have conflicts with CBT and change all port numbers that have such conflicts:

Port 8105: JVM server port

Port 9080: Non-SSL HTTP port

Port 9443: SSL HTTP port

Port 9082: AJP 1.3 Connector


Saving System Message Logs for Troubleshooting

When troubleshooting CBT 3.3, we recommend that message logs be saved and filtered using the following steps.


Step 1 To view saved message logs, click Utilities then Message log in the user interface.

This is a fixed-size log file that continuously removes the oldest entry as it is updated. This log file is not viewable using an editor (such as vi).

Step 2 Filter the message log entries by severity, by user, by module, or by date.

Step 3 To set the logging level, click Configuration then Message Log in the user interface. Setting the logging level helps to prevent unnecessary messages occupying the fixed-size log file.

For Solaris and Linux, the log file is located at /opt/CSCOcbt/httpServer/logs/catalina.out. This file is viewable with viewing utilities such as vi, Cat, Tail, or others.


Note This message log file grows in size. For Windows, these messages go to the console and a message log file is not created.



Verifying Installation Status on the CBT 3.3 Server

Perform the following steps to verify the status of the installed CBT 3.3 server.


Step 1 Verify the correct file location. There should be no exceptions in the catalina.out file located as follows:

Solaris and Linux—/opt/CSCOcbt/httpServer/logs/

Windows—The CBT 3.3 console

Step 2 Verify that the dbeng8 process is running:

Solaris and Linux:

ps -ef|grep dbe
root 26449 1 0 Aug 18 ? 0:24 dbeng8 -x tcpip{ServerPort=2640} -q -ud -s local0 -m -c 
16M -n cbtdbengine /opt

Windows:

There is a Sybase icon shown as a running process.

Step 3 Verify that the following Java processes are running:

Solaris and Linux:

ps -ef|grep java
root 26478  1 0 Aug 18 ?  0:39 /opt/CSCOcbt/jre/bin/java -DCBTpoller -cp 
/opt/CSCOcbt/httpServer/webapps/ROOT/

root 26489 1 0 Aug 18 ?  148:55 /opt/CSCOcbt/jre/bin/java 
-Djava.endorsed.dirs=/var/CSCOcbt/httpServer/common/e

Note Linux displays threads; therefore, there are many Java entries.


Windows:

Use the command console for Tomcat and the CBT poller to view log messages.


Verifying the CBT 3.3 License

Perform the following steps to verify the CBT 3.3 installation license.


Step 1 For Solaris and Linux, perform these steps:

1. Verify that the license is present and the file is correct in the following location:

/opt/CSCOcbt/httpServer/webapps/ROOT/WEB-INF/classes/lic/License

2. Log in and allow for error prompts to identify any issues.

3. Scan the catalina.out file for error messages in the following location:

/opt/CSCOcbt/httpServer/logs/

Step 2 For Windows, perform these steps:

1. Verify that the license is present and the file is correct in the following location:

INSTALLATION_DIR\httpServer\webapps\ROOT\WEB-INF\classes\lic\License

2. Log in and allow for error prompts to identify any issues.

3. Scan the command console for Tomcat and see if there are error messages in the console.


Troubleshooting the Poller

This section provides directions to troubleshoot error messages for the following CBT components:

Script—Retrieves subscriber or provisioning information from an external source.

Application on an HTTP server—Retrieves subscriber or provisioning information from an external source.

Poller—Queries CMTSs by using SNMP and stores provisioning information in the CBT local Sybase database.

Troubleshooting Poller Problems

You can use the Poller, a separate Java application, to store provisioning information in the CBT local Sybase database. This database Poller queries the CMTSs by using SNMP. If the Debug parameter in POLLER.INI is set to true, the Poller sends information to the Poller.log file in the following location:

Linux and Solaris—/opt/CSCOcbt/logs

Windows—%CSCOCBT_ROOT%\logs


Note See the "Parameters in the POLLER.INI File" section on page 31 for an explanation of each parameter in POLLER.INI.


This section:

Shows output from Poller.log that indicates there is a problem

Describes the cause of the problem

Provides a resolution for the problem

Error Message: No Router

Problem

The following output from Poller.log indicates that no router was found:

Thu Jun 27 15:47:35 EDT 2002: INFO: Start polling...
Thu Jun 27 15:47:37 EDT 2002: INFO: No router found in routers list file: 
/opt/CSCOcbt/jakarta-tomcat-4.0.3/webapps/ROOT/WEB-INF/classes/config/myrouters
Thu Jun 27 15:47:37 EDT 2002: INFO: Finished polling... elapsed time: 2371 
milliseconds
Thu Jun 27 15:47:37 EDT 2002: INFO: Poller exited.

Cause

No router was found in the routers list file, myrouters, because of one of the following conditions:

The routers list file does not exist. This is the case when you first install CBT.

The CMTS routers list is empty; it does not contain any routers.

Resolution

To add routers in CBT, see the "Adding Router Information" section on page 25.

Error Message: Database Login Failed

Problem

The following output from Poller.log indicates that the login to the database failed:

Wed Jun 26 11:14:12 PDT 2002: INFO: Start polling...
Wed Jun 26 11:14:13 PDT 2002: FATAL: SQL Exception in setupDB makeDBConnection: 1
JZ00L: Login failed. Examine the SQLWarnings chained to this exception for the 
reason(s).
Software aborted.

Cause

The database login failure occurs because the database ran out of connections; that is, it tried to use more connections than it can support.

Resolution

If the database ran out of connections, restart CBT to clear all existing database connections and start the database.

Error Message: SNMP Timeout

Problem

The following output from Poller.log indicates that an SNMP timeout has occurred:

Thu Jun 20 13:45:08 EDT 2002: INFO: Start polling...
Thu Jun 20 13:45:18 EDT 2002: ERROR: Thread16 got SNMP exception on Router: 
24.216.122.254. Exception: Snmp Timeout... Router 24.216.122.254 is unreachable

Cause

The SNMP timeout occurs when the network connection to the router, or the router itself, is so busy that it cannot satisfy the request within the specified time.

Resolution

If the SNMP timeout occurs too frequently, change the SNMP settings in the POLLER.INI file:


Step 1 Open the POLLER.INI file for your operating system:

Linux and Solaris—/opt/CSCOcbt/bin

Windows—%CSCOCBT_ROOT%\bin

Step 2 Increase one or both of the following settings:

SnmpTimeout—You can increase the number of seconds before the attempt to create the SNMP connection reaches a timeout.

SnmpRetry—You can increase the number of times to try to create the SNMP connection.

Step 3 To activate the new settings, restart the CBT server.



Tip Check for incorrect SNMP community strings, because an invalid community string for read/write will also result in an SNMP timeout.