Guest

Cisco Configuration Engine

Troubleshooting Guide for Cisco Configuration Engine

Table Of Contents

Troubleshooting Guide for
Cisco Configuration Engine

Checking the Version Number of Cisco Configuration Engine

Troubleshooting Logging and Connection Issues

Cannot Log in to the System

System Cannot Connect to the Network

Cannot Connect to the System Using a Web Browser

Problems Connecting to the System with Secure Shell

Cannot Connect to the System Using Telnet

Troubleshooting the Open Lightweight Directory Access Protocol and the Berkeley Data Base

OpenLDAP Server Not Responding

BDB Using Excessive Disk Space

Troubleshooting the Cisco Configuration Engine Services

Cisco Configuration Engine Not Working Properly

XML Request Sent but No Response Received

Troubleshooting the Web Services

Cisco Configuration Engine GUI Not Displaying

Undeploying Services

Connection Timeout Error Message When Working with
Cisco Networking Services Agents

Troubleshooting the Initial Configuration

Initial Configuration Does Not Work

Log Files to Monitor Event Traffic and Cisco Configuration Engine Process Status

Troubleshooting a Configuration Update

CNS-Enabled Device Unable to Connect with Cisco Configuration Engine

CNS-Enabled Device Configuration Update Failed

Configuration Update Stuck in Queue After Data Migration

Configuration Update Stuck in Queue After Data Backup and Restore

Troubleshooting an Image Update

Information About Log Files

Cannot Activate Image

Activation Failed Due to Device Error

Error Message: Image Update Stopped

Troubleshooting IMGW

Obtaining Detailed Debugging Information

Troubleshooting the Router

Enabling Debugging on the Router

General Troubleshooting

Error Message: Failed to Create the Device on Remote Database

cns-listen Command Failed to Execute

HTTPD Is Down When Crypto Is Enabled

Web Service Deployment Error When Crypto Is Enabled

Backup and Restore Not Working Properly

Device Status Changed from Green to Red After Setup

Cannot Back Up Jobs


Troubleshooting Guide for
Cisco Configuration Engine


Revised: Month day, year, OL-17767-01

This document contains troubleshooting information for the Cisco Configuration Engine. It contains the following sections:

Checking the Version Number of Cisco Configuration Engine

Troubleshooting Logging and Connection Issues

Troubleshooting the Open Lightweight Directory Access Protocol and the Berkeley Data Base

Troubleshooting the Cisco Configuration Engine Services

Troubleshooting the Web Services

Troubleshooting the Initial Configuration

Troubleshooting a Configuration Update

Troubleshooting an Image Update

Troubleshooting IMGW

Troubleshooting the Router

General Troubleshooting


Note This is not an administration manual. For comprehensive information about administering the
Cisco Configuration Engine, see the Cisco Configuration Engine Administration Guide.


Checking the Version Number of Cisco Configuration Engine

To check the version number of the Cisco Configuration Engine software, do one of the following:

Start the Cisco Configuration Engine application, and look for the version number in the displayed login screen.

Use the version command. This command is located in the
cd $CISCO_CE_INSTALL_ROOT/CSCOcnsie/bin directory.

Troubleshooting Logging and Connection Issues

To troubleshoot logging and connection issues, see the following sections:

Cannot Log in to the System

System Cannot Connect to the Network

Cannot Connect to the System Using a Web Browser

Problems Connecting to the System with Secure Shell

Cannot Connect to the System Using Telnet

Cannot Log in to the System

Problem   You cannot log in to the system.

Possible Cause    This problem could occur for one of the following reasons:

You did not run the Setup program to create the initial system configuration.

You lost all of the user account passwords.

Solution   To resolve this problem, follow these steps:


Step 1 If you did not run the Setup program, run the Setup program as described in the Cisco Configuration Engine Solaris Installation & Configuration Guide, 3.0.

Step 2 If you do not know the passwords for the system user accounts, reconfigure the system to create a new user account.

Step 3 If you still cannot log in to the system, contact the Cisco Technical Assistance Center (TAC) for assistance.


System Cannot Connect to the Network

Problem   The system cannot connect to the network.

Possible Cause    This problem could occur for the following reasons:

The network cable is not connected to an Ethernet port.

The Ethernet interface is disabled or misconfigured.

The system is configured correctly, but the network is down or misconfigured.

The system is not configured correctly.

Solution   To resolve this problem, follow these steps:


Step 1 Verify that the network cable is connected to an Ethernet port and that the Link light is on.

If the network cable is not connected, connect it.

If the network cable is connected but the Link light is not on, check these probable causes:

The network cable is faulty.

The network cable is the wrong type (for example, a crossover type is used, instead of the required straight-through type).

The port on the default gateway to which the system connects is down.

Step 2 If you still cannot connect to the network, use the ping command to perform the following tests:

a. Try to connect to a well-known host on the network. A DNS server is a good target host.

If the ping command can reach the well-known host, the system is connected to the network. If it cannot connect to the host, the problem is with the network configuration or the host. Contact your network administrator for assistance.

b. If the ping command cannot reach the well-known host, try to reach another host on the same subnet as the system.

If the ping command can reach a host on the same subnet, but cannot reach a host on a different subnet, the default gateway is probably down or misconfigured.

Step 3 If the ping command cannot reach any hosts, use the ifconfig command to determine whether the Ethernet interface is disabled or misconfigured.

If the Ethernet interface is disabled, enable it. If it is misconfigured, configure it correctly.

Step 4 If the interface is enabled and correctly configured but you still cannot connect to the network, ensure that all network setting are configured correctly. Run the Setup program again by entering the setup command in the shell prompt.


Note You cannot run the Setup program a second time by logging in as setup. For security reasons, the account is disabled after it is used once successfully.


Step 5 Contact your network administrator to verify that there are no conditions on the network that prevent the system from connecting to the network.

Step 6 If no conditions are preventing the system from connecting to the network, contact the Cisco TAC for assistance.


Cannot Connect to the System Using a Web Browser

Problem   You cannot connect to the system by entering its IP address in a web browser.

Possible Cause    This problem could occur for the following reasons:

The system cannot connect to the network.

Encryption is enabled (plain text is disabled).

The HTTP service is not running.

Solution   To resolve this problem, follow these steps:


Step 1 Make sure that the system can connect to the network.

If it cannot connect to the network, see the "System Cannot Connect to the Network" section for possible resolution.

Step 2 Try to connect to the system by using a web browser.

If encryption is enabled:

Use https://... to connect.

Verify that the certificate is correct.

Step 3 If you still cannot connect, stop and start the web server by entering the following commands:
/etc/rc.d/init.d/httpd stop
/etc/rc.d/init.d/httpd start

If the LDAP directory contains thousands of devices, restart and wait 20 minutes.

Step 4 Repeat Step 2.

Step 5 If you cannot connect, restart the system.

If the LDAP directory contains thousands of devices, restart and wait 20 minutes.

Step 6 If you still cannot connect to the system, contact the Cisco TAC for assistance.


Problems Connecting to the System with Secure Shell

Problem   When connecting to the system using Secure Shell (SSH), you experience one of these problems:

You cannot connect to the system.

The system is extremely slow, even though it is connected to the network.

The system cannot correctly process requests from management applications.

Possible Cause    The system cannot obtain DNS services from the network.

Solution   To resolve this problem, follow these steps. Connect to the console if you cannot connect by using SSH.


Step 1 Do one of the following:

Set up the name servers properly by editing the /etc/resolv.conf file.

Re-execute Setup.

Step 2 Verify that the system can obtain Domain Name System (DNS) services from the network by entering the following command:
# host <dns-name>

where <dns-name> is the DNS name of a host on the network that is registered in DNS. When you enter this command, it responds with the IP address of the host.

If the system cannot resolve DNS names to IP addresses, the DNS server is not working properly.

Step 3 Resolve the network DNS problem.

Step 4 If the system can resolve DNS names to IP addresses but you still cannot connect to the system using SSH, contact the Cisco TAC for assistance.


Cannot Connect to the System Using Telnet

Problem   You cannot connect to the system by using Telnet even though the system is connected to the network.

Possible Cause    This problem could occur if the Telnet service is disabled on the system.

Solution   To resolve this problem, use SSH to connect to the system.

Troubleshooting the Open Lightweight Directory Access Protocol and the Berkeley Data Base

To troubleshoot Open Lightweight Directory Access Protocol (OpenLDAP) and Berkeley Data Base (BDP), see the following sections:

OpenLDAP Server Not Responding

BDB Using Excessive Disk Space

OpenLDAP Server Not Responding

Problem   The OpenLDAP server is not responding.

Possible Cause    This problem could occur if, after a system crash, power outage, or manual shutdown, the OpenLDAP sever did not shut down gracefully, which caused data corruption.

Solution   To resolve this problem, stop the OpenLDAP server, and then recover the data. Follow these steps:


Step 1 To stop the OpenLDAP server, enter the following command:
/etc/init.d/NetAppOpenLDAP stop

Step 2 To recover the data, enter the following command:
$CISCO_CE_INSTALL_ROOT/bdb/bin/db_recover -h $CISCO_CE_INSTALL_ROOT/openldap/var/openldap-data

BDB Using Excessive Disk Space

Problem   The BDB is using excessive disk space.

Possible Cause    BDB creates transaction logs in the $CISCO_CE_INSTALL_ROOT/openldap/var/openldap-data file. If transaction logs are not purged, the BDB uses excessive disk space.

For information about disk space, see the "System Requirements" and "Understanding Disk Space Calculation" sections in the Cisco Configuration Engine Installation and Configuration Guide, 3.0.

Solution   To resolve this problem, follow these steps:


Step 1 To verify whether dbpurge.sh is running as a cron job, enter the following command:
crontab -l

Step 2 If crontab -l is not in the list, run the Setup program to add it.

Step 3 To manually purge BDB transaction logs, enter the following command:
$CISCO_CE_INSTALL_ROOT/CSCOcnsie/bin/dbpurge.sh


Troubleshooting the Cisco Configuration Engine Services

To troubleshoot Cisco Configuration Engine Services services, see the following sections:

Cisco Configuration Engine Not Working Properly

XML Request Sent but No Response Received

Cisco Configuration Engine Not Working Properly

Problem   The Cisco Configuration Engine is not working properly.

Possible Cause    This could occur if any of the processes fails.

Solution   Use the Cisco Configuration Engine (CE) Monitor feature to check the status of the processes. The CE Monitor checks the status of a set of processes at a configured time interval and reports the status in the /var/log/CNSCE/ce_monitor/ce_monitor.log file. The CE Monitor exits if any of the processes fails.

To check the status of the processes, follow these steps:


Step 1 Check the status of CE Monitor to determine whether the service is up or down:

For Linux, enter: /etc/rc.d/init.d/MonitorCE status

For Solaris, enter: /etc/init.d/MonitorCE status

Step 2 Check the /var/log/CNSCE/ce_monitor/ce_monitor.log file to identify which process is down.

Step 3 If a particular process is down, check the process to determine the problem.


XML Request Sent but No Response Received

Problem   An XML request was sent, but you did not get a response.

Solution   To resolve this problem, do the following in any order:

To monitor events on the bus, use the cns-listen utility.

For Intelligent Modular Gateway (IMGW) devices, do the following:

Set the IMGW logging level to verbose.

Check the following log files under the /var/log/CNSCE/imgw directory:

IMGW-LOG-<hostname> (log file for the IMGW runtime)

IMGW-DEVMOD-LOG (log file for debugging the IMGW script)

For agent-enabled devices, configure cns debug on the router.

Troubleshooting the Web Services

To troubleshoot Web Services, see the following sections:

Cisco Configuration Engine GUI Not Displaying

Undeploying Services

Connection Timeout Error Message When Working with Cisco Networking Services Agents

Troubleshooting the Initial Configuration

Cisco Configuration Engine GUI Not Displaying

Problem   The Cisco Configuration Engine GUI is not displaying.

Solution   To resolve this problem, follow these steps:


Step 1 Check whether the Cisco Configuration Engine service endpoint is up. Go to:
http://<CE hostname>/cns/services/<services>. If the web page is displayed, the service is up.

Step 2 If the web page is not displayed, check the httpd status (web server status).

Step 3 If the httpd status is okay, deploy all or individual services.

Go to: cd $CISCO_CE_INSTALL_ROOT/CSCOcnsie/bin.

To deploy all services, enter the following command:
./deploy.all.websvc

To deploy an individual service, enter the following command:
./deploy.<service>.websvc


Undeploying Services

Problem   How do I undeploy services?

Solution   To undeploy services, follow these steps:


Step 1 Go to: cd $CISCO_CE_INSTALL_ROOT/CSCOcnsie/bin.

Step 2 To undeploy all services, enter the following command:
./undeploy.all.websvc

Step 3 To undeploy an individual service, enter the following command:
./undeploy.<service>.websvc


Connection Timeout Error Message When Working with
Cisco Networking Services Agents

Problem   When working with Cisco Networking Services (CNS) agents, you get a Connection Timeout error message.

Solution   To resolve this problem, do the following in any order:

Make sure that the CNS agent is enabled and is configured correctly:

CEConfigService requires CNS Config Agent.

acquireConfig() requires CNS Exec Agent.

CEImageService requires CNS Image Agent.

CEExecService requires CNS Exec Agent.


Note Do not use execImmediate() and execImmedWithConversation() to send 12.4 XML payloads to 12.3 agents.


Check the log files. The following log files are located in the /var/log/CNSCE/ directory:

websvc/websvc.log (web service general log)

cfgsrv/cfgsrv/log (config service log)

imgsrv/imgsrv.log (image service log)

cfgsrv/exec-srv.log (exec service log)

Monitor the Event Bus. Go to: cd $CISCO_CE_INSTALL_ROOT/CSCOcnsie/tools. Then enter the following command:
./cns-listen "cisco.>"

Monitor the Simple Object Access Protocol (SOAP) XML payload. Go to:
cd $CISCO_CE_INSTALL_ROOT/CSCOcnsie/tools. Then enter the following command:
./ssldump -d port 80

Troubleshooting the Initial Configuration

To troubleshoot the initial configuration, see the following sections:

Initial Configuration Does Not Work

Log Files to Monitor Event Traffic and Cisco Configuration Engine Process Status

Initial Configuration Does Not Work

Problem   The initial configuration does not work.

Solution   To resolve this problem, follow these steps:


Step 1 Make sure that you can access the device from Cisco Configuration Engine and that you can access
Cisco Configuration Engine from the device.

Use the ping command to validate connectivity.

Step 2 Make sure that the device is agent-enabled.

In router configuration mode, enter cns ?. If the cns command list is displayed, the device is agent-enabled. If the device is not agent-enabled, this command fails.

Step 3 Make sure that the Cisco Configuration Engine is set up properly.

Cisco Configuration Engine is set up in either crypto or plaintext mode. Make sure that the device setup and the Cisco Configuration Engine setup are consistent.

Step 4 Make sure that the system processes are running properly. Enter the following on the
Cisco Configuration Engine server:

To verify that all TibGates are up, enter the following command:
ps -ef | grep tibgate


Note For information about TibGate event gateway ports, see the "Scalability Among Event Gateway Ports" chapter in the Cisco Configuration Engine Installation and Configuration Guide, 3.0.


To verify that httpd is up, enter the following command:
httpd status

To verify that the Java process is up, enter the following command:
ps -ef | grep -i java | grep ConfigEngine

Step 5 Check the object status for the device in Cisco Configuration Engine. If the status is green, the Cisco Configuration Engine and the device are connected.

If the status is red, verify that the Event ID and Config ID match with what is defined on the device. From the Cisco Configuration Engine user interface, do the following:

a. Choose Devices > Edit Device. The Edit Device page appears with a Groups list.

b. From the Groups list, choose the group that contains the device, then click the icon for the device.

c. From the left pane, choose Edit Information. The Enter Device Information page appears.

d. Click Next. The Select Group Membership page appears.

e. Click Next. The Device IDs page appears.

f. Verify that the Event ID matches with what is defined on the router.

Step 6 Verify the agent setup on the device.

In non-configuration mode, enter the show run command to display the agent settings that are running. Then verify the following:

ip host <ce_host.domain_name> <ce_ipaddress>

cns trusted-server <ce_host.domain>

cns trusted-server all-agents <ce_host.domain_name>

cns id string <ce_ipaddress>

cns id string <ce_ipaddress> event

cns event <ce_ipaddress> <event-gateway port>

cns config init <ce_ipaddress>

cns exec

Step 7 If the authentication feature is enabled in Cisco Configuration Engine, make sure that the device password (cns password <password string>), matches with what is defined in the
Cisco Configuration Engine user interface.


Note You cannot see the password setting after you have configured it on the router, nor can you edit the password in Cisco Configuration Engine. Therefore, you must reset the password. To reset the password, use the resync device feature in Cisco Configuration Engine.


Step 8 If you have tried all of the preceding steps but the initial configuration still does not work, use the
debug cns config all command to enable debugging on the agent. Analyze the output to verify that the agent is set up correctly with proper connectivity.

Step 9 If the initial configuration still does not work, reboot the device.


Log Files to Monitor Event Traffic and Cisco Configuration Engine Process Status

Use the following log files to monitor event traffic and Cisco Configuration Engine process status:

/var/log/CNSCE/cfgsrv/cfgsrv.log, error.log—Check the cfgsrv log file when the config agent is enabled and initial configuration is issued on the device.

/var/log/CNSCE/evtgateway/TibGateLog-<port>—Check the TibGate log file when the event agent is enabled on the device.

/var/log/httpd/*.log, /var/log/CNSCE/tomcat/*.out, *.txt, *.log—Check the Apache & Tomcat log files to make sure that the web server is running properly.

/var/log/CNSCE/appliance-setup.log—Check the setup log file for Cisco Configure Engine setup, especially in crypto setup mode.

/var/log/CNSCE/websvc—Check the web service log file to see whether the application programming interface (API) is invoked.

Troubleshooting a Configuration Update

To troubleshoot a configuration update, see the following sections:

CNS-Enabled Device Unable to Connect with Cisco Configuration Engine

CNS-Enabled Device Configuration Update Failed

Configuration Update Stuck in Queue After Data Migration

Configuration Update Stuck in Queue After Data Backup and Restore

CNS-Enabled Device Unable to Connect with Cisco Configuration Engine

Problem   A device is created in the Cisco Configuration Engine user interface but the device indicator displays a red status.

Possible Cause    The red status indicates that the device is unable to connect with Cisco Configuration Engine or it is still trying to connect. A connection delay might occur due to the device setting of the backoff timer. If after the time has expired, the indicator does not turn to green, follow the steps given below.

Solution   To resolve this problem, follow these steps:


Step 1 Make sure that the Event ID and Config ID match with what is defined on the device. Do the following from the Cisco Configuration Engine user interface:

a. Choose Devices > Edit Device. The Edit Device page appears with a Groups list.

b. From the Groups list, choose the group that contains the device, then click the icon for the device.

c. From the left pane, choose Edit Information. The Enter Device Information page appears.

d. Click Next. The Select Group Membership page appears.

e. Click Next. The Device IDs page appears.

f. Verify that the Event ID and Config IP match with what is defined on the router.

Step 2 Make sure that the device type is Agent Enabled Device. From the Cisco Configuration Engine user interface, do the following:

a. Choose Devices > Edit Device. The Edit Device page appears with a Groups list.

b. From the Groups list, choose the group that contains the device. Then click the icon for the device.

c. From the left pane, choose Edit Information. The Enter Device Information page appears.

d. Verify that the device type is Agent Enabled Device.

Step 3 Ping or telnet to the device to verify that the device is reachable from Cisco Configuration Engine.

Step 4 From the Cisco Configuration Engine server, make sure that TibGate, httpd, and the Java process are up.

To verify that all TibGates are up, enter the following command:
ps -ef | grep tibgate


Note For information about TibGate event gateway ports, see the "Scalability Among Event Gateway Ports" chapter in the Cisco Configuration Engine Installation and Configuration Guide, 3.0.


To verify that httpd is up, enter the following command:
httpd status

To verify that the Java process is up, enter the following command:
ps -ef | grep -i java | grep ConfigEngine

Step 5 Check the following on the device:

a. Make sure that the following Event ID string is defined:
cns id string <id string>
cns id string <id string> event

The default value of the <id string> is the hostname of the device. This ID must be the same as the Config ID defined in the Cisco Configuration Engine host.

b. To verify that the Cisco Configuration Engine hostname or IP address is specified to receive the events, enter the following command:
cns event <configengine hostname or ip address> 11011 keepalive 30 10

c. To verify that the Cisco Configuration Engine hostname or ip address is reachable from the device, enter the following command:
ping <configengine hostname or ip address>

d. If you are unable to reach the device through the ping command, use the ip host command to configure the device:
ip host <hostname> <ip address>
ip host <hostname.domainame> <ip address>

e. (Optional) To resolve hostnames, set up DNS on the device by entering the following command:
ip name-server <ip address of DNS>

Step 6 If the device status changes from green to red after Cisco Configuration Engine setup, follow the steps in "Device Status Changed from Green to Red After Setup" section.


CNS-Enabled Device Configuration Update Failed

Problem   The Device configuration update fails.

Solution   To resolve this problem, follow these steps:


Step 1 Check the following on the Cisco Configuration Engine:

a. Make sure that the Event ID and Config ID match with what is defined on the device.

b. Make sure that the object status for the device in Cisco Configuration Engine is green. Green indicates that the Cisco Configuration Engine and the device are connected.

c. To verify that TibGate is up and running, enter the following command:
ps -ef | grep tibgate


Note If encryption is enabled, the TibGate ports begin with even numbers that begin from 11012. If encryption is not enabled, the TibGate ports begin with odd numbers that begin from 11011. Each TibGate port can support a a maximum of 500 devices. You specify the number of the TibGates during the Cisco Configuration Engine Setup program. Make sure that the number of devices on each TibGate port does not exceed the maximum. For details, see the "Scalability Among Event Gateway Ports" chapter in the Cisco Configuration Engine Installation and Configuration Guide, 3.0.


d. If the authentication feature is enabled in the Cisco Configuration Engine, make sure that the device password (cns password <password string>) that is defined in the Cisco Configuration Engine user interface, matches with what is defined on the device. Otherwise, use the resync device command to reset the CNS password.

To use the resync command from the Cisco Configuration Engine user interface, do the following:

a. Go to Devices > Resync Device. The Resync Device page appears with a Groups list.

b. From the Groups list, choose the group that contains the device you want to resynchronize. Then click the icon for the device.

c. In the confirmation window, click Ok.

e. Make sure that the downloading configuration semantics and syntax for the device are correct.

f. If the device in the Cisco Configuration Engine was initially set as None, then deleted, and then re-created as an agent-enabled device, you must rename the Config ID and Event ID on both the device and the Cisco Configuration Engine user interface.

g. If during the Cisco Configuration Engine setup, a port other than the default port 80 is configured for HTTP, make sure that the same port number is also configured on the device.

Step 2 Check the following on the device:

a. Make sure that the following Event ID string is defined:
cns id string <id string>
cns id string <id string> event

The default value of the <id string> is the hostname of the device. This ID must be the same as the Config ID defined in the Cisco Configuration Engine host.

b. To verify that the Cisco Configuration Engine hostname or IP address is specified to receive events, enter the following command:
cns event <configengine hostname or ip address> 11011 keepalive 30 10


Note Make sure that the TibGate port of this device is correct. The TibGate port must match the port that is defined in the Cisco Configuration Engine.


c. If the authentication feature is enabled in the Cisco Configuration Engine, make sure that the device password (cns password <password string>) matches what is defined in the
Cisco Configuration Engine user interface.


Note You cannot see the password setting after you configure it on the router, nor can you edit the password in Cisco Configuration Engine. Therefore, you must reset the password. To reset the password, use the Resync Device feature in the Cisco Configuration Engine.


d. During the Cisco Configuration Engine setup, if a port other than the default port 80 is configured for HTTP, make sure that the same port number is also configured on the device. To configure the http port on the device, enter the following command:
cns config partial <CE hostname> <http port>

Step 3 If you have tried all the preceding steps and the device configuration update still fails, enable the debugging tools.

In the Cisco Configuration Engine host, do the following:

To start event listener, enter the following commands:
cd $CISCO_CE_INSTALL_ROOT/CSCOcnsie/tools
./cns-listen "cisco.>"

Check the cfgsrv log file. This file is located at: /var/log/CNSCE/cfgsrv/cfgsrv.log.

In the device, use the debug cns config all command to enable debugging. Analyze the output to verify that the device is set up correctly with proper connectivity.

Step 4 Rerun the scenario, check the event traffic and the information from the device, capture the data, and then contact the Cisco TAC for assistance.


Configuration Update Stuck in Queue After Data Migration

Problem   The configuration update is stuck in queue after data migration.

Possible Cause    This problem could occur if you did not enter the correct country code and company code information during the Setup program.

Solution   After data migration from release 2.0 to 3.0, the OpenLDAP schema is transferred to a new host. To reuse the existing OpenLDAP schema for the new host, make sure that the country code and the company code information on the new host matches what is defined on the old host. Follow these steps:


Step 1 To reinitialize the system, enter the following command:
/opt/ConfigEngine/CSCOcnsie/reinitialize

Step 2 To run data migration again, enter the following command:
/opt/ConfigEngine/CSCOcnsie/bin/datamigrate

Step 3 To run the Setup program again, enter the following command:
/opt/ConfigEngine/CSCOcnsie/setup


Note Make sure that you run the Setup program in bash shell. If the shell is not in bash, press ctrl-c to exit. Configure your shell in bash, and then rerun the Setup program.


Step 4 When entering the Setup parameters, make sure that the country code and the company code information for the new host matches what is defined on the old host.


Note The country code and the company code in the OpenLDAP schema are case sensitive.


For detailed information about the parameters in the Setup program, see the Cisco Configuration Engine Administration Guide.

Example

Choose operational mode of system. 0=internal directory mode, 1=external directory mode. 
[0] 
Enter country code: us 
Enter company code: cisco


Configuration Update Stuck in Queue After Data Backup and Restore

Problem   The configuration update is stuck in the queue after data backup and restore.

Possible Cause    This problem could occur if you did not enter the correct country code and company code information during the Setup program.

Solution   When you back up data and restore it, the OpenLDAP schema is transferred to a new host. To reuse the existing OpenLDAP schema for the new host, make sure that the country code and the company code information on the new host matches what is defined on the old host. Follow these steps:


Step 1 To reinitialize the system, enter the following command:
/opt/ConfigEngine/CSCOcnsie/reinitialize

Step 2 To run data restore again, enter the following command:
/opt/ConfigEngine/CSCOcnsie/bin/datarestore

Step 3 To run the Setup program again, enter the following command:
/opt/ConfigEngine/CSCOcnsie/setup


Note Make sure that you run the Setup program in bash shell. If the shell is not in bash, press ctrl-c to exit. Configure your shell in bash, and then rerun the Setup program.


Step 4 When entering the Setup parameters, make sure that the country code and the company code information for the new host matches what is defined on the old host.


Note The country code and the company code in the OpenLDAP schema are case sensitive.


For detailed information about the parameters in the Setup program, see the Cisco Configuration Engine Administration Guide.

Example

Choose operational mode of system. 0=internal directory mode, 1=external directory mode. 
[0] 
Enter country code: us 
Enter company code: cisco


Troubleshooting an Image Update

To troubleshoot an image update, see the following sections:

Information About Log Files

Cannot Activate Image

Activation Failed Due to Device Error

Error Message: Image Update Stopped

Information About Log Files

The Log4j file is used as the logging facility for the Cisco Configuration Engine server and the image server. The property file is located at: <INSTALL_DIR>/CSCOcfgs/conf/logs.properties. You can control the logging behavior by editing the logs.properties configuration file. This file is located at:
cd $CISCO_CE_INSTALL_ROOT/CSCOcnsie/conf. The default level for logging is set to Debug. Accepted values are Debug, Info, Warn, Error, and Fatal.

/var/log/CNSCE/imgsrv/imgsrv.log—Contains log messages from the server concerning the actions that you have performed that pertain to images, such as creating, updating, or deleting images. This log file also contains detailed message exchanges between the image server and devices during image distribution and activation.

/var/log/httpd/*.log, /var/log/CNSCE/tomcat/*.out, *.txt, *.log—Contains log messages regarding the status of the web server.

/var/log/CNSCE/websvc—Contains log messages regarding web service APIs.

/var/log/CNSCE/imgw/*—Contains log messages regarding the IMGW.

Cannot Activate Image

Problem   You are trying to activate an image but cannot activate it.

Possible Cause    This problem could occur if the activation template does not contain the correct configuration.

Solution   To resolve this problem, make sure that the configuration is correct. Then try again to activate the image.

Activation Failed Due to Device Error

Problem   Activation failed due to a device error. The device does not load the specified image.

Solution   To resolve this problem, make sure that the image information matches the image that you have downloaded.

Error Message: Image Update Stopped

Problem   Image update stops and you receive the following error message:

2004-01-13 19:04:52,677 [c7200-1] DEBUG message.EvtMsgSender - Sent msg to 
Identifier=1074049490996 of Type=MSG_IMAGE_UPDATE_STOPPED. 

Possible Cause    This problem could occur for one of the following reasons:

The file system could not be found.

The space was insufficient for distributing the specified image.

The server was unable to access the image file from a specified location.

Solution   To resolve this problem, follow these steps:


Step 1 If the job stopped because the file system was not found, check the imagsvr log file to verify whether the file system name in the destination field is correct. This log file is located at: /var/log/CNSCE/imgsvr.log.

Example:

2005-11-03 15:31:39,974 [TP-Processor9] DEBUG action.UpdateImageProcess - RefCISDevice: ImageID=[d2NonAgent],CN=[d2NonAgent],Inventory Device Ref=[d2NonAgent],Password=[null],Activations=[{}],ActivationTemplate=[DemoRouter.cfgtpl],Img_And_Dist=[{image1=HashCode=[558448476],Name=[DIST1131057049654],ImgRef=[image1],
Destination=[Colorado],Location=[http://cluster-rm/cns/LoadPage?HtmlFilename=home.html],EraseFileSys=[true],OverWrite=[true]., image2=HashCode=[457703260],Name=[DIST1131057049658],ImgRef=[image2],Destination=[Denver],Location=[http://cluster-rm/cns/LoadPage?HtmlFilename=home.html],EraseFileSys=[true],OverWrite=[false].}].

Step 2 If the job stopped because the space was insufficient for distributing the specified image, check the imagsvr log file to verify whether the file system has sufficient space for downloading the specified image. This log file is located at: /var/log/CNSCE/imgsvr.log.

Example:

2004-01-13 19:18:21,563 [c7200-1] DEBUG evaluation.DeviceEvaluator -DeviceEvaluation=[Reachable=[true], Distribution Eval List Size=[1]: List=[Required=[true],Reason=[Compare ImageFile in RunningImageInfo, Check FreeSpace and Running Image MD5.],ErrorInfo=[null],SufficientSpace=[false].,], Activation Eval List Size=[1]: List=[Required=[true],Reason=[Compare ImageFile in RunningImageInfo, Check FreeSpace and Running Image MD5.],ErrorInfo=[null],SufficientSpace=[false].,].].

2004-01-13 19:18:21,563 [c7200-1] DEBUG distribution.DevicePerformer - Distribution is required, but Space is not sufficient.

Step 3 If the job stopped because the server was unable to access the image from the specified location, make sure that the you can access the URL in the image location field.

Example:

2005-11-04 15:52:52,690 [Thread-377] DEBUG evaluation.DeviceEvaluator - Retrieving Inventory from Device=[ImageID=[d1],CN=[d1],Inventory Device Ref=[d1],Password=[null],Activations=[{}],ActivationTemplate=[DemoRouter.cfgtpl],Img_And_Dist=[{img4=HashCode=[1543307114],Name=[DIST1131144742987],ImgRef=[img4],Destination=[California],Location=[http://hostname/cns/LoadPage?HtmlFilename=home.html],EraseFileSys=[true],OverWrite=[true].}].]...


Troubleshooting IMGW

To troubleshoot IMGW, see the following section:

Obtaining Detailed Debugging Information

Obtaining Detailed Debugging Information

Problem   How do I obtain debugging information?

Solution   To obtain detailed debugging information, you must configure the log files for IMGW. Follow these steps:


Step 1 Configure the logging level for the IMGW daemon. During the Cisco Configuration Engine Setup program, configure the IMGW parameters to one of the listed values. Logging levels are Verbose, Error, and Silent.

Step 2 To configure the logging level for the IMGW servlet, edit the following two lines in the
$CISCO_CE_INSTALL_ROOT/CSCOimgw/conf/imgw.properties file:

IMGW_LOGFILE /var/log/CNSCE/IMGW/imgwservlet.log
(/* location of IMGW servlet log file */)

IMGW_LOGGING_LEVEL DEBUG
(/* debug level - ERROR or DEBUG */)


Troubleshooting the Router

To troubleshoot the router, see the following section:

Enabling Debugging on the Router

Enabling Debugging on the Router

Problem   How do I enable debugging on the router?

Solution   To enable debugging on the router, follow these steps:


Step 1 To enable debugging on the router, use the debug cns image all command.

Step 2 If you are not on the console, enter the term mon command.

Step 3 After the job completes, verify the file on the router by entering the dir command. The image file should display.


General Troubleshooting

For general troubleshooting tips, see the following sections:

Error Message: Failed to Create the Device on Remote Database

cns-listen Command Failed to Execute

HTTPD Is Down When Crypto Is Enabled

Web Service Deployment Error When Crypto Is Enabled

Backup and Restore Not Working Properly

Device Status Changed from Green to Red After Setup

Cannot Back Up Jobs

Error Message: Failed to Create the Device on Remote Database

Problem   You get the following error message:

Failed to create the Device. Could not create Object: DN=  
[cn=jctest, ou=CISDevices,ou=CISObjects,ou=configengine,o=cisco
[LDAP: error code 50 - no write access to parent]

Solution   To resolve this problem, follow these steps:


Step 1 On the remote directory server machine, stop the OpenLDAP server by entering the following commands:

In Solaris, enter: /etc/init.d/NetAppOpenLDAP stop

In Linux, enter: /etc/rc.d/init.d//NetAppOpenLDAP stop

Step 2 Open the $CISCO_CE_INSTALL_ROOT/openldap/etc/openldap/slapd.conf file. Then add the following:
# open write permission to support external directory
access to *
        by * write
        by * read
         by anonymous auth

Step 3 To start the OpenLDAP server, enter the following commands:

In Solaris, enter: /etc/init.d/NetAppOpenLDAP start

In Linux, enter: /etc/rc.d/init.d//NetAppOpenLDAP start


cns-listen Command Failed to Execute

Problem   The cns-listen command failed to execute.

Possible Cause    This problem could occur if the values you entered for the CNS Event Bus Service and the CNS Event Bus Daemon parameters do not match the values you used in the $cns-listen command.

Solution   To resolve this problem, make sure that you use the same value in the command that you entered for the parameters. For example:

Enter CNS Event Bus Service Parameter: [7500] 7800
Enter CNS Event Bus Daemon Parameter: [7500] 7900
cns-listen command:
$cd $CISCO_CE_HOME/tools
$cns-listen -service 7800 -daemon 7900

HTTPD Is Down When Crypto Is Enabled

Problem   The HTTPD service is down when crypto is enabled.

Possible Cause    This problem could occur if, during the Cisco Configuration Engine Setup program, you used invalid values for the remote key file and remote certificate file.

Solution   To resolve the problem, make sure that you use valid values for the remote key file and remote certificate file. For example:

Enable cryptographic (crypto) operation between Event Gateway(s)/Config
server and device(s) (y/n)? [n] y
Enter absolute pathname of remote key file: /opt/server.key
Enter absolute pathname of remote certificate file: /opt/server.crt

Web Service Deployment Error When Crypto Is Enabled

Problem   You get the following web service deployment error messages:

Following command failed: see /var/log/CNSCE/appliance-setup.log for 
details/opt/CSCOcnsie/bin/deploy.config.websvc [-wsdl]

Deploying image web services ...

Following command failed: see /var/log/CNSCE/appliance-setup.log for 
details/opt/CSCOcnsie/bin/deploy.image.websvc [-wsdl]

Solution   To resolve this problem, follow these steps:


Step 1 Make sure that the Tomcat and HTTPD status is up.

Step 2 Enter the following command:
wget https://$HostName/cns/services/CEAdminService

If the command fails to execute, the domain name might not be set up correctly.

Step 3 Verify the host network settings at:
/etc/hosts, /etc/resolv.conf


Backup and Restore Not Working Properly

Problem   Backup and restore is not working properly.

Possible Cause    This problem could occur for the following reasons:

The time base for the host system is not set to the UTC time zone.

The time has changed.

The cron job has not started.

Solution   To resolve this problem, follow these steps:


Step 1 Connect to the console if you cannot connect using SSH.

Step 2 Log in to the host system as root.

Step 3 To determine whether the time is correct, enter the following command:
# date

Step 4 To determine the state of the cron job, enter the following command:
# /etc/rc.d/init.d/crond restart

Example:

# /etc/rc.d/init.d/crond restart
Stopping cron daemon:                                      [  OK  ]
Starting cron daemon:                                      [  OK  ]
#

Device Status Changed from Green to Red After Setup

Problem   After Cisco Configuration Engine setup, the device status changes from green to red in a few minutes. This problem occurs on the Solaris 10 platform, right after restarting the Cisco Configuration Engine services.

Possible Cause    This problem could occur if the TibGate processes shut down a few minutes after starting.

Solution   To resolve this problem, follow these steps:


Step 1 To check whether the TibGate processes are running, enter one of the following commands:
/etc/init.d/EvtGateway
/etc/init.d/EvtGatewayCrypto

Step 2 If the TibGate processes are not running, ask your System Administrator to disable NISPlus service.

Step 3 If the device status is still red, see the "CNS-Enabled Device Unable to Connect with Cisco Configuration Engine" section for a possible solution.


Cannot Back Up Jobs

Problem   Cannot back up jobs.

Possible Cause    The crontab command is used to schedule backup jobs. This command requires space in the /var partition to execute. If the /var partition is full, the crontab command fails to execute, which causes backup job failure.

Solution   To resolve this problem, clean up the /var partition on the system (move some files to the /home/ directory). Then resubmit the backup job from the Cisco Configuration Engine user interface.