Cisco DCNM Troubleshooting Guide
Troubleshooting Issues with Cisco DCNM Installation
Downloads: This chapterpdf (PDF - 570.0KB) The complete bookPDF (PDF - 1.02MB) | Feedback

Troubleshooting Issues with Cisco DCNM Installation

Troubleshooting Issues with Cisco DCNM Installation

This chapter describes troubleshooting issues with solutions while installing Cisco Data Center Network Manager.

Launching Cisco DCNM Installer in Debug Mode

To launch the Cisco DCNM installer in debug mode:

  • On Microsoft Windows, press the CTRL key when you extract the installer.
  • On Linux, set the variable in the terminal to export LAX_DEBUG=true. Execute the installer after the setting the variable.

Note


When you install Cisco DCNM in debug mode, dcnm_installer.log is not created and you will need to copy and paste the console logs into a log file for future reference.

Troubleshooting Installation Issues

The following table displays a list of issues and recommended solutions that can assist you with troubleshooting a installation issue. If an attempt to install Cisco DCNM fails, you should implement these solutions.

Table 1 Installer and Installation Issues
Issue Recommended Solution
Installer launch issues. Ensure Cisco DCNM installation requirements are met before proceeding with the installation.
"Choose another location to extract the files to…" error occurs. The installer could be corrupted. Try downloading the installer from the Cisco website and then restart the installation.
Windows 216 error and cannot execute binary file error in Linux occurs. You may be installing a 64-bit installer on 32-bit platform.
DCNM-SAN Server failed to start after installation.
  • Browse to <DNCM_HOME>\fm\bin and start #service.bat FMServer server manually.

or

  • Browse to fmserver.conf file:
    • On Microsoft Windows at <DCNM_HOME>\dcm\fm\conf\fmserver.conf
    • On Linux at <DCNM_HOME>/dcm/fm/conf/fmserver.conf
    Change: wrapper.java.additional.12=-XX:MaxPermSize=512m to wrapper.java.additional.12=-XX:MaxPermSize=256m.
  • Restart the server.
Unable to launch the DCNM-SAN Client after installation.
  • Possible cause for this issue is that the recommended memory settings are not met for Cisco DCNM installation.
  • Try installing the Cisco DCNM in the recommended setup and launch the Cisco DCNM-SAN Client.
JBOSS Server startup failure. After installing Cisco DCNM, the JBOSS server startup fails with an exception:
java.lang.RuntimeException: Exception 
creating identity: <HOST_NAME> at org.jboss.remoting.
ident.Identity.get(Identity.java:211)org.jboss.
deployment.DeploymentException: - nested throwable: 
(java.lang.reflect.InvocationTargetException)
  • The \etc\hosts file does not contain the correct entries.
  • Edit the etc/hosts file to map the loopback address with the hostname.
  • The etc/hosts file should contain the hostname (in addition to whatever is already present in the file).

For example, host-ip <space> host-name.

JBOSS Server startup failure in Secondary Node - After installing Cisco DCNM in the secondary node, DCNM failed to start.
Note    This issue is seen in Release 5.2(2a) and resolved in Release 5.2(2c). Download the latest installers from the Cisco website and upgrade to the latest release.
  • Add the property as FM_SERVER_INSTALLED=FALSE in the "installer.properties" file that was copied from master node.
  • Trigger the silent mode installation using the command in secondary node.

PostgreSQL installation issues— An "init db" error occurs due to file system.

Cisco DCNM Installation failed during PostgreSQL installation and an "init db"error appears, and the installation is aborted.

  • Check that the file system is present. It might be FAT/FAT32.
  • PostgreSQL installation requires an NTFS file system, and will fail on a FAT/FAT32 file system. The NTFS file system is more reliable than FAT/FAT32 file system. Check your file system before starting the DCNM installation.

PostgreSQL installation issues— An "init db" permission-denied error appears.

Cisco DCNM installation failed during PostgreSQL installation and a permission-denied error appears.

  • Check that the local user has administrator privileges. Only administrators can perform this task.
  • Try installing PostgreSQL manually.

PostgreSQL installation issues—Cisco DCNM installation failed during the PostgreSQL installation on Linux.

The following error is displayed in the database options screen:

Shared memory settings are low for 
PostgreSQL installation.
  • PostgreSQL installation requires a minimum of 256 MB of shared memory. You can view them at /proc/sys/kernel/shmmax.
  • The shared memory settings can be increased using the sysctl -w kernel.shmmax=268435456 command.

Oracle or Oracle RAC issues—"No appropriate service handler found" exception occurs.

Cisco DCNM installation failed to add a node in SAN Federation using Oracle as the Database.

This exception occurs in the SAN Server log:
"java.sql.SQLException: Listener refused the
 connection with the following error:ORA-12519, 
TNS:no appropriate service handler found"
  • This exception occurs when the "concurrent db processes" is limited by the database. Ensure that the number of concurrent connections is set according to the requirement.
  • Cisco DCNM recommends a minimum of 150 sessions.
  • The connections can be increased using the following command in the sqlplus.exe prompt:

alter system set processes=150 scope=spfile

Oracle or Oracle RAC issues—"Table or view does not exist" exception occurs. The database user might not have the privileges to create schema or tables. Ensure that the Oracle or Oracle RAC database user has the required privileges and continue with the installation of Cisco DCNM.
Oracle or Oracle RAC issues—"Network Adapter could not establish connection" exception occurs.
  • This error is displayed when the entries for Oracle RAC setup are not entered in these hosts file.
    • Linux– /etc/hosts
    • Microsoft Windows – C:\WINDOWS\system32\drivers\etc\hosts
  • Ensure that you enter the Oracle RAC VIP (virtual IP address ) and host-ip in the hosts file.
Unauthorized port scanner–After installing Cisco DCNM in an Linux environment, pnsscan multi-threaded port scanner attempts to scan the entire network. The operating system probably had a virus attack.
Login issue in DCNM-LAN–After installing Cisco DCNM, and when the DCNM-LAN and DCNM-SAN are launched, the following error is displayed:

"Authentication Failure: Invalid username/password"

Reset the login password using the pwreset tool under <DCNM_HOME>\dcm\dcnm\bin.
Note    For Linux, configure x-windows to display the pwreset GUI. For more information, see http://www.hungry.com/~jamie/xexport.html.
Port conflicts while installing Cisco DCNM.
  • Ensure that you stop the firewall, security agent, antivirus, and so on before continuing with the installation. This blocks the port for security constraints.
  • To stop the Windows firewall, use the following command:
    • OFF: netsh adfirewall set allprofiles state off
    • ON: netsh adfirewall set allprofiles state on
  • We recommend that you install only Cisco DCNM Server in the server for better performance and to avoid port conflict issues.
  • When you change the naming service port during installation, enter the port in DCNM-SAN client as follows: <HOST_IP_ADDRESS> : < NAMING _SERVICE _ PORT >
  • When you change the naming service port during installation, click More (<< More) to enter the port in the DCNM-LAN client.
Chose Install Directory field and the DB User credentials fields grayed out during recent install with PostgreSQL.
  • Previous DCNM and PostgreSQL uninstallation was not complete.
  • Delete the postgreSQL service.
  • Install Cisco DCNM after deleting the following files:
    • Windows–C:\Program Files\Zero G Registry\.com.zerog.registry.xml <USER_HOME>\.cisco_mds9000
    • Linux–/var/.com.zerog.registry.xml /root/.cisco_mds9000
Cisco DCNM authentication fails in cluster or federation mode. After Cisco DCNM is installed in cluster or federation mode, you can log in to the Cisco DCNM WEB GUI and DCNM SAN Client but cannot log in to DCNM LAN Client. The following error message is displayed:

"Authentication Fails"

  • The multicast addresses used during the installation of the cluster nodes are not correct.
  • Verify that the multicast addresses used are correct by following this procedure:
    1. Navigate to the following location: On Windows– <DCNM_HOME>\jboss-4.2.2.GA\server\dcnm\lib On Linux– <DCNM_HOME>/jboss-4.2.2.GA/server/dcnm/lib
    2. Execute the following command on all the four DCNM cluster nodes: java -cp jgroups.jar org.jgroups.tests.McastReceiverTest -mcast_addr multicast_addrress1 -port port1 -bind_addr dcnm-server-ip-address
      Note    Multicast_address1, and port1 are paired values and will be used on all of the machines together.
      Note    You will not see the output after executing the command if the multicast addresses are wrong.
    3. Resolve the multicast addresses by following this procedure: Go to < DCNM_HOME > \ dcm \ jboss-4.2.2.GA \ server \ dcnm\ deploy \ jboss-web-cluster.sar \META-INF \ jboss-service.xml, and change the multicast IP address:
      <config>
      <UDP mcast_addr="${jboss.partition.udpGroup:239.
      28.71.119}" 
      mcast_port="${jboss.webpartition.mcast_port:
      45503}"
      tos="8"
      ucast_recv_buf_size="20000000"
      ucast_send_buf_size="640000"
      
      Go to < DCNM_HOME > \ dcm \ jboss - 4.2.2. GA \ server\ dcnm \ deploy \ cluster-service.xml , and change the multicast IP address:
      <Config>
      <UDP mcast_addr="${jboss.partition.udpGroup
      :239.28.71.118}" 
      mcast_port="${jboss.hapartition.mcast_port:45502
      }"
      tos="8"
      ucast_recv_buf_size="20000000"
      ucast_send_buf_size="640000"
      mcast_recv_buf_size="25000000"
      
      Go to < DCNM_HOME > \ dcm \ jboss-4.2.2.GA \ server\ dcnm \ deploy \ dcnm-cache-service.xml, and change the multicast addresses:
      <attribute name="ClusterConfig">
      <config>
      <UDP mcast_addr="${jboss.partition.udpGroup:
      239.28.71.120}"
      
Federation nodes are in an inactive state in the Cisco DCNM web GUI under Admin > Federation. Ensure that the master and slave nodes are running at the same time and restart the Cisco DCNM Servers.

Launching the web client or other clients (DCNM-LAN and DCNM-SAN java web start clients fails).

When Cisco DCNM is installed on a new Windows 2008 server. By default, the firewall settings are enabled and you will not be able to launch the web client from any other server machine (other then the machine where Cisco DCNM servers are installed).

Turn off the default firewall or create appropriate rules to allow the ports specified during Cisco DCNM installation.

Oracle backup script fails in Oracle 11g and the following error message is displayed:

"ORACLE error 12546 encountered. TNS permission denied. EXP-00000. Export terminated unsuccessfully."

When this error message is displayed, the Cisco DCNM Log packager tool stops backing up the Oracle database. Try the following:
  • Ensure that you have not logged in to the Linux system as a "root" user.
  • Log in as a DB user used during Oracle 11G installation and execute the backup script.

IPv6 on Microsoft Windows 2008

By default, IPv6 is enabled on Windows 2008 systems. Ensure that you disable IPv6 before you start installing Cisco DCNM. To disable IPv6, see http://support.microsoft.com/kb/929852.

Installing Cisco DCNM in Unsupported Platforms

To install Cisco DCNM in unsupported platforms, enter the following command in the command line interface (CLI):
<DCNM_INSTALLER_NAME> -Dforce=true

Note


Because unsupported platforms are not tested, we recommend that you install Cisco DCNM on supported platforms.

Postgres Database Installation Failure Issues

The following table includes a number of Postgres database installation issues and solutions.

Table 2 Postgres Database Installation Issues
Issue Recommended Solution
Postgres database installation fails. Solution A – The username specified to run the Postgres service already exists on the server. Specify a different username or remove the existing username from the server.

Solution B – Antivirus software or intrusion detection software, such as Cisco Security Agent, blocked the installation. Temporarily disable any antivirus software and intrusion detection software, and then reinstall Cisco DCNM.

The PostgreSQL installer shows the following message:

Failed to run initdb: 128

Solution A – (Microsoft Windows only) Remote Desktop Connection client not running in console mode. If you are installing Cisco DCNM on a supported Microsoft Windows operating system and using Remote Desktop Connection (RDC) to access the Cisco DCNM server system, start RDC from a command prompt and use the /console option, as follows:

C:\>mstsc /console /v:server

server is the DNS name or IP address of the Cisco DCNM server system.

If the /console option is not supported in the Microsoft Windows version that you are running, use the /admin option, as follows:

C:\>mstsc /admin /v:server

Solution B – (Macintosh only) If you are using Remote Desktop Connection on a Macintosh computer to access the Cisco DCNM server system, follow this procedure:

  1. Enter the IP address of the Windows server where Cisco DCNM is installed.
  2. Hold down the Command key and click Connect. This action forces RDC to use console mode.
  3. Release the Command key when the authentication window appears.

The PostgreSQL installer shows the following message:

Failed to create process for initdb. The service cannot be started, either because it is disabled or because it has no enabled devices associated with it.

(Microsoft Windows only) The Secondary Logon service is not running. To verify if the Secondary Logon service is running, follow this procedure:
  1. On the Cisco DCNM Server system, open the control panel, locate Administrative Tools and click Services.
  2. In the list of services, find the Secondary Logon service.
  3. If the status of the Secondary Logon service is not Started, right-click the service and choose Start.
  4. Close the Services window.
  5. Restart the Cisco DCNM server installation.

Troubleshooting VSB Installation Issues

While deploying Cisco DCNM on a Nexus 1010, a VSB not present error is displayed when the following command is entered:

N1010(config-vsb-config)#show virtual-service-blade summary

To resolve this issue, consider the following:

  • The Cisco DCNM installer (.iso) possibly was copied to the Nexus 1010 using the Windows environment.
  • We recommend that you always copy the installer to the Nexus 1010 using this command: copy scp: bootflash:

Path to the Perl Binary Directory Not Found

Table 3 Path to the Perl Binary Directory Not Found
Issue Recommended Solution
An error message states that the path to the bin directory for Perl is not in the system PATH environment variable. Perl is not installed on the server system or the server system PATH environment variable does not include the path to the directory that contains the Perl executable.
  • Verify that a supported version of ActivePerl is installed on the server system. If not, install a supported version of ActivePerl. For more on the supported version, see the Cisco DCNM Installation and Licensing Guide.
  • Ensure that the system PATH environment variable includes the path to the directory that contains the Perl executable. On Microsoft Windows, the default path to the ActivePerl bin directory is C:\Perl\bin.
  • Restart the DCNM server installation.

Launching Cisco DCNM-SAN and DCNM-LAN Client

After Cisco DCNM is installed and Cisco DCNM-LAN and Cisco DCNM-SAN servers are started, you can launch Cisco DCNM-LAN and DCNM-SAN clients.

To launch the Cisco DCNM-LAN and DCNM-SAN client on any client machine:


    Step 1   When you have installed Cisco DCNM with the HTTP option, use the following link:

    http://<DCNM_HOST_IP_ADDRESS>:<LAN_WEB_PORT>/dcnm-client

    Step 2   When you have installed Cisco DCNM with the HTTPS option, use the following link:

    https://<DCNM_HOST_IP_ADDRESS>:<LAN_WEB_PORT>/dcnm-client

    Step 3   Click the download button to download the jnlp file to launch the DCNM- LAN client in your system.
    Note   

    When the SAN web port is changed (other than the default 80), use the following:

    http://<DCNM_HOST_IP_ADDRESS>:<SAN_WEB_PORT>

    https://<DCNM_HOST_IP_ADDRESS>:<SAN_WEB_PORT>


    Restoring Script Failures

    Cisco DCNM database restore using the DCNM restore script fails with the following error:

    'dbuser' cannot be dropped as it is connected

    Stop the DCNM servers and run the restore script again.

    Prerequisites for Upgrading Cisco DCNM

    Ensure that Cisco DCNM-LAN and Cisco DCNM-SAN servers are stopped before upgrading.

    To stop the Cisco DCNM-LAN server, follow this procedure:


    Note


    The following shortcuts are available only in Cisco DCNM releases prior to Release 6.x.
    • On Microsoft Windows, select Start > Programs > Cisco DCNM Server > Stop DCNM LAN Server or stop the Cisco DCNM-LAN server from the Windows services (services.msc).
    • On Linux, execute the following command from the directory where the link is created (default path is /root): ./Stop_DCNM_LAN_Server

    To stop the Cisco DCNM-SAN server, follow this procedure:

    • On Microsoft Windows, stop the DCNM-SAN server from the Windows services (services.msc).
    • On Linux, browse to <DCNM_HOME>/dcm/fm/bin/, and execute ./FMServer.sh stop.

    Uninstalling Cisco DCNM

    The following table includes a number of uninstallation issues and solutions.

    Table 4 Uninstalling Cisco DCNM
    Issue Recommended Solution
    Applications are not installed using the InstallAnywhere tool.
    1. Delete the XML file from the following location:
      • On Microsoft Windows – C:\Program Files\Zero G Registry\.com.zerog.registry.xml
      • On Linux – /var/.com.zerog.registry.xml
    2. (Only on Windows) Delete the Cisco DCNM services sc delete FMServer and sc delete DCNMServer if they are still running.
    3. Delete the .cisco_mds9000 folder.
      • On Microsoft Windows – <USER_HOME>\.cisco_mds9000
      • Linux – /root/.com.zerog.registry.xml
    Applications are installed using InstallAnywhere tool.
    1. Back up the .com.zerog.registry.xml file:
      • On Microsoft Windows – C:\Program Files\Zero G
      • On Linux – /var/.com.zerog.registry.xml
    2. Open the XML file in a text editor.
    3. Under the <products> element, delete the <product> element and all its descendant elements:
      <product name="Cisco DCNM" id="9e458447-
      1ee6-11b2-85ed-d4ed684e9c05" version="x.x.x.x" 
      copyright="2007". . .
    4. Under the <components> element, delete every instance of the <component> element:
      <component id="9e458484-1ee6-11b2-860c-
      d4ed684e9c05" version="x.x.x.x" name=
      "InstallAnywhere VM Component" location="C:\Program
       Files\Cisco Systems\Cisco DCNM\jre" vendor="Cisco
       Systems Inc."/>
    5. Save and close the file.

    Troubleshooting Event Browser Issues

    Table 5 Troubleshooting Event Browser Issues
    Issue Recommended Solution
    Issues starting the Perl processor.

    Cisco DCNM is packaged with Strawberry Perl (open source software) since DCNM server is dependent on Perl. Strawberry Perl is installed under the following location:

    • On Microsoft Windows-<DCNM_HOME>\dcm\dcnm\strawberry
    • On Linux-<DCNM_HOME>/dcm/dcnm/strawberry
    Check the following:
    • Perl path is mapped properly under the system/user PATH variables.
    • Perl service is started and running. See the syslog_filter log file for below information:
      [PerlProcessor] Using strawberry perl
      	 [PerlProcessor] Perl Processor started
      	 [PerlProcessor] Using port 5445 trying to 
      Connect with DCNM Server
      	 [PerlProcessor] Server timestaamp 1349935192236
      	 [PerlProcessor] Connecting with DCNM Server - 
      <dcnm server ip>:5445
      
    • Perl version using the perl -v command.
    • Ensure the local or current user has administrative privileges. If the current user does not have administrative privileges before starting the server, perform the following steps in the Windows services panel:
      1. Right-click on the Cisco DCNM-LAN Server service and select Properties .
      2. Click Logon.
      3. Select This account button and provide the user details that have administrative privileges.

    On Microsoft Windows - Under Administrative Tool > Advanced Tab > System/User Setting, modify the PATH variable by including <DCNM_HOME>\dcm\dcnm\strawberry entry in the PATH variable ( Replace <DCNM_HOME> with the directory where Cisco DCNM is installed).

    On Linux - Modify the Perl path to PATH=<DCNM_HOME>/dcm/dcnm/strawberry:$PATH export PATH ( Replace <DCNM_HOME> with the directory where Cisco DCNM is installed).

    Events disappear from the event browser. Ensure the switch, Cisco DCNM Server, and the Cisco DCNM Client clocks are synchronized. To verify if the clocks are synchronized, in the Cisco DCNM Event Browser, check the Local Time and Switch Time.

    To change the device clock or Cisco DCNM server timestamp, use the clock set and clock timezone command.

    To view event messages older then 24 hours. By default, the Cisco DCNM-LAN client fetches events from the server messages that are not older then 24 hours. To view events that are older than 24 hours, under the Event Filter option, enter the start date, end date, and the perceived severity to view events.
    Cisco DCNM-LAN client only displays Cisco DCNM supported events.

    Cisco DCNM-LAN client event browser only displays events that are supported by DCNM and events with Emergency, Alert, and Critical severity.

    Event Database Administration Settings.

    By default, the event database purge is executed automatically at 11.30 PM and retains the last 1,00,000 entries and deletes the older events from the database.

    Click DCNM Server Administration > Auto Synchronization. The default setting in the Events Database Administration page is displayed.

    Event Browser screen only displays 2000 events.

    The Event Browser screen only displays 2000 events with the latest events at the top of the table. After reaching 2000 rows, the older events are removed from the Event Browser screen and latest events are added at the top.

    To view more than 2000 events or older events that are removed from the Event Browser, use the Event Filter option by providing the start time, end time, and the severity.