User Guide for Device Fault Manager 3.2 (With LMS 3.2)
Appendix G: Frequently Asked Questions in DFM

Table Of Contents

Frequently Asked Questions in DFM


Frequently Asked Questions in DFM


The following section lists the frequently asked questions about DFM.

What are the OIDs polled by DFM for card status?

Where can I find the log and rps files of Incharge/Smarts?

Why are devices with SysObjID 1.3.6.1.4.1.311.1.1.3.1.2 or .1.3.6.1.4.1.311.1.1.3.1.3 not managed by DFM?

Why does DFM display false Card Down events?

How can I collect Mibwalk for a device?

Can I have HPOV/Netview installed in one drive, for example C: and DFM HPOV/Netview Adapters in another drive, D:?

In the Search Results, the selected devices are not displayed as selected. They are displayed as selected only in All Devices group. Why?

How can I avoid DFM generating Unreponsive alerts when the devices are in Natted Enviroment?

How can I get rid of alerts for a device that has been deleted?

Why HighUtilization on interface gets generated by DFM?

Why does a device go into Unsupported state although it is Supported ?

What is the difference between Snmp Raw Trap Forwarding and Processed Snmp Trap alert/event Trap Forwarding? Does DFM support both of these methods?

How does DFM detect Trunk and Access ports?

Why devices in DCR do not get added to DFM in ACS mode?

Is the AttachToNCM script supported in DFM2.x/DFM3.x versions?

What is the meaning of different discovery percentages?

How can I troubleshoot device discovery stuck at 10%?

How can I troubleshoot device discovery stuck at 40%?

How can I troubleshoot device discovery stuck at 90%?

How can I stop nGenius RealTimeMonitoring in Solaris ?

How can I receive Syslog messages from a dfm server?

How can I perform rediscovery of devices in DFM through CLI ?

How can I manage or unmanage ports and interfaces from CLI?

How can I import devices into DCR through CLI?

How can I enable CAM logs for debugging ACS Configuration?

How can I create a link to the Java Plug-in in Netscape7.x and Mozilla 1.7.x?

How can I avoid getting an alert when the device Cisco Intelligent Gigabit Switch Module for IBM Bladecenter does not contain a fan?

How does DFM react to a Cisco ISR with Inline POE switch module when there is no -48V power supply installed?

What happens when VG200 Routers have an ISDN PRI carrying voice traffic?

I am unable to sort Cleared Alerts in DFM. Why?

Q. What are the OIDs polled by DFM for card status?

A. The following OIDs are polled to monitor the cards:

In EntityFRU:

1.3.6.1.4.1.9.9.117.1.2.1.1.1

1.3.6.1.4.1.9.9.117.1.2.1.1.2

In OLD-CISCO-CHASSIS-MIB: 1.3.6.1.4.1.9.3.6.11.1.9.0

In CISCO-STACK-MIB: 1.3.6.1.4.1.9.5.1.3.1.1.10

Q. Where can I find the log and rps files of Incharge/Smarts?

A. The log and rps files of Incharge/Smarts can be found at the following locations:

log files: NMSROOT/objects/smarts/local/logs

rps files: NMSROOT/objects/smarts/local/repos/icf

broker.rps files: \CSCOpx\objects\smarts\local\repos\broker

You need to send all these files for debugging if there is a problem with Incharge.

Q. Why are devices with SysObjID 1.3.6.1.4.1.311.1.1.3.1.2 or .1.3.6.1.4.1.311.1.1.3.1.3 not managed by DFM?

A. Devices with SysObjID 1.3.6.1.4.1.311.1.1.3.1.2 or .1.3.6.1.4.1.311.1.1.3.1.3 on Windows 2000 are not discovered by DFM. This could be because CDP MIB does not respond on the devices. For a device to be managed ny DFM, it should respond to CDP MIB.

Q. Why does DFM display false Card Down events?

A. For all ARTG routers that can hold VWIC2-xMFT cards, you may see Card Down events. There is a bug (CSCsj58422) on the agent side. You may need to upgrade to the latest IOS Version to resolve this problem. Please refer to the case ID 607434565 and 607553227 for more details.

Q. How can I collect Mibwalk for a device?

A. To collect the Mibwalk for a device, do the following:


Step 1 Go to NMSROOT/objects/smarts/bin

Step 2 Enter the following command for:

Snmp v1 and snmp v2 devices:

For Solaris: ./sm_snmpwalk --community= deviceIp

For eg: ./sm_snmpwalk --community=cisco 4.1.1.1

For Windows: sm_snmpwalk --community= deviceIp

For eg: sm_snmpwalk --community=cisco 4.1.1.1

Snmp v3 devices:

For Solaris: ./sm_snmpwalk --snmp=3 --user=desuser --auth=MD5 --authPass=changeme --priv=DES --privPass=despass --authengine=8000000903000019563F8338 bq-gwhsrp.lss.emc.com

For Windows: sm_snmpwalk --snmp=3 --user=desuser --auth=MD5 --authPass=changeme --priv=DES --privPass=despass --authengine=8000000903000019563F8338 bq-gwhsrp.lss.emc.com

The above command will generate three files, xxxxx.walk, xxxxx.mimic, and xxxxx.snap files [where xxxxx is the device IP] in the same location, that is inNMSROOT/objects/smarts/bin. You can zip the 3 generated files.

Q. Can I have HPOV/Netview installed in one drive, for example C: and DFM HPOV/Netview Adapters in another drive, D:?

A. No. It is recommended to install both HPOV/Netview and DFM HPOV/Netview Adapters in the same drive. For more information, see: http://www.cisco.com/en/US/docs/net_mgmt/ciscoworks_device_fault_manager/2.0_IDU_2.0.6/installation_windows/guide/nt_Inst.pdf

Q. In the Search Results, the selected devices are not displayed as selected. They are displayed as selected only in All Devices group. Why?

A. This is the default behavior of HOSTree. The devices will be displayed as selected only in All Devices group, and not wherever the devices are listed.

Q. How can I avoid DFM generating Unreponsive alerts when the devices are in Natted Enviroment?

A. Based on the solution offered by the case 606342543, DFM may have walked the ipAddrTable for these Natted devices and got back the untranslated IP address. If you try to poll that IP address it will not work. In such a case, you can disable polling for selected IP addresses from the Alerts and Activities display screen.

To disable polling:


Step 1 From the Alerts and Activities display, start the view that contains your device.

Unmanaged devices will be in the Suspended Devices view.

The DDV opens. Depending upon the managed state of the device, either the Suspend or the Resume button is shown.

Step 2 Click Suspend to change the device's current managed state to Suspended.

DFM no longer polls any device components, nor does it process any traps. All alerts and activities change to the Cleared state, and the device is moved to the Suspended Devices view.

Subsequent events (including traps) are ignored and no longer processed.


Q. How can I get rid of alerts for a device that has been deleted?

A. To get rid of the alerts of a deleted device, you have to cleanup the database and re-initialize the DFM databases using dbRestoreOrig.pl.


Step 1 Go to the command prompt and stop the daemon manager by entering net stop crmdmgtd

Step 2 Go to C:Program Files\CSCOpx\bin

Step 3 Reinitialize the DFM databases by entering:

perl dbRestoreOrig.pl dsn=dfmInv dmprefix=INV

perl dbRestoreOrig.pl dsn=dfmFh dmprefix=FH

perl dbRestoreOrig.pl dsn=dfmEpm dmprefix=EPM

Step 4 Restart the daemon managers by entering net start crmdmgtd

After a few minutes, all processes will restart.


Q. Why HighUtilization on interface gets generated by DFM?

A. The reason could be a bug at the agent side. The agent side bug for MSFC is CSCdy46229. It was fixed in 12.2(15) and later versions. The agent side bug for VG248 device is CSCsj51190.

Q. Why does a device go into Unsupported state although it is Supported ?

A. Some devices, inspite of being supported go to Unsupported state because CDP is not enabled on them. DFM manages such devices only when CDP is enabled on them. Examples of such devices are CCC and CPA.

Q. What is the difference between Snmp Raw Trap Forwarding and Processed Snmp Trap alert/event Trap Forwarding? Does DFM support both of these methods?

A. Yes, DFM supports both ways of Trap forwarding.

Raw Trap is forwarded by the Device to DFM and DFM has to process it. To configure Raw Trap Forwarding, go to DFM > Other configuration > SNMP Trap Forwarding.

When DFM receives certain SNMP traps, it analyzes the data found in fields such as Enterprise/Generic trap identifier, Specific Trap identifier, and variable-bindings of each SNMP trap message.

If needed, DFM changes the property value of the object property. These are Processed Traps. To configure Processed event/alert trap forwarding, go to DFM > Notification Services > SNMP Trap Forwarding. This configuration can also send trap notifications if there is a threshold violation in the DFM managed devices.

For more information, refer to the DFM User Guide available at:

http://www.cisco.com/en/US/products/sw/cscowork/ps2421/products_user_guide_list.html

Q. How does DFM detect Trunk and Access ports?

A. If a port is connected to a system interface, the PortType is labelled as ACCESS and if the port is connected to another port, it is labelled as TRUNK.

To verify the PortType in Incharge, check the NeighboringSystems attribute under the port. This attribute indicates whether the port is connected to a switch, router, or a host interface. If the discovered port has no connection, then the default PortType is ACCESS.

To find the NeighboringSystem attribute, go to dmctl and enter get Port::PORT-<Port Name>

Q. Why devices in DCR do not get added to DFM in ACS mode?

A. For devices to get added into DFM in ACS mode, you have to create a Netwok Device Group in ACS server and add the devices that you want to be managed by DFM.


Note Netwok Device Groups will not be displayed under Network Configurations by default.


To create a Netwok Device Group in ACS server:


Step 1 In the ACS server, select Interface Configurations > Advanced Options

Step 2 Check the Netwok Device Groups check box

Step 3 Select Network Configurations > Add Entry

Step 4 Enter the name of the group (say DFM_Devices).

A new group named DFM_Devices will be displayed.

Step 5 Click DFM_Devices, and add an entry for devices under the DFM_Devices AAA client

Step 6 Add an entry for AAA server for DFM_Devices. The default is localhost IP

Step 7 Click Submit

Step 8 Restart the server.

Step 9 Add those devices to DFM through DCR.

The devices will be managed by DFM.


Q. Is the AttachToNCM script supported in DFM2.x/DFM3.x versions?

A. No. AttachToNCM script is not supported in DFM2.x/DFM3.x versions.

Q. What is the meaning of different discovery percentages?

A. The following list explains the different discovery percentages:

10% — Startup, where devices have not been handed over to incharge processes yet

40% — Devices have been successfully handed over to incharge processes

70% — Incharge processes have successfully discovered devices and handed the information over to Cisco code

90% —Discovered devices must be placed in appropriate groups. Device information has been sent to OGS and group information from OGS is awaited. From DFM 3.0, this scenario is not available.

Q. How can I troubleshoot device discovery stuck at 10%?

A. If device discovery is stuck at 10%, it means that devices to be discovered have not been communicated to Incharge processes yet. To troubleshoot device discovery stuck at 10%:

On Solaris:


Step 1 Enter the pdshow command to check if DfmServer and DfmBroker processes are running.

Step 2 Check if there are multiple instances of brstart and sm_server.

The name of binary for DfmBroker is brstart and for DfmServer, it is sm_server.

Step 3 Enter the following commandline:

/usr/ucb/ps -auxww |grep brstart

/usr/ucb/ps -auxww |grep sm_server

For example:

marver-sol-daily# /usr/ucb/ps -auxww |grep brstart

root 11751 0.1 0.1 984 648 pts/7 S 11:34:49 0:00 grep brstart

casuser 11577 0.0 0.42465613496 ? S 09:42:38 0:00 brstart --output --port=9002 ---user=casuser

If you see more than one instance of each process, enter the command ptree to get all related process.

For example:

marver-sol-daily# ptree 11577

11461 /opt/CSCOpx/objects/dmgt/dmgtd.sol

11577 brstart --output --port=9002 ---user=casuser

11588 /opt/CSCOpx/objects/smarts/bin/system/sm_logerror 131

11604 /opt/CSCOpx/objects/smarts/bin/system/sm_authority

11605 /opt/CSCOpx/objects/smarts/bin/system/sm_logerror 136

Step 4 Except for dmgtd.sol, manually terminate all processes found in the ptree command by entering the command: kill -9 .

Step 5 Stop the daemon manager by entering net stop crmdmgtd.

Wait for five minutes.

Step 6 Enter netstat -a |grep 9002 to make sure that port 9002 is not in listening state or timed wait state.

Step 7 Enter netstat -a | grep 435 to make sure that ports 43501 to port 43508 are not in listening or timed wait state.

If the ports are in listening state, use the Unix utility to find the process that owns the ports and terminate that process.

Step 8 Start the daemon manager by entering net start crmdmgtd.

Step 9 Enter the following command: /objects/smarts/bin/brcontrol

The following is an example of the output:

Broker is located at: localhost:9002 Started: Oct 26 09:42:39 2005

Domain Host Name Port Proc ID State Last Chg Time

DFM marver-sol-daily 50449 11589 RUNNING Oct 26 09:42:58 2005

Step 10 Check whether the displayed host name is the host name in DNS or /etc/hosts


On Windows:


Step 1 Download the tools from the following location: http://www.sysinternals.com/Utilities/PsTools.html

Step 2 Unzip it into a directory.

Step 3 Enter pdshow DfmServer pdshow DfmBroker to check whether the processes DfmServer and DfmBroker are running.

Step 4 Enter /objects/smarts/bin/brcontrol

Step 5 Check whether the host name is the same as in DNS.

Step 6 Check whether there are multiple instances of brstart and sm_server

You can use the downloaded Pstool to see one branch each of brstart and sm_serve.

For example:

pslist -t

brstart 5708 8 1 16 8476 1104 264

brstart 5880 8 7 86 32720 11240 9664

sm_authority 6452 8 1 52 14376 3616 1528

sm_server 6332 8 1 16 8476 1104 264

sm_server 6416 8 124 678 212696 58784 56812

sm_authority 6444 8 1 47 14376 3572 1528

There should be only one branch. You need to terminate any extra branches.

Step 7 Stop the daemon manager by entering net stop crmdmgtd.

Wait for five minutes.

Step 8 Enter netstat -a -n -p tcp to make sure that ports 9002 and 43501 to 43508 are not in listening state or timed wait state.

Step 9 Start the daemon managerby entering net start crmdmgtd.


Q. How can I troubleshoot device discovery stuck at 40%?

A. To troubleshoot device discovery stuck at 40%:


Step 1 Make sure that the device under question is responding to ping messages.

Step 2 Make sure that the device responds to snmpwalk by entering:

/objects/smarts/bin/sm_snmp -c --dest= walk

Step 3 Enable discovery logging in incharge process by entering:

/objects/smarts/bin/dmctl -s DFM


Note In some cases, you need to enter your UserID and Password.


Step 4 From dmctl prompt, enter:

ICF_TopologyManager::ICF-TopologyManager::DebugEnabled TRUE

Step 5 Exit out of dmctl

Step 6 Trigger the rediscovery of the device

The log file from the location /objects/smarts/local/logs/DFM.log will have discovery information in detail. It will provide the root cause for the failing device discovery.

Step 7 Restore debug to False.


Q. How can I troubleshoot device discovery stuck at 90%?

A. To troubleshoot devices stuck at 90% discovery:


Step 1 Increase the log level for Inventory Service to debug.

Step 2 Stop the daemon manager by entering net stop crmdmgtd

Step 3 Take a backup of the following file:

/MDC/tomcat/webapps/triveni/WEB-INF/classes/log4j-ogs.properties

Step 4 Replace the string FATAL by string DEBUG

Step 5 Start the daemon manager by entering net start crmdmgtd

Step 6 Collect the contents of /log/dfmLogs/TIS file by entering:

On Solaris :

/var/adm/CSCIpx/log/DFMServer.log

On Windows :

/log/DFMServer.log


Q. How can I stop nGenius RealTimeMonitoring in Solaris ?

A. If nGenius RealTimeMonitoring is not getting uninstalled, stop it before CDONE or DFM installation and do not restart it manually.

Enter /opt/NetScout/rtm/bin/stop1 to stop it. It will get started again when the system is rebooted.

Q. How can I receive Syslog messages from a dfm server?

A. To receive Syslog messages from a dfm server:


Step 1 Enable Syslog from Notification Services > SysLog Notification

Step 2 Point it to any Solaris machine and run the following:

/etc/init.d/syslog start

tail -f /var/adm/messages


Q. How can I perform rediscovery of devices in DFM through CLI ?

A. To perform a rediscovery of:

All devices:

Enter dmctl -s DFM invoke ICF_TopologyManager::ICF-TopologyManager discoverAll

A specific device:

Enter dmctl -s DFM invoke ICF_TopologyManager::ICF-TopologyManager ::

Q. How can I manage or unmanage ports and interfaces from CLI?

A. To manage or unmanage ports and interfaces from CLI, enter dmctl -s=DFM invoke class::instance op [arg1 ...]

The following is the list of examples:

dmctl> invoke Port::PORT-5.1.2.2/10123 manage

dmctl> invoke Interface::IF-5.1.3.2/1 manage

dmctl> invoke Port::PORT-5.1.2.2/10123 unmanage

Q. How can I import devices into DCR through CLI?

A. To import devices into DCR through CLI:


Step 1 Run the command dcrcli -u admin with the CiscoWorks username as the argument.

Step 2 Enter the CiscoWorks password when prompted.

Step 3 At the dcrcli prompt, enter dcrcli>impFile fn="path to csv import file" ft=csv


Q. How can I enable CAM logs for debugging ACS Configuration?

A. To enable CAM logs:


Step 1 Go to the command prompt and enter /opt/CSCOpx/MDC/bin/ccraccess -updateLog Core cam DEBUG

Step 2 Restart the daemon manager after enabling logs.

The logs will be under NMSROOT/MDC/log


Q. How can I create a link to the Java Plug-in in Netscape7.x and Mozilla 1.7.x?

A. Create a symbolic link to the Java Plug-in libjavaplugin_oji.so file in the Netscape 6.x/7.x or Mozilla Plugins directory. To create the link, go to the command prompt and enter:


Step 1 cd /plugins

Step 2 ln -s /plugin/sparc/ns610/libjavaplugin_oji.so .


Include the period at the end.

For Netscape 6.x/7.x or Mozilla browsers, restart your browser.

In Netscape, go to Help > About Plug-ins to confirm that the Java Plug-in is loaded.

Q. How can I avoid getting an alert when the device Cisco Intelligent Gigabit Switch Module for IBM Bladecenter does not contain a fan?

A. Though the device Cisco Intelligent Gigabit Switch Module for IBM Bladecenter does not contain a fan, the current SNMP MIB expects to see a fan resulting in an Alert.

To avoid these Alerts, do the following:


Step 1 Go to DFM > Device Management > Device Details

Step 2 Select the device and click View

Step 3 Click the device link

Step 4 Select Environment > Fan

Step 5 Change the Managed State to False


Q. How does DFM react to a Cisco ISR with Inline POE switch module when there is no -48V power supply installed?

A. When there is no -48V power supply installed, the device Cisco ISR with Inline POE switch module shows up in SNMP as being in a Critical state. There is no option to unmanage it from the GUI. You need to perform a command line edit of the database whenever the device is added or re-added to LMS or DFM.

Enter the following command to edit the database:


Step 1 Find out to which DFM domain (DFM or DFM1) the device belongs.

From the command prompt, enter:

NMSROOT/objects/smarts/bin/

Step 2 Enter

dmctl -s DFM geti ICIM_UnitaryComputerSystem

and

dmctl -s DFM1 geti ICIM_UnitaryComputerSystem

The list of devices managed by DFM and DFM 1 domains are listed.

Step 3 Enter dmctl -s Domain Name

For example, if you find the device listed under DFM domain, enter: dmctl -s DFM

Your login will be successful and you will see the prompt dmctl>

Step 4 From Detailed Device View (DDV), get the power component name of the device

Or

Enter dmctl>geti PowerSupply to list the power components of the devices:

The power components are listed.

Step 5 Enter dmctl>get PowerSupply::componentname::IsManaged to check the management status of the component.

For example, if PWR-fl-69-69-9-45.dyn.embarqhsd.net/2 is the power component, enter

dmctl>get PowerSupply::PWR-fl-69-69-9-45.dyn.embarqhsd.net/2::IsManaged

If it returns TRUE, set it to False.

Step 6 Enter:

dmctl>invoke PowerSupply::PWR-fl-69-69-9-45.dyn.embarqhsd.net/2 unmanage

Step 7 Run Step 5 again to check if the management state is set as FALSE

Step 8 To apply the changes, enter:

dmctl>invoke ICF_PolicyManager::ICF-PolicyManager reconfigure


Q. What happens when VG200 Routers have an ISDN PRI carrying voice traffic?

A. When VG200 Routers have an ISDN PRI carrying voice traffic, by default, DFM generates a Warning alert whenever a phone call uses a B-channel. For every ISDNBCHANNEL, set the Managed State as False in Device Detailed View. They are then listed with the Backup Mode.

However, removing the Backup Interface Support Settings is not a workaround since that will cause LMS to report an error whenever any B-channel is not in use.

Q. I am unable to sort Cleared Alerts in DFM. Why?

A. Alerts are moved to Cleared state when their last active event has been in the Cleared state for 20 minutes. Cleared alerts remain in that state for twenty minutes. After that time it is removed from the Alerts and Activities Display. So, whenever the Status column is sorted, the Cleared alerts remain only in the end of the Status column. Sorting is not available for Cleared alerts.