This document describes how to troubleshoot Simple Network Management Protocol (SNMP) framework architecture from a Quantum Policy Suite (QPS) perspective.
There are no specific requirements for this document.
The information in this document is based on QPS Release 5.5 and later.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Each virtual machine (VM) runs an "snmpd" process. The alerts or traps generated by a VM in QPS are forwarded to a user Network Management System (NMS) through an active load balancer such as lbvip01 or lbvip02.
lbvip01 sends SNMP traps to an external NMS server for external communication. A high QPS availability heartbeat ensures that
lbvip01is always available, either on the load balancer VM
lbvip02 is used by all VMs (for example
lb01x, qns0x, portal0x, and so forth) in order to send SNMP traps to an active load balancer for internal communication.
lbvip02 IP addresses on
lb02 are located in the
162 is the default UDP port used for SNMP trap messages.
Broadhop is the default SNMP community string to access the framework.
SNMP Configuration Files
Four important QPS Release 5.5 SNMP configuration files or scripts reside on the load balancer VMs:
/etc/snmp/snmpd.conf is used for IP address and port configuration. The "snmp"service polls the VM every minute for values that are configured in this file. The file sends version 2 traps to
lbvip02 through the line:
trap2sink lbvip02 broadhop.
/etc/snmp/scripts/snmp_communities defines the SNMP community name to communicate with the external NMS system, for example,
/etc/snmp/scripts/component_trap_conver transforms the trap to a Quantum Network Suite (QNS) trap, as defined in the BROADHOP-NOTIFICATION MIB, and sends it to the external user NMS IP address.
/etc/snmp/snmptrapd.conf is the Net-SNMP notification receiver configuration file which uses one or more configuration files to control its operation and how received traps must be processed. The file sends the actual traps to the external NMS system through
The Policy and Changing Rules Function (PCRF) client virtual machines, such as pcrfclient01 and pcrfclient02, contain three XML files related to QPS Release 5.5 SNMP configuration:
attributes.xml contains the mapping between an SNMP Object Identifier (OID) and the Java Management Extension (JMX) javabean or attribute that holds the value.
managers.xml lists the remote hosts to receive the SNMP traps.
notifications.xml lists the SNMP traps that can be sent and contains a mapping of the attributes to OIDs.
The three XML files are located in the
/etc/broadhop/<servers>/snmp directory. In order to find the correct directory:
Use the cat utility in order to locate the correct server file in the PCRF client virtual machine.
After you view the file of the server, you can find the directory with the three XML files for a particular VM as follows:
The MIBs used by SNMP is located on the respective VM in the
BROADHOP-QNS-MIB.mib defines retrievable statistics and Key Performance Indicators (KPIs).
BROADHOP-NOTIFICATION-MIB.mib defines available notifications and traps.
Basic Steps to Troubleshoot SNMP
Perform these steps on the active load balancer virtual machine, either on lb01 or lb02.
- Ensure that the snmpd process is functional on the respective VM with the command:
service snmpd status
- If the snmpd service has stopped, then start the service with the command:
service snmpd start
- Ensure that the iptables are turned off and check the status of UDP port 162 (if UDP port 162 is also used at the NMS).
- Ensure that the Red Hat Package Manager (RPM) versions for SNMP are the same, with the command:
rpm –qa | grep snmp
Output Example with All Versions at 220.127.116.11:
net-snmp is a package required to deploy SNMP.
net-snmp-utils is a package that contains various utilities for use with NET-SNMP and is required to run utilities that manage the network with the SNMP protocol.
net-snmp-libs contains the runtime libraries for shared binaries and applications.
- Enter the
rpm-qi command to view detailed information on RPM.
rpm -qi net-snmp-18.104.22.168-17.el5
rpm -qi net-snmp-utils-22.214.171.124-17.el5
rpm -qi net-snmp-libs-126.96.36.199-17.el5
- In order to ensure that QPS sends SNMP traps to the appropriate NMS, check to see that the external NMS IP is defined in the load balancer VM under
/etc/snmp/scripts/ component_trap_convert with the command:
Note: Where 'corporate_nms_ip' is the NMS IP, for example, 10.106.74.10.
- Ensure that the
snmpd.conf file contains the line
rocommunity Broadhop with the command:
This line is required because all of the internal traps from the various QNS VMs to the active load balancer VM are sent over the default community name
- Check to ensure that the load balancer and external NMS system use the same trap community name with the command:
For example, ensure that the load balancer and external NMS system both use
- Check the
/var/log/snmp/trap log file to ensure that the traps from the respective QNS VMs have properly reached the active load balancer VM.
Trap Log File Example:
Jun 15 04:02:16 AIO-POD1 snmpd: Received SNMP packet(s) from UDP: [127.0.0.1]
Jun 15 04:02:16 AIO-POD1 snmpd: Connection from UDP: [127.0.0.1]:48071
Jun 15 04:02:14 AIO-POD1 snmptrapd: 2014-06-15 04:02:14 AIO-POD1.cisco.com
[10.106.74.34] (via UDP: [127.0.0.1]:39334) TRAP, SNMP v1, community broadhop
NET-SNMP-MIB::netSnmpNotificationPrefix Enterprise Specific Trap (NET-SNMP-AGENT-MIB
::nsNotifyRestart) Uptime: 62 days, 5:04:42.43
- Capture the logs from
/var/log/messages on the active load balancer and look for any error messages related to SNMP.