Guest

Virtual LANs/VLAN Trunking Protocol (VLANs/VTP)

Troubleshooting the Catalyst VMPS Switch

Cisco - Troubleshooting the Catalyst VMPS Switch

Document ID: 23797

Updated: Aug 30, 2005

   Print

Introduction

With the VLAN Management Policy Server (VMPS), a Catalyst switch administrator can dynamically assign a network device to a particular VLAN. This technology is useful in sites that contain a large number of mobile users. This document covers how to troubleshoot host registration on a Catalyst switch VMPS. Host registration refers to the ability to assign a VLAN based on the Ethernet MAC address of a PC. This document covers the required minimum software level necessary to run both the VMPS and VMPS client, and offers suggestions on how to troubleshoot the various stages and components of a Dynamic VLAN (DVLAN) assignment.

Note: A Catalyst switch VMPS only provides host registration. In order to assign a VLAN through NT authentication (also known as user registration) use the Cisco Secure User Registration Tool .

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This table lists the minimum software requirements to support VMPS on various Cisco Catalyst switch products:

Product VMPS Support VMPS Client Support
Catalyst 4000 Family (Catalyst OS) Yes, 7.2(x) and later Yes, all software releases
Catalyst 4000/4500 (Cisco IOS software) Not currently supported Yes, 12.1(13)EW and later
Catalyst 2900XL/3500XL Not supported Yes, 11.2(8)SA4 and later, Enterprise Software Edition only
Catalyst 2950/2955/3550 Not supported Yes, all software releases
Catalyst 2948G-L3/4908G-L3 Not supported Not supported
Catalyst 5000/5500 Family Yes, 2.3.x and later Yes, 2.3.x and later
Catalyst 6000/6500 Family (Catalyst OS) Yes, 6.1(x) and later Yes, all software releases
Catalyst 6000/6500 Family (Cisco IOS software) Not currently supported Not currently supported

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Troubleshoot DVLAN Membership

The VLAN Query Protocol (VQP) is the primary transport for VMPS data. The VQP uses User Datagram Protocol (UDP) port 1589. This example illustrates the important steps in the DVLAN membership process, and shows how a client is dynamically assigned a VLAN on the basis of the MAC address:

157-a.gif

  1. The PC sends a frame to the switch.

  2. The VMPS client learns the PC MAC address on the dynamic port.

  3. The VMPS client sends a VQP request to the VMPS. The request contains the VMPS client IP address, the PC MAC address, the PC port number, and the VTP Domain.

  4. The VMPS parses the database file for PC VLAN assignment.

  5. The VMPS sends a VQP response to the VMPS client.

  6. If the VQP response contains a VLAN assignment, the VMPS client assigns it to the VLAN. Otherwise, the client denies the PC access.

You can classify most problems that you encounter into these three categories:

Before You Troubleshoot DVLAN Membership

Before you troubleshoot DVLAN membership problems on a VMPS client that runs Catalyst OS (CatOS), increase the DVLAN logging level on the VMPS client from logging level 2 to logging level 7 (debugging). Then, issue the set logging level dvlan 7 default command.

Note: In some CatOS releases, when you increase the DVLAN logging level, an error can occur, which states that this is an invalid facility. This error is a result of Cisco bug ID CSCdu19163 (registered customers only) , and this issue is resolved in Cisco IOS Software Releases 5.5(8), 6.3(1), and later.

You must enable this command on VMPS clients when you troubleshoot because this command provides vital information about the DVLAN membership failure.

After you complete troubleshooting and resolve the issue, you can reduce the DVLAN logging level from logging level 7 to logging level 2. Issue the set logging level dvlan 2 default command.

In CatOS switches, you can perform additional debugging if you issue the set trace dynvlan 6 command directed by a Technical Support engineer. Enable this command before you plug in or power up the PC with VLAN assignment issues. Wait for about one minute before you disable the command. In order to disable the command, issue the set trace dynvlan 0 command in the enable mode.

caution Caution: Enable this command with caution. The debug output can cause the switch to crash, if several PCs leave and join dynamic ports on the same switch. You must disable the console logging before you enable this command.

Troubleshoot Connectivity Between the PC and the VMPS Client Switch

Connectivity issues between the PC and the VMPS client can cause the DVLAN membership to fail if the VMPS client switch cannot obtain the MAC address of the PC. In this case, the port remains in the “inactive” state with a VLAN assignment of dyn-, as shown in this example:

vmps_client> (enable) show port 3/2

Port  Name               Status     Vlan       Level  Duplex Speed Type
----- ------------------ ---------- ---------- ------ ------ ----- ------------
 3/2                     inactive   dyn-       normal a-half a-10  10/100BaseTX

When a VMPS reconfirmation occurs on the VMPS client switch and the switch cannot obtain the MAC address of any PC on any dynamic port, then this message appears:

%DVLAN-4-NOHOST:No host connected to dynamic ports, reconfirm
aborted

VMPS reconfirmation occurs when the VMPS client asks the VMPS if the dynamic port assignments are correct and if the correct MAC addresses have been assigned to the correct ports. By default, this reconfirmation occurs about every 60 minutes. Issue the show vmps command on the VMPS client to determine the VMPS reconfirmation time.

If you are sure that there is at least one PC connected to a dynamic port, perform these steps:

  1. Disconnect the PC from the switch.

  2. Issue a ping command from the PC to anywhere.

  3. Issue the reconfirm vmps command on the connecting VMPS client.

The VMPS client tries to confirm with the VMPS that the address of the PC must be assigned to that port. If the MAC address cannot be reconfirmed, this message appears:

%DVLAN-2-MACNOTRECONFIRMED:Mac [00-40-f4-22-31-0f] is not reconfirmed
%DVLAN-1-DENYHOST:Host 00-40-f4-22-31-0f denied on port 3/11

The issue can be either a problem with the VMPS database (see the Troubleshooting the VMPS Database File section of this document) or with communication between the VMPS client and the VMPS (see the Troubleshooting Connectivity Between the VMPS Client and VMPS section of this document).

If the VMPS client switch continues to state that there is no host connected to the dynamic port, and the reconfirmation aborts, troubleshoot the issue as a physical layer connectivity issue between a VMPS client and a PC. For more information, refer to the Physical Layer Troubleshooting section of Troubleshooting Switch Port and Interface Problems.

Troubleshoot Connectivity Issues Between the VMPS Client and the VMPS

When a loss of connectivity occurs between a VMPS client and a VMPS, the VMPS reconfirmation can fail and produce the DVLAN-2-MACNOTRECONFIRMED error message. The port loses the DVLAN assignment, as in this example:

%DVLAN-2-MACNOTRECONFIRMED:Mac [00-00-f4-11-11-0f] is not reconfirmed
%DVLAN-1-DENYHOST:Host 00-00-11-11-11-0f denied on port 3/10

VMPS reconfirmation occurs when the VMPS client asks the VMPS if the dynamic port assignments are correct and if the correct MAC addresses have been assigned to the right ports. By default, this check occurs about every 60 minutes. Issue a show vmps command on the VMPS client to determine the VMPS reconfirmation time.

Complete these steps to troubleshoot connectivity issues between a VMPS client and a VMPS:

  1. Ping the VMPS from the VMPS client. If the ping fails, troubleshoot the problem as a general switch connectivity issue or as a general routing problem.

    For more information, refer to Configuring InterVLAN Routing and ISL/802.1Q Trunking on a Catalyst 2900XL/3500XL/2950 Switch Using An External Router.

  2. If the ping command is successful between the VMPS client and the VMPS, ensure that there is no device—between the VMPS client and the VMPS network path—that blocks UDP port 1589.

  3. If the connectivity between the VMPS client and VMPS is intermittent (some data gets lost along the way), you can try to increase the VMPS retry interval on the VMPS client, as a workaround. Issue the set vmps server retry command. By default, the VMPS client attempts thrice. In an environment with intermittent connectivity, when you increase the VMPS retry interval, you give the client more chances to connect to the VMPS before the client gives up and VLAN membership fails.

Troubleshoot the VMPS Database File

The VMPS database file defines all of the parameters that control Host registration on the Catalyst switch VMPS. You must manually create the file with a text editor. These are the four primary components of the VMPS database:

  • Port group—A collection of ports found on various switches.

  • VLAN group—A collection of VLANs that can be associated to a port group.

  • Port policy—Associates a port group with a VLAN group or VLAN name.

  • MAC address to VLAN association table—Specifies to which VLAN a MAC address is assigned.

    Note: You can assign a MAC address only to one VLAN. If you have a MAC address associated with two different VLANs, only the first one listed is used.

    Note: The next case study illustrates how the VMPS database file works and helps you troubleshoot any VMPS database failure issue.

Case Study Description

The XYZ Company has these three areas:

  • Area 1 contains the training rooms and facilities.

  • Area 2 is for executives and sales officials.

  • Area 3 is for engineers.

Three VLANs called “executive_vlan,” “sales_vlan,” and “eng_vlan” are created. Executives must be put on VLAN executive_vlan, if they are in their office or in the training room. All PCs of sales representative are assigned to the sales_vlan, and all PCs of engineers are assigned to the eng_vlan.

This is an example of the design scheme of the XYZ Company:

157-b.gif

Switch Area Configuration
A Training Area Executives, sales officials, and engineers are granted access to all ports and are assigned to their respective VLANs.
B Sales/Executive Area Only executive PCs are granted access to port 2/1-5; executive PCs are assigned to executive_vlan. Only sales official PCs have access to port 2/10-15; sales official PCs are assigned to sales_vlan. Access to other ports on the switch is denied to executives, sales officials, and engineers.
C Engineering Area Only engineering PCs are granted access to the switch; engineering PCs are assigned to eng_vlan.

VMPS Domain Name and VTP Domain Name

The VMPS domain name and the VLAN Trunk Protocol (VTP) domain name must match. The VMPS domain name is case sensitive. For example, if the VTP domain name is “XYZ_company”, the VMPS domain name must be “XYZ_company” and not “xyz_company.”

VMPS Port Policy

Three port policies will be created: the first for executives, the second for sales officials, and the third for engineers.

Again, the VLAN names used to create the VMPS port policies are case sensitive and must match the VLAN name in the VLAN database. As a general rule, Cisco recommends that you use lower case to name all VLANs and VTP domains.

The executive port policy states that an executive PC connected to port 2/1-5 on Switch B (192.168.2.2) or any port on Switch A (192.168.2.1) is assigned to executive_vlan.

The sales port policy states that a sales official PC connected to port 2/11-15 on Switch B (192.168.2.2) or any port on Switch A (192.168.2.1) is assigned to sales_vlan.

The engineering port policy states that an engineer PC connected to any port on Switch A (192.168.2.1) or Switch C (192.168.2.3) is assigned to eng_vlan.

All other users who attempt to connect to the dynamic ports are denied access and the port is shut down. A syslog message informs the administrator of the port shutdown, and appropriate action is taken thereafter.

This sample VMPS configuration file for XYZ Company shows the resultant VMPS database file:


!--- VMPS domain name must be the same as the VTP domain of the network.
!--- This value is case sensitive. If the VTP domain is TestVmps, the VMPS
!--- domain must also be TestVmps.

vmps domain xyz_company
!

!--- If the VMPS cannot assign the host a VLAN, shut down the port. If the 
!--- VMPS mode is "open," a log message, which states that access is denied 
!--- is produced and the port becomes inactive.

vmps mode secure
!

!--- No fallback VLAN is configured for the XYZ network, so it is commented out.
!--- A fallback VLAN is assigned to a PC whose MAC address is not present in the 
!--- database. The fallback VLAN is usually a VLAN where a user cannot access  
!--- sensitive network resources.

!! vmps fallback nonsecure_vlan
!

!--- List of MAC addresses that will be assigned to a VLAN.
!--- The VLAN-name matches the names given to VLANs on the VMPS.
!--- VLAN names are case sensitive, as is the VMPS domain name.

!

!--- MAC address format must be xxxx.xxxx.xxxx . Any other format will not work.

!vmps-mac-addrs
address 0000.0000.0001 vlan-name eng_vlan
address 0000.0000.0002 vlan-name eng_vlan
address 0000.0000.0003 vlan-name sales_vlan
address 0000.0000.0004 vlan-name sales_vlan
address 0000.0000.0005 vlan-name executive_vlan
address 0000.0000.0006 vlan-name executive_vlan
!
!!!!!!!!!!!!! Executive policy !!!!!!!!!!!!!!!!!!!!!!!!!!!!

!--- This port policy states that the VMPS checks the MAC address of the
!--- PC plugged in any port in Switch A (192.168.2.1) or port 2/1-5 in 
!--- Switch B (192.168.2.2) against the MAC addresses associated to the 
!--- executive_vlan in the vmps-mac-addrs database.

!

!--- When you create a port group, a range command such as
!--- device x.x.x.x port 2/1-5 is not allowed. This will produce 
!--- a parse error when the VMPS database downloads.

vmps-vlan-group executive
vlan-name executive_vlan
!
vmps-port-group executive_ports
device 192.168.2.1 all-ports
device 192.168.2.2 port 2/1
device 192.168.2.2 port 2/2
device 192.168.2.2 port 2/3
device 192.168.2.2 port 2/4
device 192.168.2.2 port 2/5
!
vmps-port-policies vlan-group executive
port-group executive_ports
!
!!!!!!!!!!!!! Sales policy !!!!!!!!!!!!!!!!!!!!!!!!!!!!

!--- This port policy states that the VMPS checks the MAC address
!--- of the PC plugged in any port in Switch A (192.168.2.1) or port
!--- 2/10-15 in Switch B (192.168.2.2) against the MAC addresses associated
!--- to the sales_vlan in the vmps-mac-addrs database.

!

!--- Notice that you can bind a port group to a VLAN name instead of a 
!--- VLAN group. A VLAN group allows a port group to be bound to multiple 
!--- VLANs. In this case, the ports defined in the port group sales_port can use 
!--- the MAC addresses defined in the sales_vlan.

vmps-vlan-group sales
vlan-name sales_vlan
!
vmps-port-group sales_ports
device 192.168.2.1 all-ports
device 192.168.2.2 port 2/10
device 192.168.2.2 port 2/11
device 192.168.2.2 port 2/12
device 192.168.2.2 port 2/13
device 192.168.2.2 port 2/14
device 192.168.2.2 port 2/15
!
vmps-port-policies vlan-name sales_vlan
port-group sales_ports
!
!!!!!!!!!!!!! Engineer policy !!!!!!!!!!!!!!!!!!!!!!!!!!!!

!--- This port policy states that the VMPS checks the MAC address of
!--- the PC plugged in any port in Switch A (192.168.2.1) or Switch C 
!--- (192.168.2.3) against the MAC addresses associated to the eng_vlan
!--- in the vmps-mac-addrs database. 

!
vmps-vlan-group engineering
 vlan-name eng_vlan
 !
 vmps-port-group eng_ports
 device 192.168.2.1 all-ports
 device 192.168.2.3 all-ports
 !
vmps-port-policies vlan-group engineering
 port-group eng_ports
!

Troubleshoot the VMPS Database File Download

In order for the VMPS to function properly, you must download the database and configuration file from a Remote Copy Protocol (RCP) or Trivial File Transfer Protocol (TFTP) server to the VMPS switch. This process fails in these scenarios:

  • When the VMPS database file does not exist or is incorrectly named on the RCP or TFTP Server.

    If the database file does not exist or does not match the database file field in the output from the show vmps command of the VMPS switch, the VMPS switch produces this error:

    %VMPS-2-DOWNLOADFAIL2:Unable to download file vmps_db
  • When the VMPS switch cannot contact the RCP or TFTP Server.

    If the VMPS switch cannot connect to the RCP or TFTP server, the VMPS switch produces this error:

    %VMPS-2-DOWNLOADFAIL2:Unable to download file vmps_db

    Notice that this is the same error as the one produced if the VMPS switch can contact the RCP or TFTP Server, but the database file does not exist or is incorrectly named. In this case, you must verify network connectivity between the RCP or TFTP server and the VMPS switch. If network connectivity exists between the VMPS and the VMPS client, verify whether the RCP or TFTP port of the server is open and ready to receive connections.

  • When the database file contains configuration errors.

    If a database configuration error is detected during the download, the VMPS switch produces this error:

    %VMPS-2-PARSEMSG:PARSER: 31 lines parsed, Errors 2

    This error is probably the hardest to troubleshoot, because the VMPS switch does not always tell you which line contains the errors. If the incorrectly configured line is not mentioned, try to download the VMPS database in sections. For example, the sample configuration file for XYZ Company has a sample database file. If there is an error in the file, download a file that only contains the vmps domain, vmps mode, and vmps-mac-addr sections. If this download is successful, add the executive port policy to this file and repeat the download. Continue until the parse error is produced, then inspect the last-attached section carefully for any configuration errors. Usually, these are typographical errors.

Note: If you reset or power cycle the VMPS server switch, the VMPS database downloads from the TFTP server automatically and VMPS is enabled again. However, if you reset or power cycle the TFTP server there are no options for TFTP backup for VMPS. As a result, the switch continues to use the last learned information from the TFTP server.

Troubleshoot 2900XL/3500XL/2950/3550 VMPS Clients

The 2900XL, 3500XL, 2950, and 3550 Catalyst Series switches can all act as VMPS clients. Enable the debug switch vqpc command to perform VMPS debugging on the 2950 and 3550. Cisco IOS Software Release 12.1(13)EA1 and later versions support this debug command. VMPS debugging and troubleshooting on the 2900XL and 3500XL switches is limited to the show vmps command and the interpretation of log messages produced during VMPS problems. This section discusses and explains some of the most commonly encountered VMPS syslog messages.

%VQPCLIENT-2-DENY: Host 0028.5192.4000 denied on interface	Fa0/x

This log message is an informational message, and commonly appears when the VMPS refuses to assign a VLAN to the specified MAC address. If this MAC address must be allowed on the specified port, verify the VMPS configuration. See the Troubleshooting the VMPS Database File section of this document for more information.

If a hub with several PCs is connected to a dynamic port, you can see several VQPCLIENT-2-DENY messages for the PCs connected to the hub. The VQPCLIENT-2-TOOMANY message can appear subsequently. Cisco recommends that you connect only one PC to each dynamic port.

Some Network Interface Cards (NICs), such as the 3Com 3C574/3C575, can cause the switch to repeatedly produce the VQPCLIENT-2-DENY log message. In this case, upgrade to the latest NIC drivers to resolve the issue.

This message appears when the dynamic port receives a burst of MAC addresses that are all associated with the same port and the port is unable to process any VQP request:

%VQPCLIENT-2-TOOMANY: Interface Fa0/x shutdown by active host	limit

This log message appears when the same port is given two different VLAN assignments within 10 seconds of each other:

%VQPCLIENT-3-THROTTLE: Throttling VLAN change on Fa0/x

When this problem occurs, leave the port in the old VLAN and delete the MAC address that provoked the change, so that the address can be relearned and a new request can be sent to the VMPS if it transmits again.

This is an informational message and does not usually indicate any problem. If several %LINK-3-UPDOWN log messages accompany this message, check whether the affected port is flapping. These log messages indicate a link change on the affected port. In this case, check physical connectivity between the PC and the switch port. For more information, refer to the Physical Layer Troubleshooting section of Troubleshooting Switch Port and Interface Problems.

Known VMPS Caveats

This table lists known VMPS caveats that are useful when you troubleshoot VMPS:

Bug ID Symptom Resolution
CSCdw23807 When the end station is moved from one port of a hub to another port on a second hub (with both the hubs connected to a CatOS switch, configured as a VMPS client), the end station is denied assignment of a VLAN. Even if the end station is connected directly, the MAC address is denied. Fix integrated in 6.3(6), 7.1(2).
CSCdr09366 The set port membership mod/port dynamic command on a Supervisor II/IIIG can fail and produce the trap not supported in hardware error message. Fix integrated in 5.5(6), 6.2(1), 6.1(3).
CSCdp68303 CSCdr95115 VMPS download fails and produces this error message:
%VMPS-2-NOMEM:Out of memory
%VMPS-2-DNLDFAIL:Download Failed.
VMPS is now inactive
Fix integrated in 5.4(1), 4.5(6).
CSCdx12337 When an IP phone is connected to a dynamic port and a PC is connected to the IP phone, the VMPS client cannot properly assign a VLAN to the PC, if the PC connected to the IP phone is changed. Currently, this is a limitation in VMPS. DVLAN membership occurs only when a dynamic port leaves and joins the bridge. If you change the PC connected to the IP phone and want the DVLAN membership to work properly, disconnect the IP phone and reconnect the IP phone to the switch port.
CSCds77648 UDP socket overflow on the VMPS socket 1589 upon VMPS reconfirm or VMPS download. This causes the download to fail. Fix integrated in 6.3(1), 5.5(8), 4.5(13).
CSCdu19163 You cannot set the log severity level to 7 for the DVLAN facility in certain versions of CatOS software. When you set the debugging severity level, the switch states that the facility is invalid, as in this example:
Console> (enable) set logging level dvlan 7

Invalid Facility

Console> (enable)
Fix integrated in 5.5(9), 6.3(1).
CSCeb36856 Sometimes, a Catalyst 6000 switch that runs 7.6(1) is unable to dynamically assign its own switch ports to a VLAN. When the bug occurs, the ports remain in an inactive state. Fix integrated in 7.6(3)

Information to Gather Before You Contact Cisco Technical Support

In order to better assist customers, Cisco Technical Support asks that you issue these commands to obtain information from the VMPS client and the VMPS:

From the VMPS Client

  • show tech-support command log

  • show logging buffer -1000 command log (CatOS)

  • show log command log (Cisco IOS software)

From the VMPS

  • show tech-support command log

  • a copy of the VMPS database file

  • show logging buffer -1000 command log (CatOS)

  • show log command log (Cisco IOS software)

Related Information

Updated: Aug 30, 2005
Document ID: 23797