The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter describes how to troubleshoot Cisco Virtual Security Gateway (VSG) system issues.
The Cisco VSG provides firewall functionality for the VMs that have the vEths with port profiles created by the Virtual Supervisor Module (VSM). To allow the Cisco VSG to function properly, the Cisco VSG should have registered with a Cisco Virtual Network Management Center (VNMC) and the Cisco VSG data interface MAC address should be seen by the VSM.
The example shows how to display information about the system:
Make sure that the Cisco VSG MAC address is learned by the VSM by entering the show vsn details command as follows:
For more information, see the following documents:
When troubleshooting problems with intra-host VM traffic, follow these guidelines:
When troubleshooting problems with inter-host VM traffic, follow these guidelines:
This section includes the following topics:
Use the following commands to display Virtual Ethernet Module (VEM) information:
These commands provide additional VEM details:
This example shows how to display the number of entries in the circular buffer:
This example shows how to display log entries:
This example shows how to display the vemcmd help:
Use the following commands to control the vemlog:
You can display the list of debug filters by entering the vemlog show debug | grp vpath command.
This example shows how to display the list of debug filters:
This section includes the following topics:
This section includes the following topics:
Cisco VSG with a VN Service vmknic in Layer 3 Mode
Cisco VSGs with Multiple l3-vn-service vmknics in Layer 3 Mode
When encapsulated traffic that is destined to a Cisco VSG is connected to a different subnet other than the vmknic subnet, the VEM does not use the VMware host routing table. Instead, the vmknic initiates an Address Resolution Protocol (ARP) for the remote Cisco VSG IP addresses.
You must configure the upstream router to respond by using the proxy ARP feature. If the proxy ARP feature is not configured on the upstream router, the ARP fails and the show vsn brief indicates that the service node state is down.
To resolve this issue configure the proxy ARP feature on the router as follows:
sg-cat3k-L14-qa(config)
# int vlan 3756
The data path traffic and the ARP packets for the Cisco VSGs in Layer 3 mode can use any vmknic that is configured on the VEM host for packet forwarding to the Cisco VSG when you enter the capability l3-vn-service command.
Therefore, all vmknics that are on a VEM host must be able to reach all Cisco VSGs in Layer 3 mode.
If a router is between the vmknics and the Cisco VSGs, all vmknics must have an interface in the router network (VLAN), and all the Cisco VSGs in the Layer 3 mode must have an interface in the router network (VLAN) to ensure that each vmknic has a route to each Cisco VSG.
To resolve this issue ensure that all l3-vn-service vmknics can reach all the Cisco VSGs in the Layer 3 mode that are used by the VEM host.
Note You must enable Proxy ARP on all vmknics that facie interfaces on the router.
If a router lies between the vmknic and the Cisco VSG in the Layer 3 mode, and the router receives a packet that it cannot forward due to a large packet size, the router generates an ICMP Too Big message for the vmknic. The vmknic cannot forward the ICMP Too Big message of the router to the client and the vmknic drops the message. The client never receives the ICMP Too Big message and cannot refragment the packet for successful end-to-end traffic and the end-to-end traffic fails. This problem is typically seen if the router interface to the VEM is set at a higher maximun transmission unit (MTU) than the router interface to the Cisco VSG. For example, the router interface to the VEM has an MTU of 1600 and the interface to the Cisco VSG has an MTU of 1500.
This problem can be seen as an increase in the ICMP Too Big Rcvd counter in the show vsn statistics command.
To resolve this issue, configure an oversized MTU (for example, 1600) on both of the router interfaces.
When the VEM communicates with the Cisco VSG in the Layer 3 mode, an additional header with 94 bytes is added to the original packet. The VEM does not support fragmentation in the Layer 3 mode and the ports or network elements (which carry a vPath encapsulated packet) must be configured in such a way that the vPath overhead is accommodated.
If end-to-end traffic fails with a Cisco VSG Layer 3 mode, set the uplink MTU to 1594 bytes to accommodate the additional overhead. This solution assumes that the client and server VM MTUs are at the default of 1500 bytes.
Traffic with the Layer 3 encapsulation fails even with the uplink MTU set to 9000 bytes.
If jumbo frames are enabled in the network and the end-to-end traffic fails, make sure that the MTU of the client and server VMs are 94 bytes smaller than the uplink. For example, if the uplink MTU is 9000, set the MTU of the client and server VMs to 8906 bytes.
By default, TCP state checks are disabled in Cisco vPath for the traffic protected by the Cisco VSG. Sometimes, you might see delays in the TCP traffic. To diagnose TCP state checks related issues, you need to enable TCP state checks.
Check the following counters at the VSM in show vsn statistics output:
VSM #
show vsn statistics vpath | grep "TCP chkfail"
TCP chkfail InvalACK 0 TCP chkfail SeqPstWnd 0
This example shows how to enable the TCP state checks on a VSM:
The Cisco VSG can have up to 256,000 active connections at any given point of time. If for some reason new connections slows down or connections see too many failures, you can check the Cisco VSG for any connection limits that it experiences. If the VEM-to-Cisco VSG connection is not smooth or have some issues that indicates that the Cisco VSG might have missed a few updates from vPath which results in an accumulation of large active connections in its flow table.
This example shows how to check the active connection count on the Cisco VSG:
VSG#
show service-path statistics | inc "Active Connections"