Based on the (prestandard) IEEE 802.1Qbh port extender technology, Cisco Virtual Machine Fabric Extender (VM-FEX) extends the fabric from the switch chassis to the Virtual Machine (VM). Each VM is associated with a network adapter vNIC, which is associated with a virtual Ethernet (vEthernet or vEth) port on the parent switch. This dedicated virtual interface can be managed, monitored, and spanned in the same way as a physical interface. Local switching in the hypervisor is eliminated, with all switching being performed by the physical switch.
VM-FEX Components
Server
VM-FEX is supported by Cisco UCS C-Series rack-mount servers with the VMware virtualization environment as the hypervisor.
The configuration of the server is performed using the Cisco Integrated Management Controller (CIMC) interface, which provides both a GUI and a CLI interface. The configuration of the hypervisor and virtualization services is performed using the VMware vSphere client.
For information about CIMC and VM-FEX configuration, see the following documents:
Cisco UCS Manager VM-FEX for VMware GUI Configuration Guide
Virtual Interface Card Adapter
VM-FEX is supported by the Cisco UCS P81E Virtual Interface Card (VIC), a dual-port 10 Gigabit Ethernet PCIe adapter that supports static or dynamic virtualized interfaces, including up to 128 virtual network interface cards (vNICs).
The configuration of the VIC and its vNICs is performed using the CIMC interface on the Cisco UCS C-Series servers.
FEX
The physical ports of the server can be connected directly to the switch or to a fabric extender (FEX) connected to the switch. VM-FEX is supported by the Cisco Nexus Fabric Extender.
VM-FEX and AFEX require that the FEX is connected with a fabric PO and not individual links.
Switch
VM-FEX is supported by the Cisco Nexus device. Although a single switch chassis can be connected with VM-FEX, a typical application uses a pair of switches deployed as a virtual port channel (vPC) domain.
On the switch, a vEthernet interface represents the vNIC. All operations performed by the network administrator are performed on the vEthernet interface.
VM-FEX Terminology
The following terms are used in describing VM-FEX components and interfaces:
virtual Ethernet interface
A virtual Ethernet interface (vEthernet or vEth) represents the switch port that is connected to the vNIC of a virtual machine. Unlike a traditional switch interface, a vEth interface's name does not indicate the module with which the port is associated. Where a traditional physical switch port is specified as GigX/Y, where X is the module number and Y is the port number on the module, a vEth interface is specified as vEthY. This notation allows the interface to keep the same name when the VM migrates to another physical server.
dynamic interface
A dynamic interface is a vEthernet interface that is configured automatically as a result of adapter and switch communications. The provisioning model of a dynamic interface consists of the configuration on the switch of a vEthernet port profile, which is propagated to the network adapter as a port group, followed by the association of the port group with the vNIC. The port profile is created in the switch by the network administrator, while the association with the vNIC is performed on the adapter by the server administrator.
static interface
A static interface is configured manually on the switch and the adapter. A static virtual adapter can be a vNIC or a virtual host adapter bus (vHBA). A static interface can be a vEthernet or a virtual Fibre Channel (vFC) interface bound to a static vEthernet interface.
In one method of creating a static vEthernet, the network administrator assigns a channel number (equivalent to a VN-Tag or prestandard IEEE 802.1BR tag number) to the vEthernet. The server administrator must be sure to define a vNIC on the adapter with the same channel number.
In another method, the network administrator can create a static floating vEthernet by configuring the vEthernet with a virtual switching instance (VSI) MAC address and DVPort ID.
floating vEthernet interface
In a hypervisor environment, each vNIC on the network adapter is associated with one virtual machine (VM). VMs can migrate from one physical server to another. A virtual interface that migrates with a VM and virtual network link is called a floating vEthernet interface.
fixed vEthernet interface
A fixed vEthernet interface is a virtual interface that does not support migration across physical interfaces. For fixed vEthernet (static or dynamic), an administrator can change configurations at any time. The binding of the vEthernet interface number to a channel number is persistent unless the administrator changes it.
Licensing Requirements for VM-FEX
The following table shows the licensing requirements for this feature:
Product
License Requirement
Cisco NX-OS
A VM-FEX license is required for each Cisco Nexus device. The license package name is VMFEX_ FEATURE_PKG. A grace period of 120 days starts when you first configure the licensed feature.
For a complete explanation of the Cisco NX-OS licensing scheme and how to obtain and apply licenses, see the Cisco NX-OS Licensing Guide.
Default Settings for VM-FEX
The following table lists the default settings for parameters that are relevant to VM-FEX:
Parameters
Default
Virtualization feature set
Disabled
FEX
Disabled
VM-FEX
Disabled
LLDP
Enabled
vPC
Disabled
svs vethernet auto-setup
Enabled
FCoE
Disabled
Configuring VM-FEX
Overview of the VM-FEX Configuration Steps
The following steps outline the necessary sequence of procedures for configuring VM-FEX between the switch and the server hosting the VMs. Procedures to be performed on the switch are described in this document. For procedures to be performed on the server or the VMware vCenter, refer to the server and vCenter documentation.
Procedure
Step 1
Server: Create vNICs on VIC adapter.
Create two static vNICs to be used as uplinks from the host.
Create up to 112 VM-FEX interfaces.
Reboot the server.
Step 2
Switch: Enable VM-FEX and other required services.
Server: Verify that the interfaces are active and assigned to the VMs.
Step 14
vCenter: Verify that the dynamic vNICs are active.
Enabling Features Required for VM-FEX
Procedure
Command or Action
Purpose
Step 1
switch# configure terminal
Enters global configuration mode.
Step 2
installfeature-setvirtualization
Installs the virtualization feature set on the switch.
Step 3
feature-setvirtualization
Enables the virtualization feature set on the switch.
This feature set enables the use of static vEthernet interfaces.
Step 4
featurefex
Enables FEX features on the switch.
Step 5
featurevmfex
Enables VM-FEX features on the switch. This feature set enables the use of dynamic vEthernet interfaces.
Step 6
featurevpc
Enables a virtual port channel (vPC) on the switch.
Step 7
vethernetauto-create
(Optional)
Globally enables the automatic creation of virtual Ethernet interfaces. This feature is not required if the fixed vEthernet interfaces are statically configured.
Step 8
featurefcoe
(Optional)
Enables Fibre Channel over Ethernet (FCoE) on the switch.
Step 9
end
(Optional)
Return to privileged EXEC mode.
Step 10
copyrunning-configstartup-config
(Optional)
Copies the running configuration to the startup configuration.
Step 11
reload
(Optional)
Reloads the switch.
This example shows how to enable the features required for VM-FEX:
You can configure two physical interfaces and binds two virtual interfaces to each physical interface, creating fixed static vEthernet interfaces. For more information on configuring fixed static interfaces, see the Adapter-FEX Configuration Guide for your device.
With redundant switches, you can perform the following procedure with identical settings on both the primary and secondary switches.
Before You Begin
VM-FEX and other required services must be enabled on the switches.
Two static vNICs must be configured on the VIC adapter installed in the host server.
Procedure
Command or Action
Purpose
Step 1
switch# configure terminal
Enters global configuration mode.
Step 2
interfaceethernetslot/port
Enters interface configuration mode for the first Ethernet port.
Note
If this is a 10G breakout port, the slot/port syntax is slot/QSFP-module/port.
Step 3
shutdown
Disables local traffic on the interface.
Note
Shutting down the interface before enabling VN-Tag mode prevents the dynamic creation of a fixed vEthernet interface.
Step 4
switchportmodevntag
Enables port extender support on the interface.
Step 5
interfaceethernetslot/port
Enters interface configuration mode for the second Ethernet port.
Step 6
shutdown
Disables local traffic on the interface.
Step 7
switchportmodevntag
Enables port extender support on the interface.
Step 8
interfacevethernetinterface-number
Enters configuration mode for the first virtual interface for the first Ethernet port.
Verify the status of the connection between the static interfaces and the static vNICs on the host server.
Configuring a Port Profile for the Dynamic Interfaces
You can configure a port profile for dynamic virtual interfaces. This port profile is exported to the VMware vCenter distributed virtual switch (DVS) as a port-group.
With redundant switches, you can perform the following procedure with identical settings on both the primary and secondary switches.
Before You Begin
Dynamic vNICs must be configured on the VIC adapter installed in the host server.
The VLAN specified in the port profile must be created.
Procedure
Command or Action
Purpose
Step 1
switch# configure terminal
Enters global configuration mode.
Step 2
port-profiletypevethernetprofilename
Enters configuration mode for the specified port profile, creating it if necessary.
Step 3
switchportmodeaccess
(Optional)
Configures the interface to be in access mode.
Step 4
switchportaccessvlanvlan-id
(Optional)
Specifies the VLAN when the interface is in access mode.
Step 5
dvs-name {all | name}
Specifies the vCenter DVS to which the port profile is exported as a port-group. With the keyword all, the port profile is exported to all DVSs in the vCenter.
Step 6
port-bindingdynamic
(Optional)
Specifies dynamic port binding. The port is connected when the VM is powered on and disconnected when the VM is powered off. Max-port limits are enforced. The default is static port binding.
Step 7
stateenabled
Enables the port profile.
This example configures a port profile for dynamic virtual interfaces:
switch-1# configure terminal
switch-1(config)# port-profile type vethernet vm-fex-vlan-60
switch-1(config-port-prof)# switchport mode access
switch-1(config-port-prof)# switchport access vlan 60
switch-1(config-port-prof)# dvs-name all
switch-1(config-port-prof)# port-binding dynamic
switch-1(config-port-prof)# state enabled
switch-1(config-port-prof)#
Configuring an SVS Connection to the vCenter Server
You can configure a secure connection from the switch to the vCenter Server.
With redundant switches, perform this procedure on both the primary and the secondary switches. In normal operation, only the primary switch connects to the vCenter, with the secondary switch connecting only upon a failure of the primary.
Procedure
Command or Action
Purpose
Step 1
switch# configure terminal
Enters global configuration mode.
Step 2
svsconnectionsvs-name
Enables and enters configuration mode for an SVS connection from the switch to the vCenter Server.
Step 3
protocolvmware-vim
Enables the VMware Infrastructure Software Development Kit (VI SDK), which allows clients to communicate with the vCenter.
Step 4
vmwaredvsdatacenter-namedc-name
Creates a VMware distributed virtual switch (DVS) in the specified datacenter.
Step 5
dvs-namedvs-name
Configures a name for the DVS in the vCenter Server.
Installs a certificate that is used to connect to the vCenter Server.
The server argument specifies the boot flash memory location to install the certificate. The argument value can be module-1, sup-1, sup-active, or sup-local.
Step 8
extension-key:extn-ID
Configures the extension key to be used to connect to the vCenter Server.
Note
With redundant switches, perform this step only on the primary switch. The key is automatically synchronized with the secondary switch.
This example shows how to configure the SVS connection on the primary switch and the secondary switch:
Use the following commands to display status information for virtual interfaces.
Command
Purpose
showinterfacevethernetinterface-number [detail]
Displays the status of the virtual interface.
Perform this procedure on each static virtual interface to verify that the interface is active and bound to the physical interface.
showinterfacevirtualstatusvm-fex
Displays information about all floating virtual interfaces.
showinterfacevirtualsummaryvm-fex
Displays summary information about virtual Ethernet interfaces.