Configuring FCoE VLANs and Virtual Interfaces

This chapter contains the following sections:

Information About Virtual Interfaces

Cisco Nexus devices support Fibre Channel over Ethernet (FCoE), which allows Fibre Channel and Ethernet traffic to be carried on the same physical Ethernet connection between the switch and the servers.

The Fibre Channel portion of FCoE is configured as a virtual Fibre Channel interface. Logical Fibre Channel features (such as interface mode) can be configured on virtual Fibre Channel interfaces.

A virtual Fibre Channel interface must be bound to an interface before it can be used. The binding is to a physical Ethernet interface (when the converged network adapter (CNA) is directly connected to the Cisco Nexus device), a MAC address (when the CNA is remotely connected over a Layer 2 bridge), or an EtherChannel when the CNA connects to the Fibre Channel Forwarder (FCF) over a virtual port channel (vPC).

Information About Shutting Down LAN Traffic

Converged Network Adapters (CNA) enable both FCoE and LAN traffic (Unified I/O) to co-exist over a physical link.

In vPC configurations with CNAs, network parameters need to be consistent across peer switches. If the system detects an inconsistency, the secondary vPC leg goes down. Since vPC legs carry both FCOE and LAN traffic, the FCoE link goes down also.

To avoid having the FCoE link go down in this situation, you can use the shutdown lan command to shutdown only the LAN traffic on port-channels and individual Ethernet ports.


Note

When vPC triggers the vPC secondary leg to be brought down, only the Ethernet VLANs are brought down for the secondary vPC leg. FCoE/storage VLANs of the secondary vPC leg remain up.

Notes About the shutdown lan Command

  • The shutdown lan command is only configurable on port-channel interfaces, FEX HIF ports, or on individual Ethernet interfaces that vFC interfaces are bound upon.

  • The shutdown lan command is only configurable on port-channel interfaces or on individual Ethernet interfaces that are in an operational trunking state.

  • The shutdown lan command cannot be enabled on the secondary vPC leg, if the vPC enabled shutdown lan is applied on the secondary vPC leg.

  • A vPC enabled shutdown LAN is not operable if the shutdown lan command is applied on the secondary vPC leg.

  • The shutdown lan command is not configurable on port-channel members.

  • The shutdown lan command default is no shutdown lan (shutdown lan is disabled).

  • The shutdown lan command has a prerequisite that the Link Layer Discovery Protocol (LLDP) feature be enabled.

  • A port with a shutdown LAN configuration enabled cannot be added to a port channel.

  • The shutdown LAN enable/disable configuration is on a per interface basis.

  • If a shutdown lan is configured on an interface, a no shut command on the interface does not bring up LAN VLANs.

  • A shutdown LAN is triggered when a Type-1 inconsistency occurs in a VPC network.

Examples of Shutdown LAN Traffic

  • Shutdown the LAN traffic on port-channel. 

    
switch(config)#interface port-channel 955
switch(config-if)# shutdown lan

  • Shutdown the LAN traffic on individual Ethernet port. 

    
switch(config)#interface Ethernet 2/5
switch(config-if)# shutdown lan

Examples of Verifying Shutdown LAN Traffic

  • Verifying when the shutdown lan command is issued on port-channel 955 with Ethernet interface 2/5 as member. 

     
switch# sh interface port-channel 955 | grep LAN
All LAN VLANs are administratively shut 

switch# sh interface ethernet 2/5 | grep LAN 
All LAN VLANs are administratively shut 

switch# sh run interface port-channel 955 | grep shut 
shutdown lan 

switch# sh run interface e2/5 | grep shut 
shutdown lan

  • Verifying when the vPC triggers shutdown LAN on the secondary vPC leg (port-channel 231 with Ethernet 2/31 as member). 

    
switch# sh interface port-channel 231 | grep LAN 
All LAN VLANs are administratively shut

Guidelines and Limitations for FCoE VLANs and Virtual Interfaces

FCoE VLANs and Virtual Fiber Channel (vFC) interfaces have these guidelines and limitations:

  • Each vFC interface must be bound to an FCoE-enabled Ethernet or EtherChannel interface or to the MAC address of a remotely connected adapter. FCoE is supported on 10-Gigabit and 40-Gigabit Ethernet interfaces.

    The Ethernet or EtherChannel interface that you bind to the vFC interface must be configured as follows:

    • The Ethernet or EtherChannel interface must be a trunk port (use the switchport mode trunk command).

    • The FCoE VLAN that corresponds to a vFC’s VSAN must be in the allowed VLAN list.

    • You must not configure an FCoE VLAN as the native VLAN of the trunk port.


      Note

      The native VLAN is the default VLAN on a trunk. Any untagged frames transit the trunk as native VLAN traffic.

    • You should use an FCoE VLAN only for FCoE.

    • Do not use the default VLAN, VLAN1, as an FCoE VLAN.

    • You must configure the Ethernet interface as PortFast (use the spanning-tree port type edge trunk command).


      Note

      You are not required to configure trunking on the server interface even if the switch interface is configured with trunking enabled. All non-FCoE traffic from the server is passed on the native VLAN.

  • The vFC interface cannot be bound to Ethernet port channel with multiple member ports connected to FCoE Initialization Protocol (FIP) snooping bridges. It is recommended to use Mac bound vFC when hosts are connected via snooping bridges.

  • For VF mode, each vFC interface is associated with only one VSAN.

    For VNP mode, each vFC interface is associated with multiple VSANs.

  • You must map any VSAN with associated vFC interfaces to a dedicated FCoE-enabled VLAN.

  • FCoE is not supported on private VLANs.

  • If the converged access switches (in the same SAN fabric or in another) need to be connected to each other over Ethernet links for a LAN alternate path, then you must explicitly configure such links to exclude all FCoE VLANs from membership.

  • You must use separate FCoE VLANs for FCoE in SAN-A and SAN-B fabrics.

  • FCoE connectivity to pre-FIP CNAs over virtual port channels (vPCs) is not supported.

  • FCoE VLANs do not support Multiple Spanning Trees (MST). Creating an MST instance for an FCoE VLAN might cause SAN traffic disruption.


Note

Virtual interfaces are created with the administrative state set to down. You must explicitly configure the administrative state to bring the virtual interface into operation.

Configuring Virtual Interfaces

Mapping a VSAN to a VLAN

A unique, dedicated VLAN must be configured at every converged access switch to carry traffic for each VSAN in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2, and so on).

Procedure

  Command or Action Purpose
Step 1

switch# configure terminal

Enters global configuration mode.

Step 2

switch(config)# vlan vlan-id

Enters VLAN configuration mode. The VLAN number range is from 1 to 4096.

Step 3

switch(config-vlan)# fcoe [vsan vsan-id]

Enables FCoE for the specified VLAN. If you do not specify a VSAN number, a mapping is created from this VLAN to the VSAN with the same number.

Configures the mapping from this VLAN to the specified VSAN.

Step 4

switch(config-vlan)# exit

Exits VLAN configuration mode. You must exit this mode to execute the configured commands on your Cisco Nexus device.

Step 5

(Optional) switch(config-vlan)# show vlan fcoe

(Optional)

Displays information about the FCoE configuration for a VLAN.

Step 6

(Optional) switch(config)# copy running-config startup-config

(Optional)

Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

Example

This example shows how to map VLAN 200 to VSAN 2: 

switch(config)# vlan 200
switch(config-vlan)# fcoe vsan 2

Creating a Virtual Fibre Channel Interface

You can create a virtual Fibre Channel interface. You must bind the virtual Fibre Channel interface to a host MAC address before it can be used.

Procedure

  Command or Action Purpose
Step 1

switch# configure terminal

Enters global configuration mode.

Step 2

switch(config)# interface vfc vfc-id

Creates a virtual Fibre Channel interface (if it does not already exist) and enters interface configuration mode.

The virtual Fibre Channel interface ID range is from 1 to 8192.

Step 3

switch(config-if)# bind {interface {ethernet slot/port | port-channel channel-number} | mac-address MAC-address}

Binds the virtual Fibre Channel interface to the specified interface.

Note 

If this is a 10G breakout port, the slot/port syntax is slot/QSFP-module/port .

Step 4

(Optional) switch(config-if)# no bind {interface {ethernet slot/port | port-channel channel-number} | mac-address MAC-address}

(Optional)

Unbinds the virtual Fibre Channel interface from the specified interface.

Note 

If this is a 10G breakout port, the slot/port syntax is slot/QSFP-module/port .

Step 5

(Optional) switch(config)# no interface vfc vfc-id

(Optional)

Deletes a virtual Fibre Channel interface.

Example

  • This example shows how to bind a virtual Fibre Channel interface to an Ethernet interface: 

    switch# configure terminal
switch(config)# interface vfc 4
switch(config-if)# bind interface ethernet 1/4

  • This example shows how to bind a virtual Fibre Channel interface to a port-channel: 

    switch# configure terminal
switch(config)# interface vfc 3
switch(config-if)# bind interface port-channel 1

  • This example shows how to bind a virtual Fibre Channel interface to a MAC address: 

    switch# configure terminal
switch(config)# interface vfc 2
switch(config-if)# bind mac-address 00:0a:00:00:00:36

    Note

    A MAC bound vFC can be configured to a host sitting behind a FSB.


    Note

    When both MAC bound vFC and port-bound vFC are configured for the same interface, the port-bound vFC takes precedence.


    Note

    As a best practice, you should have either a MAC bound vFC or a port- bound vFC for a physical ethernet port or a port-channel, but not have both.

  • This example shows how to delete a virtual Fibre Channel interface: 

    switch# configure terminal
switch(config)# no interface vfc 4

Associating a Virtual Fibre Channel Interface to a VSAN

A unique, dedicated VLAN must be configured at every converged access switch to carry traffic for each Virtual Fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2, and so on).

Procedure

  Command or Action Purpose
Step 1

switch# configure terminal

Enters global configuration mode.

Step 2

switch(config)# vsan database

Enters VSAN configuration mode.

Step 3

switch(config-vsan)# vsan vsan-id interface vfc vfc-id

Configures the association between the VSAN and virtual Fibre Channel interface.

The VSAN number must map to a VLAN on the physical Ethernet interface that is bound to the virtual Fibre Channel interface.

Step 4

(Optional) switch(config-vsan)# no vsan vsan-id interface vfc vfc-id

(Optional)

Disassociates the connection between the VSAN and virtual Fibre Channel interface.

Example

This example shows how to associate a virtual Fibre Channel interface to a VSAN: 

switch# configure terminal
switch(config)# vsan database
switch(config-vsan)# vsan 2 interface vfc 4

Creating an Implicit Virtual Fibre Channel Port Channel Interface

You can create a virtual Fibre Channel (vFC), and implicitly bind it to an Ethernet interface or a port-channel using a single command. For this, the vFC identifier must match the Ethernet interface or port-channel identifier. The Ethernet interface can be a module (slot or port).

Configuring virtual Fibre Channel Interface

Before you begin

  • Ensure you have installed the correct license for FCoE.

  • Ensure you have enabled FCoE.

Procedure

Step 1

switch# configure terminal

Enters global configuration mode.
Step 2

switch(config) # interface vfc {id | slot/port | chassis/slot/port

Create a vFC (if it does not already exist).
Step 3

switch(config-if) # no shutdown

Bring up the vFC interface.

Step 4

Exit the interface configuration mode:

switch(config-if) # exit

Step 5

Enters VSAN configuration mode.

switch(config)#vsan database

Step 6

switch(config-vsan)# vsan vsan-id interface vfc vfc-id

Assigns the membership of the specified interface to the VSAN.

Configuring virtual Fibre Channel Interface

This example shows how to implicitly bind a virtual Fibre Channel interface to an Ethernet interface: 


switch# configure terminal
switch(config)# interface vfc 1/1/1
switch(config-if)# no shutdown

switch(config)# vsan database 
switch(config-vsan)# vsan 2 interface vfc 4 

Switch# show interface vfc 1/1/1
    vfc101/1/1 is trunking
    Bound interface is Ethernet1/1/1
    Hardware is Ethernet 
    Port WWN is 20:00:00:2a:6a:15:d2:7b
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 600
    Trunk vsans (admin allowed and active) (1,500,600)
    Trunk vsans (up)                       (600)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1,500)
    1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
    1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
      49 frames input, 5516 bytes
        0 discards, 0 errors
      49 frames output, 5772 bytes
        0 discards, 0 errors
    last clearing of "show interface" counters Fri Oct 30 06:19:33 2015
    Interface last changed at Fri Oct 30 06:19:33 2015

Configuring virtual Fibre Channel – Port Channel Interface

Procedure

Step 1

Enter global configuration mode:

switch# configure terminal

Step 2

Create a vFC that implicitly binds to the Ethernet port-channel based on its number:

The port number range is from 1 to 4096.

switch(config) # interface vfc-port-channel port number

Step 3

Bring up the vFC port:

switch(config-if) # no shutdown

Step 4

Exit from the current interface configuration mode:

switch(config-if) # exit

Step 5

Enters VSAN configuration mode.

switch(config)#vsan database

Step 6

switch(config-vsan)# vsan vsan-id interface vfc vfc-id

Assigns the membership of the specified interface to the VSAN.

Configuring virtual Fibre Channel - Port Channel Interface

The example shows how you can create a vFC-port-channel that implicitly binds to Ethernet port-channel: 

switch# configure terminal
switch(config)# interface vfc-port-channel 300
switch(config-if)# no shutdown

switch(config)# vsan database 
switch(config-vsan)# vsan 2 interface vfc 4

switch# show interface vfc-port-channel 258

    vfc-po258 is trunking
    Bound interface is port-channel258
    Hardware is Ethernet 
    Port WWN is 21:01:8c:60:4f:59:31:3f
    Admin port mode is E, trunk mode is on
    snmp link state traps are enabled
    Port mode is TE
    Port vsan is 600
    Trunk vsans (admin allowed and active) (1,100,500,600)
    Trunk vsans (up)                       (600)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1,100,500)
    1 minute input rate 3448 bits/sec, 431 bytes/sec, 4 frames/sec
    1 minute output rate 9064 bits/sec, 1133 bytes/sec, 4 frames/sec
      977735 frames input, 77172556 bytes
        0 discards, 0 errors
      977733 frames output, 205924892 bytes
        0 discards, 0 errors
    last clearing of "show interface" counters Thu Oct 29 06:35:41 2015
    Interface last changed at Thu Oct 29 06:35:41 2015

Verifying the Virtual Interface

To display configuration information about virtual interfaces, perform one of the following tasks:

Command

Purpose

switch# show interface vfc vfc-id

Displays the detailed configuration of the specified Fibre Channel interface.

switch# show interface brief

Displays the status of all interfaces.

switch# show vlan fcoe

Displays the mapping of FCoE VLANs to VSANs.

This example shows how to display a virtual Fibre Channel interface bound to an Ethernet interface: 


switch(config-if)# sh int vfc 172

vfc172 is trunking (Not all VSANs UP on the trunk)
    Bound interface is Ethernet1/72
    Hardware is Ethernet 
    Port WWN is 20:ab:e0:0e:da:4a:5d:9d
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 200
    Speed is auto
    Trunk vsans (admin allowed and active) (1,10,100,200)
    Trunk vsans (up)                       (200)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1,10,100)
    799 fcoe in packets 
    80220 fcoe in octets
    2199 fcoe out packets
    2219828 fcoe out octets
    Interface last changed at Thu Sep 15 08:52:51 2016

This example shows how to display a virtual Fibre Channel interface bound to a MAC address: 


switch(config-if)# sh int vfc 132

vfc132 is trunking (Not all VSANs UP on the trunk)
    Bound MAC is 000e.1e1b.c1c9
    Hardware is Ethernet 
    Port WWN is 20:83:00:2a:10:7a:89:bf
    Admin port mode is F, trunk mode is on
    snmp link state traps are enabled
    Port mode is TF
    Port vsan is 2101
    Speed is auto
    Trunk vsans (admin allowed and active) (1,2001-2003,2101-2103)
    Trunk vsans (up)                       (2101)
    Trunk vsans (isolated)                 ()
    Trunk vsans (initializing)             (1,2001-2003,2102-2103)
    Interface last changed at Wed Sep 14 12:14:29 2016

This example shows how to display the status of all the interfaces on the switch (some output has been removed for brevity): 

switch# show interface brief
-------------------------------------------------------------------------------
Interface  Vsan   Admin  Admin   Status          SFP    Oper  Oper   Port
                  Mode   Trunk                          Mode  Speed  Channel
                         Mode                                 (Gbps)
-------------------------------------------------------------------------------
fc3/1      1      auto   on      trunking         swl    TE      2    --
fc3/2      1      auto   on      sfpAbsent        --     --           --
...
fc3/8      1      auto   on      sfpAbsent        --     --           --
-------------------------------------------------------------------------------
Interface                Status     IP Address        Speed    MTU   Port
                                                                     Channel
-------------------------------------------------------------------------------
Ethernet1/1              hwFailure  --                --       1500  --
Ethernet1/2              hwFailure  --                --       1500  --
Ethernet1/3              up         --                10000    1500  --
...
Ethernet1/39             sfpIsAbsen --                --       1500  --
Ethernet1/40             sfpIsAbsen --                --       1500  --
-------------------------------------------------------------------------------
Interface                Status     IP Address        Speed    MTU
-------------------------------------------------------------------------------
mgmt0                    up         172.16.24.41      100      1500
-------------------------------------------------------------------------------
 
-------------------------------------------------------------------------------
Interface  Vsan   Admin  Admin   Status          SFP    Oper  Oper   Port
                  Mode   Trunk                          Mode  Speed  Channel
                         Mode                                 (Gbps)
-------------------------------------------------------------------------------
vfc 1        1      F      --      down             --     --           --
...

This example shows how to display the mapping between the VLANs and VSANs on the switch: 

switch# show vlan fcoe
VLAN      VSAN      Status
--------  --------  --------
15        15        Operational
20        20        Operational
25        25        Operational
30        30        Non-operational

Mapping VSANs to VLANs Example Configuration

The following example shows how to configure the FCoE VLAN and a virtual Fibre Channel interface:

Procedure

Step 1

Enable the associated VLAN and map the VLAN to a VSAN. 

switch(config)# vlan 200
switch(config-vlan)# fcoe vsan 2
switch(config-vlan)# exit
Step 2

Configure the VLAN on a physical Ethernet interface. 

switch# configure terminal
switch(config)# interface ethernet 1/4
switch(config-if)# spanning-tree port type edge trunk
switch(config-if)# switchport mode trunk
switch(config-if)# switchport trunk allowed vlan 1,200
switch(config-if)# exit
Step 3

Create a virtual Fibre Channel interface and bind it to a physical Ethernet interface. 

switch(config)# interface vfc 4
switch(config-if)# bind interface ethernet 1/4
switch(config-if)# exit
Note 

By default, all virtual Fibre Channel interfaces reside on VSAN 1. If the VLAN to VSAN mapping is to a VSAN other than VSAN 1, then proceed to Step 4.

Step 4

Associate the virtual Fibre Channel interface to the VSAN. 

switch(config)# vsan database
switch(config-vsan)# vsan 2 interface vfc 4
switch(config-vsan)# exit
Step 5

(Optional) Display membership information for the VSAN. 

switch# show vsan 2 membership
vsan 2 interfaces
           vfc 4
Step 6

(Optional) Display the interface information for the virtual Fibre Channel interface. 

switch# show interface vfc 4

vfc4 is up
Bound interface is Ethernet1/4
Hardware is Virtual Fibre Channel
Port WWN is 20:02:00:0d:ec:6d:95:3f 
Port WWN is 20:02:00:0d:ec:6d:95:3f 
snmp link state traps are enabled
Port WWN is 20:02:00:0d:ec:6d:95:3f 
APort WWN is 20:02:00:0d:ec:6d:95:3f
snmp link state traps are enabled
Port mode is F, FCID is 0x490100
Port vsan is 931
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec 
1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes 0 discards, 0 errors 
0 frames output, 0 bytes 0 discards, 0 errors
Interface last changed at Thu Mar 11 04:44:42 2010

SAN Boot with vPC

Cisco Nexus 9000 Series devices support the SAN boot of initiators on Link Aggregation Control Protocol (LACP) based vPC. This limitation is specific to LACP-based port channels. The host-facing vFC interfaces are bound to port channel members instead of the port channel itself. This binding ensures that the host-side vFC comes up during a SAN boot as soon as the link on the CNA/Host Bus Adapter (HBA) comes up, without relying on the LACP-based port channel to form first.


Note

Cisco Nexus 9000 Series devices support the SAN boot of channel mode on also.


Note

LACP suspend-individual should be removed from the port-channel, otherwise the physical interface will be suspended when LACP BPDU is not received from the host.

SAN Boot with vPC Configuration Example

In this example, virtual Fibre Channel interface 1 is bound to physical Ethernet interface 1/2/4 on Fabric A and on interface 1/2/3 on Fabric B. The interface is also associated with virtual port channel 1 on both fabrics. 


switch1(config)# interface port-channel 1
switch1(config-if)# no lacp suspend-individual
switch1(config)# interface vfc 1
switch1(config-if)# bind interface eth 1/2/4
switch1(config)# interface eth 1/2/4
switch1(config-if)# channel-group 1 mode active
switch1(config-if)# interface port-channel 1
switch1(config-if)# vpc 1
switch1(config-if)#

switch2(config)# interface port-channel 1
switch2(config-if)# no lacp suspend-individual
switch2(config)# interface vfc 1
switch2(config-if)# bind interface eth 1/2/3
switch2(config)# interface eth 1/2/3
switch2(config-if)# channel-group 1 mode active
switch2(config-if)# interface port-channel 1
switch2(config-if)# vpc 1