Guest

Cisco Nexus 5000 Series Switches

Nexus 5500 to Nexus 7000 Multi-Hop FCoE Configuration Example

Document ID: 116298

Updated: Nov 25, 2013

Contributed by Carlo Schmidt, Ozden Karakok, and William Douglas, Cisco TAC Engineers.

   Print

Introduction

This document describes Multi-Hop Fibre Channel over Ethernet (FCoE) virtual E (VE)-port-to-VE-port configuration between Nexus 7000 and Nexus 5500 switches.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Configure

In order to begin this process, you must verify that:

  • The Nexus 7000 has the FCoE feature enabled.
  • The module F1 is licensed for FCoE.
  • The system Quality of Service (QoS) is set correctly.
  • The storage Virtual Device Contexts (VDC) are created.
  • The dedicated and shared interfaces are configured.

In order to configure the Nexus 7000 Switch for Multi-Hop FCoE, complete these steps.

  1. Enter these commands:
    default_vdc(config)# license fcoe module 1
    default_vdc(config)#system qos
    default_vdc(config-sys-qos)# service-policy type
    network-qos default-nq-7e-policy
    default_vdc(config)# install feature-set fcoe
    default_vdc(config)# vdc fcoe type storage
    default_vdc(config)# configure terminal
    default_vdc(config)# allocate interface
    ethernet1/31-32
     <-- For FCoE VE port
    connectivity, you NEED dedicated interfaces;
    they cannot be shared!

    default_vdc(config)# switchto vdc fcoe
  2. Configure the Ethernet and Virtual Fibre Channel (VFC) interface in the storage VDC. In order to configure the features needed for FCoE in the storage VDC (called FCoE in this example), enter:
    fcoe(config)# feature lacp
    fcoe(config)# feature-set fcoe
    fcoe(config)# feature lldp
  3. Create the virtual storage area network (VSAN) and map the FCoE VLAN to the VSAN. This example uses VLAN 2 and VSAN 2.
    fcoe(config)# vlan [VLAN#]
    fcoe(config-vlan)# exit
    fcoe(config)# vsan database
    fcoe(config-vsan-db)# vsan [VSAN#]
    fcoe(config-vsan-db)# exit
    fcoe(config)# vlan [VLAN#]
    fcoe(config-vlan)# fcoe vsan [VSAN#]
    fcoe(config-vlan)# exit
  4. In order to configure the Ethernet interface as trunk and enable the FCoE VLAN on the trunk, enter:
    fcoe(config)# interface eth1/31-32
    fcoe(config-if)# channel-group 2 force mode active
    fcoe(config-if)# int port-channel 2
    fcoe(config-if)# switchport mode trunk
    fcoe(config-if)# switchport trunk allowed vlan 2
    fcoe(config-if)# no shut

    Note: On Nexus 7000, by default the "source-destination-oxid" load balancing mechanism is used for FCoE traffic.

  5. In order to create the VFC interface and bind it to the Ethernet interface, and enable the VSAN on the trunked interface, enter:
    fcoe(config)# interface vfc-port-channel 2 <--  There is an implicit bind; 
    it will be automatically mapped to the port-channel created in the previous set
    .
    fcoe(config-if)# switchport mode E 
    fcoe(config-if)# switchport trunk allowed vsan 2
    fcoe(config-if)# no shut

In order to configure the Nexus 5500 switch for Multi-Hop FCoE, complete these steps.

  1. The first step on the Nexus 5000 switch is to enable the FCoE switching operation, and then ensure that LACP is enabled. The feature fcoe command is required in order to enable both Fibre Channel (FC) and FCoE on the Nexus 5500 switch.
    AwesomeN5k(config)# feature lacp
    AwesomeN5k(config)# feature fcoe
  2. Enable QoS for FCoE operation on the Nexus 5500. Four lines of QoS statements map the baseline system QoS policies for FCoE. Without these commands, the virtual FC interface does not function when activated.
    system qos
    service-policy type qos input fcoe-default-in-policy
    service-policy type queuing input fcoe-default-in-policy
    service-policy type queuing output fcoe-default-out-policy
    service-policy type network-qos fcoe-default-nq-policy
    end
  3. Create the VSAN and map the FCoE VLAN to the VSAN. This example uses VLAN 2 and VSAN 2.
    AwesomeN5k(config)# vlan [VLAN#]
    AwesomeN5k(config-vlan)# exit
    AwesomeN5k(config)# vsan database
    AwesomeN5k(config-vsan-db)# vsan [VSAN#]
    AwesomeN5k(config-vsan-db)# exit
    AwesomeN5k(config)# vlan [VLAN#]
    AwesomeN5k(config-vlan)# fcoe vsan [VSAN#]
    AwesomeN5k(config-vlan)# exit
  4. In order to configure the Ethernet interface as a trunk and to enable the FCoE VLAN on the trunk, enter:
    AwesomeN5k(config)# interface eth1/47-48
    AwesomeN5k(config-if)# channel-group 2 mode active
    AwesomeN5k(config-if)# int channel-group 2
    AwesomeN5k(config-if)# switchport mode trunk
    AwesomeN5k(config-if)# switchport trunk allowed vlan 2
    AwesomeN5k(config-if)# no shut

    Note: On Nexus 5000 the default load balancing mechanism on the LACP port-channel for FCoE traffic is "source-destination". In this default state, all the FCoE traffic takes the same link in the port-channel when the Nexus 5000 forwards frames over FCoE VE ports. In order to enable all links to be used in the port-channel for FCoE traffic, enter the port-channel load-balance ethernet source-dest-port command to configure Nexus 5000 "port-channel load balancing" to "source-dest-port". With this configuration "source-destination-oxid" load balancing is used for FCoE traffic.

  5. In order to create the VFC interface and bind it to the Ethernet interface, and enable the VSAN on the trunked interface, enter:
    AwesomeN5k(config)# interface vfc 2
    AwesomeN5k(config-if)# bind interface port-channel 2
    AwesomeN5k(config-if)# switchport mode E
    AwesomeN5k(config-if)# switchport trunk allowed vsan 20

The VFCs are now online!

Verify

There is currently no verification procedure available for this configuration.

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.

Updated: Nov 25, 2013
Document ID: 116298