Ports and Port Channels

Server and Uplink Ports

Each fabric interconnect can include the following port types:

Server Ports

Server ports handle data traffic between the fabric interconnect and the adapter cards on the servers.

Uplink Ethernet Ports

Uplink Ethernet ports handle Ethernet traffic between the fabric interconnect and the next layer of the network. All network-bound Ethernet traffic is pinned to one of these ports.

By default, Ethernet ports are unconfigured. However, you can configure them to function in the following ways:

  • Uplink

  • FCoE

  • Appliance

You can configure uplink Ethernet ports on either the fixed module or an expansion module.

Uplink Fibre Channel Ports

Uplink Fibre Channel ports handle FCoE traffic between the fabric interconnect and the next layer of the storage area network. All network-bound FCoE traffic is pinned to one of these ports.

By default, Fibre Channel ports are uplink. However, you can configure them to function as Fibre Channel storage ports. This is useful in cases where Cisco UCS requires a connection to a Direct-Attached Storage (DAS) device.

You can only configure uplink Fibre Channel ports on an expansion module. The fixed module does not include uplink Fibre Channel ports.

Unified Ports

Unified ports can be configured to carry either Ethernet or Fibre Channel traffic. These ports are not reserved. A Cisco UCS domain cannot use these ports until you configure them.

All ports on the following fabric interconnects are unified:

  • Cisco UCS 6248 UP Fabric Interconnect

  • Cisco UCS 6296 UP Fabric Interconnect

  • Cisco UCS 6324 Fabric Interconnect

  • Cisco UCS 6332-16UP Fabric Interconnect

  • Cisco UCS 6454 Fabric Interconnect

  • Cisco UCS 64108 Fabric Interconnect

  • Cisco UCS 6536 Fabric Interconnect


Note

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. If the port is connected to another device, this may cause traffic disruption. You can disable the port after configuring it.

Unified Storage Ports

Unified storage is configuring the same physical port as an Ethernet storage interface and FCoE storage interface. You can configure any appliance port or FCoE storage port as a unified storage port on either a fixed module or an expansion module. To configure a unified storage port, the fabric interconnect must be in Fibre Channel switching mode.

In a unified storage port, you can enable/disable individual FCoE storage or appliance interfaces.

  • In an unified storage port, if you do not specify a non default VLAN for the appliance port the fcoe-storage-native-vlan will be assigned as the native VLAN on the unified storage port. If the appliance port has a non default native VLAN specified as native VLAN, this will be assigned as the native VLAN for unified storage port.

  • When you enable or disable the appliance interface, the corresponding physical port is enabled/disabled. So when you disable the appliance interface in a unified storage, even if the FCoE storage is enabled, it goes down with the physical port.

  • When you enable or disable FCoE storage interface, the corresponding VFC is enabled or disabled. So when the FCoE storage interface is disabled in a unified storage port, the appliance interface will continue to function normally.

Unified Uplink Ports

When you configure an Ethernet uplink and an FCoE uplink on the same physical Ethernet port, it is called the unified uplink port. You can individually enable or disable either FCoE or Ethernet interfaces independently.

  • Enabling or disabling the FCoE uplink results in corresponding VFC being enabled or disabled.

  • Enabling or disabling an Ethernet uplink results in corresponding physical port being enabled or disabled.

If you disable an Ethernet uplink, it disables the underlying physical port in an unified uplink. So, even if the FCoE uplink is enabled, the FCoE uplink also goes down. But if you disable an FCoE uplink, only the VFC goes down. If the Ethernet uplink is enabled, it can still function properly in the unified uplink port.

Ports on the Cisco UCS 6300 Series Fabric Interconnects

The Cisco UCS 6300 Series Fabric Interconnect includes the Cisco UCS 6324 Fabric Interconnect for UCS Mini (Cisco UCS Manager Release 3.0), and the Cisco UCS 6332 and 6332-16UP Fabric Interconnects (Cisco UCS Manager Release 3.1).

The following table summarizes the port usage for the Cisco UCS 6300 Series Fabric Interconnects:

Fabric Interconnect Name:

Cisco UCS 6324 (Cisco UCS Mini)

Cisco UCS 6332

Cisco UCS 6332-16UP

Description:

Fabric Interconnect with 4 unified ports and 1 scalability port

32–Port Fabric Interconnect

40–Port Fabric Interconnect

Number of fixed 40 GB Interfaces:

—

6 (ports 17-32)

6 (ports 35-40)

Number of 1GB/10GB Interfaces (depending on the SFP module installed)

All

Ports 5–26 using breakout cable

Ports 17–34 using breakout cable

Unified Ports (8 Gb/s, FC, FCoE)

4

None

Ports 1–16


Note

Cisco UCS 6300 Series Fabric Interconnects support breakout capability for ports. For more information on how the 40G ports can be converted into four 10G ports, see Scalability and Breakout Ports.

Port Modes

The port mode determines whether a unified port on the fabric interconnect is configured to carry Ethernet or Fibre Channel traffic. The fabric interconnect does not automatically discover the port mode. You configure the port mode in Cisco UCS Central.

Changing the port mode deletes the existing port configuration and replaces it by a new logical port. Any objects associated with that port configuration, such as VLANs and VSANS, are removed. There is no restriction on the number of times you can change the port mode for a unified port.

Effect of Port Mode Changes on Data Traffic

Port mode changes can cause an interruption to the data traffic for the Cisco UCS domain. The length of the interruption and the traffic that is affected depend upon the configuration of the Cisco UCS domain and the module on which you made the port mode changes.


Tip

To minimize the traffic disruption during system changes, form a Fibre Channel uplink port-channel across the fixed and expansion modules.

Impact of Port Mode Changes on an Expansion Module

After you make port mode changes on an expansion module, the module reboots. All traffic through ports on the expansion module is interrupted for approximately one minute while the module reboots.

Impact of Port Mode Changes on the Fixed Module in a Cluster Configuration

A cluster configuration has two fabric interconnects. After you make port changes to the fixed module, the fabric interconnect reboots. The impact on the data traffic depends upon whether or not you have configured the server vNICs to failover to the other fabric interconnect when one fails.

If you change the port modes on the expansion module of one fabric interconnect and then wait for that to reboot before changing the port modes on the second fabric interconnect, the following occurs:

  • With server vNIC failover, traffic fails over to the other fabric interconnect and no interruption occurs.

  • Without server vNIC failover, all data traffic through the fabric interconnect on which you changed the port modes is interrupted for approximately eight minutes while the fabric interconnect reboots.

If you change the port modes on the fixed modules of both fabric interconnects simultaneously, all data traffic through the fabric interconnects are interrupted for approximately eight minutes while the fabric interconnects reboot.

Impact of Port Mode Changes on the Fixed Module in a Standalone Configuration

A standalone configuration has only one fabric interconnect. After you make port changes to the fixed module, the fabric interconnect reboots. All data traffic through the fabric interconnect is interrupted for approximately eight minutes while the fabric interconnect reboots.

Port Roles

The port role defines the type of traffic carried over a unified port connection.

All of the port roles listed are configurable on both the fixed and expansion module, including server ports, which are configurable on the 6200 and later series fabric interconnect expansion modules.

By default, unified ports changed to Ethernet port mode are set to the uplink Ethernet port role. Unified ports changed to Fibre Channel (FC) port mode are set to the FC uplink port role. You cannot unconfigure FC ports.

Changing the port role does not require a reboot.

When you set the port mode to Ethernet, you can configure the following port roles:

  • Server ports

  • Ethernet uplink ports

  • FCoE storage ports

  • FCoE uplink ports

  • Appliance ports

When you set the port mode to FC, you can configure the following port roles:

  • FC uplink ports

  • FC storage ports

Guidelines for Configuring Unified Ports

Consider the following guidelines and restrictions when configuring unified ports:

Hardware and Software Requirements

Unified ports are not supported on 6100 series fabric interconnects.

Port Mode Placement

Because the Cisco UCS Central GUI interface uses a slider to configure the port mode for unified ports on a fixed or expansion module, it automatically enforces the following restrictions which limits how port modes can be assigned to unified ports. When using the Cisco UCS Central CLI interface, these restrictions are enforced when you commit the transaction to the system configuration. If the port mode configuration violates any of the following restrictions, the Cisco UCS Central CLI displays an error:

  • Ethernet ports must be grouped together in a block. For each module (fixed or expansion), the Ethernet port block must start with the first port and end with an even numbered port.

  • Fibre Channel ports must be grouped together in a block. For each module (fixed or expansion), the first port in the Fibre Channel port block must follow the last Ethernet port and extend to include the rest of the ports in the module. For configurations that include only Fibre Channel ports, the Fibre Channel block must start with the first port on the fixed or expansion module.

  • Alternating Ethernet and Fibre Channel ports is not supported on a single module.

Example of a valid configuration— Might include unified ports 1–16 on the fixed module configured in Ethernet port mode and ports 17–32 in Fibre Channel port mode. On the expansion module you could configure ports 1–4 in Ethernet port mode and then configure ports 5–16 in Fibre Channel mode. The rule about alternating Ethernet and Fibre Channel port types is not violated because this port arrangement complies with the rules on each individual module.

Example of an invalid configuration— Might include a block of Fibre Channel ports starting with port 16. Because each block of ports has to start with an odd-numbered port, you would have to start the block with port 17.


Note

The total number of uplink Ethernet ports and uplink Ethernet port channel members that can be configured on each fabric interconnect is limited to 31. This limitation includes uplink Ethernet ports and uplink Ethernet port channel members configured on the expansion module.

The 40GB ports on the 6300 series fabric interconnects do not support expansion module configuration.

Cautions and Guidelines for Configuring Unified Uplink Ports and Unified Storage Ports

The following are cautions and guidelines to follow while working with unified uplink ports and unified storage ports:

  • You must configure a non default native VLAN on FCoE and unified uplink ports. This VLAN is not used for any traffic. Cisco UCS Central will reuse an existing fcoe-storage-native-vlan for this purpose. This fcoe-storage-native-vlan will be used as a native VLAN on FCoE and unified uplinks.

  • In an unified uplink port, if you do not specify a non default VLAN for the Ethernet uplink port the fcoe-storage-native-vlan will be assigned as the native VLAN on the unified uplink port. If the Ethernet port has a non default native VLAN specified as native VLAN, this will be assigned as the native VLAN for unified uplink port.

  • When you create or delete a member port under an Ethernet port channel, Cisco UCS Central automatically creates or deletes the member port under FCoE port channel. The same happens when you create or delete a member port in FCoE port channel.

  • When you configure an Ethernet port as a standalone port, such as server port, Ethernet uplink, FCoE uplink or FCoE storage and make it as a member port for an Ethernet or FCOE port channel, Cisco UCS Central automatically makes this port as a member of both Ethernet and FCoE port channels.

  • When you remove the membership for a member port from being a member of server uplink, Ethernet uplink, FCoE uplink or FCoE storage, Cisco UCS Central deletes the corresponding members ports from Ethernet port channel and FCoE port channel and creates a new standalone port.

  • For unified uplink ports and unified storage ports, when you create two interfaces, only one license is checked out. As long as either interface is enabled, the license remains checked out. The license will be released only if both the interfaces are disabled for a unified uplink port or a unified storage port.

  • Cisco UCS 6100 series fabric interconnect switch can only support 1VF or 1VF-PO facing same downstream NPV switch.

Configuring Ports


Note

Ports configured for Cisco UCS Manager releases prior to 4.1(3) were supported in Cisco UCS Central release 1.3, but are not supported in later releases of Cisco UCS Central. Any additional configuration of those ports must be done in Cisco UCS Manager.

Before you begin

  • You must be running Cisco UCS Manager release 4.1(3) or above.

  • All Cisco UCS Manager domains must be included in a Cisco UCS Central domain group.

  • Port Configuration must be set to Global on the Policy Resolution Control page in Cisco UCS Manager.

Procedure

Step 1

Click the Browse Tables icon and choose Fabric Interconnects.

Step 2

Click on a Fabric Interconnect to open it for editing.

Step 3

Click Ports.

Step 4

Choose the port that you want to configure.

Step 5

On the Ports page, click the Tools icon on the far right and select Configure Port.

The Configure Port page for the selected port displays.

Step 6

Select the Role for the port.

For Ethernet ports, this can be one of the following:

  • Appliance

  • FCoE Storage

  • FCoE Uplink

  • Server

  • Uplink

For FC ports, this can be one of the following:

  • FC Uplink

  • FC Storage

Step 7

Complete the fields as required for your selection.

Step 8

Click Save.

Configuring an Appliance Port

Appliance ports are used to connect fabric interconnects to directly attached NFS storage.

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. You can disable the port after it is configured. To enable or disable the port, enter the interface mode and use enable or disable command.

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope eth-storage

Enters the Ethernet storage mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric # create interface slot-id port-id

Creates the appliance port and enters interface mode.

Step 7

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/interface # set user-label user-label-name

(Optional)

Sets the user label for this port.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/interface # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure the appliance port and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope eth-storage
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric # create interface 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/interface* # set user-label EthStorage1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/interface* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/interface #

Configuring an Ethernet Uplink Port

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. You can disable the port after it is configured. To enable or disable the port, enter the interface mode and use enable or disable command.

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope eth-uplink

Enters the Ethernet uplink mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric # create interface slot-id port-id

Creates the ethernet uplink port and enters interface mode.

Step 7

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric/interface # set {eth-link-profile eth-link-profile-name | flow-control-policy policy-name | speed {1 | 10 | 40} | user-label user-label-name}

(Optional)

Enables you to specify options for this uplink port. You can specify all of the following options at the same time or do any one of them as required:

  • eth-link-profile —Sets the Ethernet link profile name.

  • flow-control-policy —Sets the flow control policy.

  • speed —Sets the desired speed. If you do not specify the speed, the default speed will be 10 gbps.

  • user-label —Sets the user label.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric/interface # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure the Ethernet uplink port and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope eth-uplink
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric # create interface 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric/interface* # set eth-link-profile ELP_1 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric/interface* # set flow-control-policy FCP_1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric/interface* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/fabric/interface #

Configuring an Ethernet Server Port

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. You can disable the port after it is configured. To enable or disable the port, enter the interface mode and use enable or disable command.

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope eth-server

Enters the Ethernet server mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server/fabric # create interface slot-id port-id

Creates the server port and enters interface mode.

Step 7

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server/fabric/interface # set user-label user-label-name

(Optional)

Sets the user label for this port.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server/fabric/interface # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure the server port and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope eth-server
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server/fabric # create interface 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server/fabric/interface* # set user-lable EthServer1 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server/fabric/interface* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-server/fabric/interface #

Configuring an FCoE Storage Port

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. You can disable the port after it is configured. To enable or disable the port, enter the interface mode and use enable or disable command.

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope fc-storage

Enters the FCoE storage mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric # create interface fcoe slot-id port-id

Creates the FCoE storage port and enters FCoE mode.

Step 7

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fcoe # set user-label user-label-name | vsan-member vsan_name}

(Optional)

Enables you to specify options for this FCoE storage port. You can specify all of the following options at the same time or do any one of them as required:

  • user-label —Sets the user label.

  • vsan-member —Sets the VSAN member name.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fcoe # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure the FCoE storage port and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope fc-storage
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric # create interface fcoe 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fcoe* # set fillpattern idle 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fcoe* # set vsan-member VSAN1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fcoe* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fcoe #

Configuring an FCoE Uplink Port

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. You can disable the port after it is configured. To enable or disable the port, enter the interface mode and use enable or disable command.

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope fc-uplink

Enters the FC uplink mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create fcoeinterface slot-id port-id

Creates the FCoE uplink port and enters interface mode.

Step 7

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoeinterface # set {eth-link-profile eth-link-profile-name | user-label user-label-name}

(Optional)

Enables you to specify options for this FCoE uplink port. You can specify all of the following options at the same time or do any one of them as required:

  • eth-link-profile —Sets the Ethernet link profile name.

  • user-label —Sets the user label.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoeinterface # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure the FCoE uplink port and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope fc-uplink
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create fcoeinterface 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoeinterface* # set eth-link-profile ELP_1 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoeinterface* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoeinterface #

Configuring an FC Storage Port

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. You can disable the port after it is configured. To enable or disable the port, enter the interface mode and use enable or disable command.

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope fc-storage

Enters the FC storage mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric # create interface fc slot-id port-id

Creates the fc storage port and enters FC mode.

Step 7

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fc # set {fillpattern {arbff | idle} | user-label user-label-name | vsan-member vsan_name}

(Optional)

Enables you to specify options for this FC storage port. You can specify all of the following options at the same time or do any one of them as required:

  • fillpattern —Sets the fill pattern.

  • user-label —Sets the user label.

  • vsan-member —Sets the VSAN member name.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fc # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure the FC storage port and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope fc-storage
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric # create interface 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fc* # set fillpattern idle 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fc* # set vsan-member VSAN1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fc* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-storage/fabric/fc #

Configuring an FC Uplink Port

When you configure a port on a fabric interconnect, the administrative state is automatically set to enabled. You can disable the port after it is configured. To enable or disable the port, enter the interface mode and use enable or disable command.

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope fc-uplink

Enters the FC uplink mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create interface slot-id port-id

Creates the ethernet uplink port and enters interface mode.

Step 7

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/interface # set {fillpattern {arbff | idle} | user-label user-label-name | vsan-member vsan_name}

(Optional)

Enables you to specify options for this FC uplink port. You can specify all of the following options at the same time or do any one of them as required:

  • fillpattern —Sets the fill pattern.

  • user-label —Sets the user label.

  • vsan-member —Sets the VSAN member name.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/interface # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure the FC uplink port and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope fc-uplink
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create interface 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/interface* # set fillpattern idle 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/interface* # set vsan-member VSAN1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/interface* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/interface #

Scalability and Breakout Ports

The Cisco UCS 6300 Series Fabric Interconnects contain scalability ports that can be broken out into groups of 4 10-Gigabit Ethernet ports. The configuration requires a Small Form-Factor Pluggable adapter (SPF) that has one 40GB QSFP+ on one end to connect to the Fabric Interconnect, and four 10 GB ports to connect to different end points supporting 10 GB connectivity.

  • The Cisco UCS 6324 Fabric Interconnect contains one scalability port that can be used as a licensed server port for supported Cisco UCS rack servers, an appliance port, or a FCoE storage port.

  • The Cisco UCS 6332 and Cisco UCS 6332-16 UP fabric interconnects contain multiple 40-Gigabet Ethernet ports that can be broken out into 10-Gigabit Ethernet ports.


Caution

Configuring breakout ports requires rebooting the Fabric Interconnect. Any existing configuration on a port is erased. It is recommended to break out all required ports in a single transaction.

Once you configure a breakout port, you can configure each 10 GB sub-port as server, uplink, FCoE uplink, FCoE storage or appliance port as required.

The following table summarizes the constraints for breakout functionality for the Cisco UCS 6332 and 6332-16UP fabric interconnects:

Fabric Interconnect

Breakout Configurable Ports

Normal Ports with no Breakout Support

UCS-FI-6332

1-12,15-26

13-14,27-32

Note

 

  • Auto-negotiate behavior is not supported on ports 27–32.

  • A maximum of four ports are allowed as breakout ports if using QoS jumbo frames.

UCS-FI-6332-16UP

17-34

1-16,35-40

Note

 

  • Auto-negotiate behavior is not supported on ports 35-40.

  • A maximum of four ports are allowed as breakout ports if using QoS jumbo frames.

Creating a Port Channel

Procedure

Step 1

Click the Browse Tables icon and choose Fabric Interconnects.

Step 2

Click on a Fabric Interconnect to open it for editing.

Step 3

In the Fabric Interconnect page, click the Tools icon and choose Create Port Channel.

Step 4

In Basic, select the type of port channel that you want to create.

This can be one of the following:

Step 5

Complete the fields as required for your selection.

Step 6

Click Save.

Configuring an Appliance Port Channel

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope eth-storage

Enters the Ethernet storage mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric # create port-channel port-channel-ID

Creates the appliance port channel and enters port channel mode.

Step 7

UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/port-channel # create member-port slot-id port-id

Creates the appliance port and enters interface mode.

Step 8

(Optional) UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/port-channel/member-port # commit-buffer

(Optional)

Commits the transaction to the system.

Example

The following example shows how to configure an appliance port channel with two members and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope eth-storage
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric # create port-channel 22
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/port-channel* # create member-port 1 1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/port-channel/member-port* # exit 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/port-channel* # create member-port 1 2 
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/port-channel/member-port* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-storage/fabric/port-channel/member-port #

Configuring an FC Uplink Port Channel

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope fc-uplink

Enters the FC uplink mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create port-channel port-channel-ID

Creates the ethernet uplink port and enters interface mode.

Step 7

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel # create member-port slot-id port-id

Creates the member port for this port channel, and enters FC uplink member port mode.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel/member-port # set fillpattern {arbff | idle}

Sets the fill pattern.

Step 9

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel/member-port # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure an FC uplink port channel with two members and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope fc-uplink
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create port-channel 22
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel* # create member-port 1 1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel/member-port* # exit
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel* # create member-port 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel/member-port* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/port-channel/member-port #

Configuring an FCoE Uplink Port Channel

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain# scope fc-uplink

Enters the FC uplink mode.

Step 5

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric {a | b}

Enters the fabric interconnect mode.

Step 6

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create fcoe-port-channel port-channel-ID

Creates the FCoE uplink port channel and enters FCoE port channel mode.

Step 7

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel # create fcoe-member-port slot-id port-id

Creates the member port for this port channel, and enters FCoE uplink member port mode.

Step 8

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel/fcoe-member-port # eth-link-profile

Sets the Ethernet link profile name.

Step 9

UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel/fcoe-member-port # commit-buffer

Commits the transaction to the system.

Example

The following example shows how to configure an FCoE uplink port channel with two members and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain 1008
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope fc-uplink
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # scope fabric a
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric # create fcoe-port-channel 22
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel* # create fcoe-member-port 1 1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel/fcoe-member-port* # exit
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel* # create fcoe-member-port 1 2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel/fcoe-member-port* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/fabric/fcoe-port-channel/fcoe-member-port #

Pin Groups

LAN Pin Groups

Cisco UCS uses LAN pin groups to pin Ethernet traffic from a vNIC on a server to an uplink Ethernet port or port channel on the fabric interconnect. You can use this pinning to manage the distribution of traffic from the servers.

To configure pinning for a server, you must include the LAN pin group in a vNIC policy. The vNIC policy is then included in the service profile assigned to that server. All traffic from the vNIC travels through the I/O module to the specified uplink Ethernet port.


Note

If you do not assign a pin group to a server interface through a vNIC policy, Cisco UCS Central chooses an uplink Ethernet port or port channel for traffic from that server interface dynamically. This choice is not permanent. A different uplink Ethernet port or port channel may be used for traffic from that server interface after an interface flap or a server reboot.

If an uplink is part of a LAN pin group, the uplink is not necessarily reserved for only that LAN pin group. Other vNIC's policies that do not specify a LAN pin group can use the uplink as a dynamic uplink.

SAN Pin Groups

Cisco UCS uses SAN pin groups to pin Fibre Channel traffic from a vHBA on a server to an uplink Fibre Channel port on the fabric interconnect. You can use this pinning to manage the distribution of traffic from the servers.

Note

In Fibre Channel switch mode, SAN pin groups are irrelevant. Any existing SAN pin groups will be ignored.

To configure pinning for a server, you must include the SAN pin group in a vHBA policy. The vHBA policy is then included in the service profile assigned to that server. All traffic from the vHBA will travel through the I/O module to the specified uplink Fibre Channel port.

You can assign the same pin group to multiple vHBA policies. As a result, you do not need to manually pin the traffic for each vHBA.


Important

Changing the target interface for an existing SAN pin group disrupts traffic for all vHBAs which use that pin group. The fabric interconnect performs a log in and log out for the Fibre Channel protocols to re-pin the traffic.

Configuring a LAN Pin Group

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain # scope eth-uplink

Enters Ethernet uplink mode.

Step 5

UCSC(resource-mgr) /domain-mgmt /ucs-domain/eth-uplink # create pin-group pin_group_name

Creates an Ethernet (LAN) pin group with the specified name, and enters Ethernet uplink pin group mode.

Step 6

UCSC(resource-mgr) /domain-mgmt /ucs-domain/eth-uplink/pin-group # set target {a | b} {port slot-num/port-num | port-channel port-channel-ID}

Sets the Ethernet pin target to the specified fabric and port, or fabric and port channel.

Step 7

UCSC(resource-mgr) /domain-mgmt /ucs-domain/eth-uplink/pin-group # commit-buffer

Commits the transaction to the system configuration.

Example

The following example shows how to create a LAN pin group named ethpingroup12, set the pin group target to slot 1, port 2, and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain 1009
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope eth-uplink
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink # create pin-group ethpingroup12
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/pin-group* # set target a port 1/2
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/pin-group* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/eth-uplink/pin-group #

Configuring a SAN Pin Group

Procedure

  Command or Action Purpose

Step 1

UCSC# connect resource-mgr

Enters resource manager mode.

Step 2

UCSC(resource-mgr)# scope domain-mgmt

Enters domain management.

Step 3

UCSC(resource-mgr) /domain-mgmt # scope ucs-domain domain ID

Enters the specified domain.

Step 4

UCSC(resource-mgr) /domain-mgmt /ucs-domain # scope fc-uplink

Enters Fibre Channel uplink mode.

Step 5

UCSC(resource-mgr) /domain-mgmt /ucs-domain/fc-uplink # create pin-group pin_group_name

Creates a Fibre Channel (SAN) pin group with the specified name, and enters Fibre Channel uplink pin group mode.

Step 6

UCSC(resource-mgr) /domain-mgmt /ucs-domain/fc-uplink/pin-group # set target {a | b} {port slot-num/port-num | port-channel port-channel-ID}

Sets the Fibre Channel pin target to the specified fabric and port, or fabric and port-channel.

Step 7

UCSC(resource-mgr) /domain-mgmt /ucs-domain/fc-uplink/pin-group # commit-buffer

Commits the transaction to the system configuration.

Example

The following example shows how to create a SAN pin group named fcpingroup12, set the pin group target to slot 2, port 1, and commit the transaction: 

UCSC# connect resource-mgr
UCSC(resource-mgr)# scope domain-mgmt
UCSC(resource-mgr) /domain-mgmt # scope ucs-domain 1009
UCSC(resource-mgr) /domain-mgmt/ucs-domain # scope fc-uplink
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink # create pin-group fcpingroup12
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/pin-group* # set target a port 2/1
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/pin-group* # commit-buffer
UCSC(resource-mgr) /domain-mgmt/ucs-domain/fc-uplink/pin-group #

Fibre Channel Switching Mode

The Fibre Channel switching mode determines how the fabric interconnect behaves as a switching device between the servers and storage devices. The fabric interconnect operates in either of the following Fibre Channel switching modes:

End-Host Mode

End-host mode allows the fabric interconnect to act as an end host to the connected fibre channel networks, representing all servers (hosts) connected to it through virtual host bus adapters (vHBAs). This behavior is achieved by pinning (either dynamically pinned or hard pinned) vHBAs to Fibre Channel uplink ports, which makes the Fibre Channel ports appear as server ports (N-ports) to the rest of the fabric. When in end-host mode, the fabric interconnect avoids loops by denying uplink ports from receiving traffic from one another.

End-host mode is synonymous with N Port Virtualization (NPV) mode. This mode is the default Fibre Channel Switching mode.


Note

When you enable end-host mode, if a vHBA is hard pinned to an uplink Fibre Channel port and this uplink port goes down, the system cannot repin the vHBA, and the vHBA remains down.

Switch Mode

Switch mode is the traditional Fibre Channel switching mode. Switch mode allows the fabric interconnect to connect directly to a storage device. Enabling Fibre Channel switch mode is useful in Pod models where there is no SAN (for example, a single Cisco UCS domain that is connected directly to storage), or where a SAN exists (with an upstream MDS).

Switch mode is not the default Fibre Channel switching mode.


Note

In Fibre Channel switch mode, SAN pin groups are irrelevant. Any existing SAN pin groups are ignored.

Configuring Fibre Channel Switching Mode

You can configure your fabric interconnect to use either FC End-Host Mode or FC Switch Mode. By default, the FI is set to end-host mode.


Note

When you change the Fibre Channel switching mode, Cisco UCS Central logs you out and restarts the fabric interconnect. For a cluster configuration, Cisco UCS Central restarts both fabric interconnects sequentially. The second fabric interconnect can take several minutes to complete the change in Fibre Channel switching mode and become system ready.

Procedure

Step 1

Click the Browse Tables icon and choose Fabric Interconnects.

Step 2

Click on a Fabric Interconnect to open it for editing.

Step 3

On the fabric interconnect page, click the Tools icon and select the FC switching mode.

If you are using end-host mode, Set FC Switching Mode displays. If you are using FC switching mode, Set FC End-Host Mode displays.

Step 4

Click Yes on the warning page to change the configuration and restart the FI.

Viewing Port Configuration Status

Procedure

Step 1

Click the Browse Tables icon and choose Fabric Interconnects.

Step 2

Click on a Fabric Interconnect to open it for editing.

Step 3

Click the Tools icon on the far right and select Configuration Status.

The Configuration Status page for the selected port displays.

Step 4

Click Close to close the window.