The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This section includes the following topics:
Port channels refer to the aggregation of multiple physical interfaces into one logical interface to provide higher aggregated bandwidth, load balancing, and link redundancy (See Figure 6-1). Port channels can connect to interfaces across switching modules, so a failure of a switching module cannot bring down the port channel link. They are referred to as port channels in Fibre Channel (FC), Fibre Channel over IP (FCIP), and Ethernet port channels in Fibre Channel over Ethernet (FCoE). In FCoE, virtual Fibre Channel (vfc) interfaces can be bound to Ethernet port channels.
Figure 6-1 Port channel Flexibility
Port channels on Cisco MDS 9000 Series switches allow flexibility in configuration. This illustrates three possible port channel configurations:
Trunking is a commonly used storage industry term. However, the Cisco NX-OS software and switches in the Cisco MDS 9000 Series implement trunking and port channeling as follows:
Figure 6-3 Port Channeling and Trunking
Port channeling and trunking are used separately across an ISL.
See the Cisco MDS 9000 Series NX-OS Fabric Configuration Guide.
Two methods support the load-balancing functionality:
Figure 6-4 illustrates how source ID 1 (SID1) and destination ID1 (DID1) based load balancing works. When the first frame in a flow is received on an interface for forwarding, link 1 is selected. Each subsequent frame in that flow is sent over the same link. No frame in SID1 and DID1 utilizes link 2.
Figure 6-4 SID1 and DID1-Based Load Balancing
Figure 6-5 illustrates how exchange-based load balancing works. When the first frame in an exchange is received for forwarding on an interface, link 1 is chosen by a hash algorithm. All remaining frames in that particular exchange are sent on the same link. For exchange 1, no frame uses link 2. For the next exchange, link 2 is chosen by the hash algorithm. Now all frames in exchange 2 use link 2.
Figure 6-5 SID1, DID1, and Exchange-Based Load Balancing
For more information on configuring load balancing and in-order delivery features, see the Cisco MDS 9000 Series NX-OS Fabric Configuration Guide.
You can configure each port channel with a channel group mode parameter to determine the port channel protocol behavior for all member ports in this channel group. The possible values for a channel group mode are as follows:
Table 6-1 compares ON and ACTIVE modes.
Port channels are created with default values. You can change the default configuration just like any other physical interface.
Figure 6-6 provides examples of valid port channel configurations.
Figure 6-6 Valid Port Channel Configurations
Figure 6-7 provides examples of invalid configurations. Assuming that the links are brought up in the 1, 2, 3, 4 sequence, links 3 and 4 will be operationally down as the fabric is misconfigured.
Figure 6-7 Misconfigured Configurations
Note Ethernet port channel is not supported for IP Storage port for MDS 9250i switch and Gigabit Ethernet ports for MDS 9000 Series switches.
Ethernet port channels offer link redundancy between the Cisco MDS 9000 Series switch’s FCoE Ethernet ports and the connecting Ethernet switch. Fibre Channel port channels also offer (E)ISL link redundancy between Fibre Channel switches. FCIP is an (E)ISL link and is only applicable for a Fibre Channel port channel. Beneath the FCIP level, an FCIP link can run on top of Gigabit Ethernet port. This link is totally transparent to the Fibre Channel layer.
Figure 6-8 displays an Ethernet port channel.
Figure 6-8 Ethernet Port Channel
The following characteristics set Ethernet port channel solutions apart from other solutions:
For configuration information, see “Configuring Ethernet Port Channels” section.
The Cisco NX-OS software load balances traffic across all operational interfaces in a port channel by hashing the addresses in the frame to a numerical value that selects one of the links in the channel. Port channels provide load balancing by default. Port-channel load balancing uses MAC addresses, IP addresses, or Layer 4 port numbers to select the link. Port-channel load balancing uses either source or destination addresses or ports, or both source and destination addresses or ports.
You can configure the load- balancing mode to apply to all port channels that are configured on the entire device or on specified modules. The per-module configuration takes precedence over the load-balancing configuration for the entire device. You can configure one load-balancing mode for the entire device, a different mode for specified modules, and another mode for the other specified modules. You cannot configure the load-balancing method per port channel.
You can configure the type of load-balancing algorithm used. You can choose the load-balancing algorithm that determines which member port to select for egress traffic by looking at the fields in the frame.
Note The default load-balancing mode for Layer 3 interfaces is the source and destination IP address, and the default load-balancing mode for non-IP traffic is the source and destination MAC address. Use the port-channel load-balance command to set the load-balancing method among the interfaces in the channel-group bundle. The default method for Layer 2 packets is src-dst-mac. The default method for Layer 3 packets is src-dst-ip.
Ethernet port channels refer to the aggregation of multiple physical Gigabit Ethernet interfaces into one logical Ethernet interface to provide link redundancy and, in some cases, higher aggregated bandwidth and load balancing.
Note The Cisco Ethernet switch’s port channel should be configured as a static port channel, and not the default 802.3ad protocol.
Note Port channel members must be one of these combinations: ports 1–2, ports 3–4, ports 5–6, or ports 7–8.
Fibre Channel port channel is for both (E)ISL and F and TF for connections to Cisco NPV switches. Fibre Channel port channels can be comprised of either Fibre Channel interfaces or FCIP interfaces. FCIP is an (E)ISL link and is only applicable for a Fibre Channel port channel.
Fibre Channel and FCIP interfaces cannot be combined into one Fibre Channel port channel. Fibre Channel port channels must be comprised of either Fibre Channel or FCIP interfaces. The maximum number of Fibre Channel ports that can be put into a Fibre Channel port channel is 16.
An E port channel refers to the aggregation of multiple E ports into one logical interface to provide higher aggregated bandwidth, load balancing, and link redundancy. Port channels can connect to interfaces across switching modules, so a failure of a switching module cannot bring down the port channel link.
A port channel has the following features and restrictions:
Note See the Cisco MDS 9000 Series NX-OS Fabric Configuration Guide for information about failover scenarios for port channels and FSPF links.
An F port channel is also a logical interface that combines a set of F ports connected to the same Fibre Channel node and operates as one link between the F ports and the NP ports. The F port channels support bandwidth utilization and availability like the E port channels. F port channels are mainly used to connect MDS core and NPV switches to provide optimal bandwidth utilization and transparent failover between the uplinks of a VSAN.
An F port channel trunk combines the functionality and advantages of a TF port and an F port channel. This logical link uses the Cisco PTP and PCP protocols over Cisco EPP (ELS).
Note If a Cisco MDS 9124 or 9134 switch is used as a core switch, only a nontrunking F port channel is supported. Trunking is not supported on this platform when NPIV enabled.
When you delete the port channel, the corresponding channel membership is also deleted. All interfaces in the deleted port channel convert to individual physical links. After the port channel is removed, regardless of the mode used (ACTIVE and ON), the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down.
If you delete the port channel for one port, then the individual ports within the deleted port channel retain the compatibility parameter settings (speed, mode, port VSAN, allowed VSAN, and port security). You can explicitly change those settings as required.
You can add or remove a physical interface (or a range of interfaces) to an existing port channel. The compatible parameters on the configuration are mapped to the port channel. Adding an interface to a port channel increases the total bandwidth of the port channel. Removing an interface from a port channel decreases the total bandwidth of the port channel.
Note For information about port channel support on Generation 2 switching modules.
You can add a physical interface (or a range of interfaces) to an existing port channel. The compatible parameters on the configuration are mapped to the port channel. Adding an interface to a port channel increases the total bandwidth of the port channel.
A port can be configured as a member of a static port channel only if the following configurations are the same in the port and the port channel:
After the members are added, regardless of the mode (ACTIVE and ON) used, the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down.
A compatibility check ensures that the same parameter settings are used in all physical ports in the channel. Otherwise, they cannot become part of a port channel. The compatibility check is performed before a port is added to the port channel.
The check ensures that the following parameters and settings match at both ends of a port channel:
Note Ports in shared rate mode cannot form a port channel or a trunking port channel.
A port addition procedure fails if the capability and administrative parameters in the remote switch are incompatible with the capability and administrative parameters in the local switch. If the compatibility check is successful, the interfaces are operational and the corresponding compatibility parameter settings apply to these interfaces.
If the operational parameters are incompatible, the compatibility check fails and the interface is placed in a suspended or isolated state based on the configured mode:
You can force the port configuration to be overwritten by the port channel. In this case, the interface is added to a port channel.
Note When port channels are created from within an interface, the force option cannot be used.
After the members are forcefully added, regardless of the mode (ACTIVE and ON) used, the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down.
When a physical interface is deleted from the port channel, the channel membership is automatically updated. If the deleted interface is the last operational interface, then the port channel status is changed to a down state. Deleting an interface from a port channel decreases the total bandwidth of the port channel.
After the members are deleted, regardless of the mode (ACTIVE and ON) used, the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down.
In earlier Cisco SAN-OS releases, port channels required additional administrative tasks to support synchronization. The Cisco NX-OS software provides robust error detection and synchronization capabilities. Any change in configuration applied to the associated port channel interface is propagated to all members of the channel group.
A protocol to exchange port channel configurations is available in all Cisco MDS switches. This addition simplifies port channel management with incompatible ISLs.
The port channel protocol is enabled by default.
The port channel protocol expands the port channel functional model in Cisco MDS switches. It uses the exchange peer parameters (EPP) services to communicate across peer ports in an ISL. Each switch uses the information received from the peer ports along with its local configuration and operational values to decide if it should be part of a port channel. The protocol ensures that a set of ports are eligible to be part of the same port channel. They are only eligible to be part of the same port channel if all the ports have a compatible partner.
The port channel protocol uses the following subprotocol:
Bringup protocol—Automatically detects misconfigurations so you can correct them. This protocol synchronizes the port channel at both ends so that all frames for a given flow (as identified by the source FC ID, destination FC ID and OX_ID) are carried over the same physical link in both directions. This helps make applications such as write acceleration, work for port channels over FCIP links.
Before configuring a port channel, consider the following guidelines:
Note On switches with Generation 1 switching modules, or a combination of Generation 1 and Generation 2 switching modules, you can configure a maximum of 128 port channels. On switches with only Generation 2 switching modules, or Generation 2 and Generation 3 switching modules, you can configure a maximum of 256 port channels.
If you misconfigure port channels, you may receive a misconfiguration message. If you receive this message, the port channel’s physical links are disabled because an error has been detected.
A port channel error is detected if the following requirements are not met:
If all three conditions are not met, the faulty link is disabled.
Enter the show interface command for that interface to verify that the port channel is functioning as required.
This section includes the guidelines and limitations for this feature:
Cisco MDS 9000 Series switches support the following number of port channels per switch:
When configuring vFC interfaces, note the following guidelines and limitations:
– A media access control (MAC) address of an FCoE Node (ENode) or a remote Fibre Channel Forwarder (FCF) identified by the virtual FC interface
The following guidelines and restrictions are applicable for F and TF port channels:
Table 6-2 lists the default settings for port channels.
|
|
---|---|
This section includes the following topics:
To create a port channel, follow these steps:
|
|
|
---|---|---|
Configures the specified port channel (1) using the default ON mode. |
By default, the CLI and the Device Manager create the port channel in ON mode in the NPIV core switches and ACTIVE mode on the NPV switches. DCNM-SAN creates all port channels in ACTIVE mode. We recommend that you create port channels in ACTIVE mode.
To configure ACTIVE mode, follow these steps:
|
|
|
---|---|---|
Configures the specified port channel (1) using the default ON mode. |
||
To delete a port channel, follow these steps:
To add an interface to a port channel, follow these steps:
|
|
|
---|---|---|
Adds physical Fibre Channel port 1/15 to channel group 15. If channel group 15 does not exist, it is created. The port is shut down. |
To add a range of ports to a port channel, follow these steps:
Note By default, the CLI adds a interface normally to a port channel, while DCNM-SAN adds the interface by force, unless specified explicitly.
To force the addition of a port to a port channel, follow these steps:
|
|
|
---|---|---|
Forces the addition of the physical port for interface fc1/1 to channel group 1. The port is shut down. |
To delete a physical interface (or a range of physical interfaces) from a port channel, follow these steps:
|
|
|
---|---|---|
Deletes the physical Fibre Channel interfaces in channel group 2. |
To display port channel configuration information, perform one of the following tasks:
|
|
---|---|
Displays the port channel configured in the default ON mode and ACTIVE mode. |
|
Displays the Ethernet port channel group, port channel, type, protocol and member ports. |
|
For detailed information about the fields in the output from these commands, refer to the Cisco MDS NX-OS Command Reference.
You can view specific information about existing port channels at any time from EXEC mode. The following show commands provide further details on existing port channels. You can force all screen output to go to a printer or save it to a file. See Examples 6-1 to 6-6 .
Example 6-1 Displays the Port Channel Summary
Example 6-2 Displays the Configured in the Default ON Mode
Example 6-3 Displays the Port Channel Configured in the ACTIVE Mode
The show port-channel consistency command has two options: without details and with details.
Example 6-4 Displays the Consistency Status without Details
Example 6-5 Displays the Consistency Status with Details
The show port-channel usage command displays details of the used and unused port channel numbers.
Example 6-6 Displays the Port Channel Usage
Example 6-7 Displays the Port Channel Compatibility
Example 6-8 Displays the Port Channel Summary
This example shows how to configure F port channel in shared mode and bring up the link between F ports on the NPIV core switches and NP ports on the NPV switches:
Note Port channel in shared mode is not supported on the MDS 91x4 switches.
Figure 6-9 Port Channel Link Between F Ports on NPIV Core switch and NP Ports on NPV Switch
Step 1 Enable the F port trunking and channeling protocol on the MDS core switch.
Step 2 Enable NPIV on the MDS core switch:
Step 3 Create the port channel on the MDS core switch:
Step 4 Configure the port channel member interfaces on the core switch:
Step 5 Create the port channel on the NPV switch:
Step 6 Configure the port channel member interfaces on the NPV switch:
Step 7 Set the administrative state of all the port channel member interfaces in both NPIV core switch and the NPV switch to ON:
Note The speed configuration must be the same for all member interfaces in a port channel. While configuring the channel in dedicated mode, ensure that required bandwidth is available to the ports.
This example shows how to configure channeling in dedicated mode and bring up the TF-TNP port channel link between TF ports in the NPIV core switch, and TNP ports in the NPV switch:
Figure 6-10 Port Channel Link Between TF Ports on NPIV Core switch and TNP Ports on NPV Switch
Step 1 Enable the F port trunking and channeling protocol on the MDS core switch:
Step 2 Enable NPIV on the MDS core switch:
Step 3 Create the port channel on the MDS core switch:
Step 4 Configure the port channel member interfaces on the MDS core switch in dedicated mode:
Step 5 Create the port channel in dedicated mode on the NPV switch:
Step 6 Configure the port channel member interfaces on the NPV switch in dedicated mode:
Step 7 Set the administrative state of all the port channel member interfaces in both NPIV core switch and the NPV switch to ON:
This section includes the following topics:
To create a Ethernet port channel, follow these steps:
|
|
|
---|---|---|
Configures the specified Ethernet port channel using the default ON mode. |
Note Ethernet port-channels are not supported on the following:
To delete a Ethernet port channel, follow these steps:
|
|
|
---|---|---|
Deletes the specified Ethernet port channel, its associated interface mappings, and the hardware associations for this Ethernet port channel. |
To add an interface to a Ethernet port channel, follow these steps:
|
|
|
---|---|---|
To add a range of ports to a Ethernet port channel, follow these steps:
|
|
|
---|---|---|
By default, the CLI adds a interface normally to a Ethernet port channel, while DCNM-SAN adds the interface by force, unless specified explicitly.
Note We recommend that you create Ethernet port channels in Active mode.
To specify a channeling mode for an Ethernet interface, follow these steps:
|
|
|
---|---|---|
switch(config-if)# channel-group 513 mode active |
To force the addition of a port to a Ethernet port channel, follow these steps:
|
|
|
---|---|---|
Forces the addition of a port for interface ethernet-port-channel to a channel group. The port is shut down. |
To delete an interface (or a range of physical interfaces) from a Ethernet port channel, follow these steps:
|
|
|
---|---|---|
Deletes the Ethernet port channel interfaces in channel group 2. |
To display Ethernet port channel configuration information, perform one of the following tasks:
|
|
---|---|
Displays the Ethernet port channel group, port channel, type, protocol and member ports. |
|
show ethernet-port-channel database interface port-channel number |
You can view specific information about existing Ethernet port channels at any time from EXEC mode. The following show commands provide further details on existing Ethernet port channels. You can force all screen output to go to a printer or save it to a file.
Example 6-9 Displays the Ethernet Port Channel Summary
Example 6-10 Displays the Ethernet Port Channel Database
The show ethernet-port-channel command displays details of used and unused Ethernet port channel numbers:
Example 6-11 Displays the Ethernet Port Channel Usage
switch# show ethernet-port-channel usage
This section includes the following topics:
An explicit vFC interface is a vFC interface where the bound Ethernet or port-channel interface is explicitly configured. To can create an explicit vFC interface you must bind the vFC interface to a physical interface before it can be used. If you want the vFC interface to automatically bound to the port-channel interface, see Creating an Implicit Virtual Fibre Channel Port Channel Interface.
This example shows how to bind a virtual Fibre Channel interface to an Ethernet interface:
This example shows how to delete a virtual Fibre Channel interface:
An implicit VFC interface is a VFC interface that has an ID with the format slot/port, unit/slot/port, or port-channel id. When this VFC is created, the ethernet interface slot/port or unit/slot/port or port-channel id is automatically (implicitly) bound to the interface. You can create a virtual Fibre Channel port channel interface that automatically binds to the port channel with the same interface number.
For the Cisco MDS 9700 switches, MDS 9500 switches, and MDS 9250i switch, ensure that you create the Ethernet port channel interface before you create the virtual Fibre Channel port channel interface.
To display configuration information about virtual interfaces, perform one of the following tasks:
|
|
---|---|
Displays the detailed configuration of the specified Fibre Channel interface. |
|
This example shows how to display a virtual Fibre Channel interface bound to an Ethernet interface:
This example shows how to display the status of all the interfaces on the switch (some output has been removed for brevity):
This example shows how to display the mapping between the VLANs and VSANs on the switch: