The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This section includes the following topics:
PortChannels refer to the aggregation of multiple physical interfaces into one logical interface to provide higher aggregated bandwidth, load balancing, and link redundancy (See Figure 6-1). PortChannels can connect to interfaces across switching modules, so a failure of a switching module cannot bring down the PortChannel link.
Figure 6-1 PortChannel Flexibility
PortChannels on Cisco MDS 9000 Family switches allow flexibility in configuration. This illustrates three possible PortChannel configurations:
An E PortChannel refers to the aggregation of multiple E ports into one logical interface to provide higher aggregated bandwidth, load balancing, and link redundancy. PortChannels can connect to interfaces across switching modules, so a failure of a switching module cannot bring down the PortChannel link.
A PortChannel has the following features and restrictions:
Note See the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide for information about failover scenarios for PortChannels and FSPF links.
An F PortChannel is also a logical interface that combines a set of F ports connected to the same Fibre Channel node and operates as one link between the F ports and the NP ports. The F PortChannels support bandwidth utilization and availability like the E PortChannels. F PortChannels are mainly used to connect MDS core and NPV switches to provide optimal bandwidth utilization and transparent failover between the uplinks of a VSAN.
An F PortChannel trunk combines the functionality and advantages of a TF port and an F PortChannel. This logical link uses the Cisco PTP and PCP protocols over Cisco EPP (ELS).
Note If a Cisco MDS 9124 or 9134 switch is used as a core switch, only a nontrunking F PortChannel is supported. Trunking is not supported on this platform when NPIV enabled.
Trunking is a commonly used storage industry term. However, the Cisco NX-OS software and switches in the Cisco MDS 9000 Family implement trunking and PortChanneling as follows:
Figure 6-3 PortChanneling and Trunking
PortChanneling and trunking are used separately across an ISL.
See the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide.
Two methods support the load-balancing functionality:
Figure 6-4 illustrates how source ID 1 (SID1) and destination ID1 (DID1) based load balancing works. When the first frame in a flow is received on an interface for forwarding, link 1 is selected. Each subsequent frame in that flow is sent over the same link. No frame in SID1 and DID1 utilizes link 2.
Figure 6-4 SID1 and DID1-Based Load Balancing
Figure 6-5 illustrates how exchange-based load balancing works. When the first frame in an exchange is received for forwarding on an interface, link 1 is chosen by a hash algorithm. All remaining frames in that particular exchange are sent on the same link. For exchange 1, no frame uses link 2. For the next exchange, link 2 is chosen by the hash algorithm. Now all frames in exchange 2 use link 2.
Figure 6-5 SID1, DID1, and Exchange-Based Load Balancing
For more information on configuring load balancing and in-order delivery features, see the Cisco MDS 9000 Family NX-OS Fabric Configuration Guide.
You can configure each PortChannel with a channel group mode parameter to determine the PortChannel protocol behavior for all member ports in this channel group. The possible values for a channel group mode are as follows:
Table 6-1 compares ON and ACTIVE modes.
When you delete the PortChannel, the corresponding channel membership is also deleted. All interfaces in the deleted PortChannel convert to individual physical links. After the PortChannel is removed, regardless of the mode used (ACTIVE and ON), the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down (see the “Graceful Shutdown” section).
If you delete the PortChannel for one port, then the individual ports within the deleted PortChannel retain the compatibility parameter settings (speed, mode, port VSAN, allowed VSAN, and port security). You can explicitly change those settings as required.
You can add or remove a physical interface (or a range of interfaces) to an existing PortChannel. The compatible parameters on the configuration are mapped to the PortChannel. Adding an interface to a PortChannel increases the channel size and bandwidth of the PortChannel. Removing an interface from a PortChannel decreases the channel size and bandwidth of the PortChannel.
This section describes interface configuration for a PortChannel and includes the following topics:
Note For information about PortChannel support on Generation 2 switching modules, see the “PortChannel Limitations” section.
You can add a physical interface (or a range of interfaces) to an existing PortChannel. The compatible parameters on the configuration are mapped to the PortChannel. Adding an interface to a PortChannel increases the channel size and bandwidth of the PortChannel.
A port can be configured as a member of a static PortChannel only if the following configurations are the same in the port and the PortChannel:
After the members are added, regardless of the mode (ACTIVE and ON) used, the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down (see the “Generation 1 PortChannel Limitations” section and “Graceful Shutdown” section).
A compatibility check ensures that the same parameter settings are used in all physical ports in the channel. Otherwise, they cannot become part of a PortChannel. The compatibility check is performed before a port is added to the PortChannel.
The check ensures that the following parameters and settings match at both ends of a PortChannel:
Note Ports in shared rate mode cannot form a PortChannel or a trunking PortChannel.
A port addition procedure fails if the capability and administrative parameters in the remote switch are incompatible with the capability and administrative parameters in the local switch. If the compatibility check is successful, the interfaces are operational and the corresponding compatibility parameter settings apply to these interfaces.
If the operational parameters are incompatible, the compatibility check fails and the interface is placed in a suspended or isolated state based on the configured mode:
You can force the port configuration to be overwritten by the PortChannel. In this case, the interface is added to a PortChannel.
Note When PortChannels are created from within an interface, the force option cannot be used.
After the members are forcefully added, regardless of the mode (ACTIVE and ON) used, the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down (see the “Graceful Shutdown” section).
When a physical interface is deleted from the PortChannel, the channel membership is automatically updated. If the deleted interface is the last operational interface, then the PortChannel status is changed to a down state. Deleting an interface from a PortChannel decreases the channel size and bandwidth of the PortChannel.
After the members are deleted, regardless of the mode (ACTIVE and ON) used, the ports at either end are gracefully brought down, indicating that no frames are lost when the interface is going down (see the “Generation 1 PortChannel Limitations” section and “Graceful Shutdown” section).
In earlier Cisco SAN-OS releases, PortChannels required additional administrative tasks to support synchronization. The Cisco NX-OS software provides robust error detection and synchronization capabilities. You can manually configure channel groups or they can be automatically created. In both cases, the channel groups have the same capability and configurational parameters. Any change in configuration applied to the associated PortChannel interface is propagated to all members of the channel group.
A protocol to exchange PortChannel configurations is available in all Cisco MDS switches. This addition simplifies PortChannel management with incompatible ISLs. An additional autocreation mode enables ISLs with compatible parameters to automatically form channel groups without manual intervention.
The PortChannel protocol is enabled by default.
The PortChannel protocol expands the PortChannel functional model in Cisco MDS switches. It uses the exchange peer parameters (EPP) services to communicate across peer ports in an ISL. Each switch uses the information received from the peer ports along with its local configuration and operational values to decide if it should be part of a PortChannel. The protocol ensures that a set of ports are eligible to be part of the same PortChannel. They are only eligible to be part of the same PortChannel if all the ports have a compatible partner.
The PortChannel protocol uses two subprotocols:
This section describes how to configure the PortChannel protocol and includes the following sections:
Note Channel groups are not supported on internal ports in the Cisco Fabric Switch for HP c-Class BladeSystem and the Cisco Fabric Switch for IBM BladeSystem.
Assuming link A1-B1 comes up first (see Figure 6-6), that link is operational as an individual link.When the next link comes up, for example, A2-B2, the PortChannel protocol identifies if this link is compatible with link A1-B1 and automatically creates channel groups 10 and 20 in the respective switches. If link A3-B3 can join the channel groups (the PortChannels), the respective ports have compatible configurations. If link A4-B4 operates as an individual link, it is because of the incompatible configuration of the two end ports with the other member ports in this channel group.
Figure 6-6 Autocreating Channel Groups
The channel group numbers are selected dynamically, and as such, the administrative configuration of the ports forming the channel group at either end are applicable to the newly created channel group. The channel group number being chosen dynamically may be different across reboots for the same set of PortChannels based on the order of ports that are initialized in the switch.
Table 6-2 identifies the differences between user-configured and auto-configured channel groups.
Note Autocreation is not supported as of MDS NX-OS Release 4.1(1b) and later.
The autocreation protocol has the following functionality:
– A port is aggregated into a compatible autocreated PortChannel.
– A port is aggregated with another compatible port to form a new PortChannel.
Tip When enabling autocreation in any switch in the Cisco MDS 9000 Family, we recommend that you retain at least one interconnected port between the switches without any autocreation configuration. If all ports between two switches are configured with the autocreation feature at the same time, you may face a possible traffic disruption between these two switches as the ports are automatically disabled and reenabled when ports are added to an autocreated PortChannel.
A user-configured channel group cannot be converted to an autocreated channel group. However, you can convert an autocreated channel group to a manual channel group. Once performed, this task is irreversible. The channel group number does not change, but the member ports operate according to the properties of the manually configured channel group, and the autocreation of channel group is implicitly disabled for all member ports.
Tip If you enable persistence, be sure to enable it at both ends of the PortChannel.
Before configuring a PortChannel, consider the following guidelines:
Note On switches with Generation 1 switching modules, or a combination of Generation 1 and Generation 2 switching modules, you can configure a maximum of 128 PortChannels. On switches with only Generation 2 switching modules, or Generation 2 and Generation 3 switching modules, you can configure a maximum of 256 PortChannels.
If you misconfigure PortChannels, you may receive a misconfiguration message. If you receive this message, the PortChannel’s physical links are disabled because an error has been detected.
A PortChannel error is detected if the following requirements are not met:
If all three conditions are not met, the faulty link is disabled.
Enter the show interface command for that interface to verify that the PortChannel is functioning as required.
This section includes the guidelines and limitations for this feature:
Cisco MDS 9000 Family switches support the following number of PortChannels per switch:
This section includes the restrictions on creation and addition of PortChannel members to a PortChannel on Generation 1 hardware:
When configuring the host-optimized ports on Generation 1 hardware, the following PortChannel guidelines apply:
– You can configure only the first port in each 4-port group as an E port (for example, the first port in ports 1–4, the fifth port in ports 5–8, and so on). If the first port in the group is configured as a PortChannel, the other three ports in each group (ports 2–4, 6–8, and so on) are not usable and remain in the shutdown state.
– If any of the other three ports are configured in a no shutdown state, you cannot configure the first port to be a PortChannel. The other three ports continue to remain in a no shutdown state.
The following guidelines and restrictions are applicable for F and TF PortChannels:
PortChannels are created with default values. You can change the default configuration just like any other physical interface.
Figure 6-7 provides examples of valid PortChannel configurations.
Figure 6-7 Valid PortChannel Configurations
Figure 6-8 provides examples of invalid configurations. Assuming that the links are brought up in the 1, 2, 3, 4 sequence, links 3 and 4 will be operationally down as the fabric is misconfigured.
Figure 6-8 Misconfigured Configurations
Table 6-3 lists the default settings for PortChannels.
|
|
---|---|
This section includes the following topics:
To create a PortChannel, follow these steps:
|
|
|
---|---|---|
Configures the specified PortChannel (1) using the default ON mode. |
By default, the CLI and the Device Manager create the PortChannel in ON mode in the NPIV core switches and ACTIVE mode on the NPV switches. DCNM-SAN creates all PortChannels in ACTIVE mode. We recommend that you create PortChannels in ACTIVE mode.
To configure ACTIVE mode, follow these steps:
|
|
|
---|---|---|
Configures the specified PortChannel (1) using the default ON mode. |
||
To delete a PortChannel, follow these steps:
To add an interface to a PortChannel, follow these steps:
|
|
|
---|---|---|
Adds physical Fibre Channel port 1/15 to channel group 15. If channel group 15 does not exist, it is created. The port is shut down. |
To add a range of ports to a PortChannel, follow these steps:
Note By default, the CLI adds a interface normally to a PortChannel, while DCNM-SAN adds the interface by force, unless specified explicitly.
To force the addition of a port to a PortChannel, follow these steps:
|
|
|
---|---|---|
Forces the addition of the physical port for interface fc1/1 to channel group 1. The port is shut down. |
To delete a physical interface (or a range of physical interfaces) from a PortChannel, follow these steps:
|
|
|
---|---|---|
Deletes the physical Fibre Channel interfaces in channel group 2. |
To configure automatic channel groups, follow these steps:
You can convert autocreated channel group to a user-configured channel group using the port-channel channel-group-number persistent EXEC command. If the PortChannel does not exist, this command is not executed.
To display PortChannel configuration information, perform one of the following tasks:
For detailed information about the fields in the output from these commands, refer to the Cisco MDS NX-OS Command Reference.
You can view specific information about existing PortChannels at any time from EXEC mode. The following show commands provide further details on existing PortChannels. You can force all screen output to go to a printer or save it to a file. See Examples 6-1 to 6-6 .
Example 6-1 Displays the PortChannel Summary
Example 6-2 Displays the PortChannel Configured in the Default ON Mode
Example 6-3 Displays the PortChannel Configured in the ACTIVE Mode
The show port-channel consistency command has two options: without details and with details.
Example 6-4 Displays the Consistency Status without Details
Example 6-5 Displays the Consistency Status with Details
The show port-channel usage command displays details of the used and unused PortChannel numbers.
Example 6-6 Displays the PortChannel Usage
Example 6-7 Displays the PortChannel Compatibility
Use the existing show commands to obtain further details on autocreated channel group attributes. Autocreated PortChannels are indicated explicitly to help differentiate them from the manually created PortChannels. See Examples 6-8 to 6-10 .
Example 6-8 Displays Autocreated PortChannels
Example 6-9 Displays the Specified PortChannel Interface
Example 6-10 Displays the PortChannel Summary
This example shows how to configure F PortChannel in shared mode and bring up the link (not supported on the MDS 91x4 switches) between F ports on the NPIV core switches and NP ports on the NPV switches:
Step 1 Enable the F port trunking and channeling protocol on the MDS core switch.
Step 2 Enable NPIV on the MDS core switch:
Step 3 Create the PortChannel on the MDS core switch:
Step 4 Configure the PortChannel member interfaces on the core switch:
Step 5 Create the PortChannel on the NPV switch:
Step 6 Configure the PortChannel member interfaces on the NPV switch:
Step 7 Set the administrative state of all the PortChannel member interfaces in both NPIV core switch and the NPV switch to ON:
Note The speed configuration must be the same for all member interfaces in a PortChannel. While configuring the channel in dedicated mode, ensure that required bandwidth is available to the ports.
This example shows how to configure channeling in dedicated mode and bring up the TF-TNP PortChannel link between TF ports in the NPIV core switch, and TNP ports in the NPV switch:
Step 1 Enable the F port trunking and channeling protocol on the MDS core switch:
Step 2 Enable NPIV on the MDS core switch:
Step 3 Create the PortChannel on the MDS core switch:
Step 4 Configure the PortChannel member interfaces on the MDS core switch in dedicated mode:
Step 5 Create the PortChannel in dedicated mode on the NPV switch:
Step 6 Configure the PortChannel member interfaces on the NPV switch in dedicated mode:
Step 7 Set the administrative state of all the PortChannel member interfaces in both NPIV core switch and the NPV switch to ON: