The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter contains the following sections:
A port channel bundles individual interfaces into a group to provide increased bandwidth and redundancy. Port channeling also load balances traffic across these physical interfaces. The port channel stays operational as long as at least one physical interface within the port channel is operational.
You create an port channel by bundling compatible interfaces. You can configure and run either static port channels or port channels running the Link Aggregation Control Protocol (LACP).
Any configuration changes that you apply to the port channel are applied to each member interface of that port channel. For example, if you configure Spanning Tree Protocol (STP) parameters on the port channel, Cisco NX-OS applies those parameters to each interface in the port channel.
You can use static port channels, with no associated protocol, for a simplified configuration. For more efficient use of the port channel, you can use the Link Aggregation Control Protocol (LACP), which is defined in IEEE 802.3ad. When you use LACP, the link passes protocol packets.
Using port channels, Cisco NX-OS provides wider bandwidth, redundancy, and load balancing across the channels.
You can collect ports into a static port channel or you can enable the Link Aggregation Control Protocol (LACP). Configuring port channels with LACP requires slightly different steps than configuring static port channels. For information on port channel configuration limits, see the Verified Scalability document for your platform. For more information about load balancing, see Load Balancing Using Port Channels.
Note |
Cisco NX-OS does not support Port Aggregation Protocol (PAgP) for port channels. |
A port channel bundles individual links into a channel group to create a single logical link that provides the aggregate bandwidth of several physical links. If a member port within a port channel fails, traffic previously carried over the failed link switches to the remaining member ports within the port channel.
Each port can be in only one port channel. All the ports in an port channel must be compatible; they must use the same speed and operate in full-duplex mode. When you are running static port channels, without LACP, the individual links are all in the on channel mode; you cannot change this mode without enabling LACP.
Note |
You cannot change the mode from ON to Active or from ON to Passive. |
You can create a port channel directly by creating the port-channel interface, or you can create a channel group that acts to aggregate individual ports into a bundle. When you associate an interface with a channel group, Cisco NX-OS creates a matching port channel automatically if the port channel does not already exist. You can also create the port channel first. In this instance, Cisco NX-OS creates an empty channel group with the same channel number as the port channel and takes the default configuration.
Note |
A port channel is operationally up when at least one of the member ports is up and that port’s status is channeling. The port channel is operationally down when all member ports are operationally down. |
Port channels can be configured in one of two ways: either in global configuration mode or in switch profile mode. Consider the following guidelines and limitations when configuring port channels via the configuration synchronization feature in Cisco NX-OS:
Note |
Several port channel sub-commands are not configurable in switch profile mode. These commands can be configured from global configuration mode even if the port channel is created and configured in switch profile mode. For example, the following command can only be configured in global configuration mode: switchport private-vlan association trunk primary-vlan secondary-vlan |
For more information on switch profiles, see the Cisco Nexus 6000 Series NX-OS System Management Configuration Guide.
When you add an interface to a port channel group, Cisco NX-OS checks certain interface attributes to ensure that the interface is compatible with the channel group. Cisco NX-OS also checks a number of operational attributes for an interface before allowing that interface to participate in the port-channel aggregation.
The compatibility check includes the following operational attributes:
Use the show port-channel compatibility-parameters command to see the full list of compatibility checks that Cisco NX-OS uses.
You can only add interfaces configured with the channel mode set to on to static port channels. You can also only add interfaces configured with the channel mode as active or passive to port channels that are running LACP. You can configure these attributes on an individual member port.
When the interface joins a port channel, the following individual parameters are replaced with the values on the port channel:
The following interface parameters remain unaffected when the interface joins a port channel:
After you enable forcing a port to be added to a channel group by entering the channel-group force command, the following two conditions occur:
Cisco NX-OS load balances traffic across all operational interfaces in a port channel by reducing part of the binary pattern formed from the addresses in the frame to a numerical value that selects one of the links in the channel. Port channels provide load balancing by default.
Note |
You have the option to include the source and destination port number for the Layer 4 frame. |
You can configure the switch to use one of the following methods (see the following table for more details) to load balance across the port channel:
Configuration |
Layer 2 Criteria |
Layer 3 Criteria |
Layer 4 Criteria |
---|---|---|---|
Destination MAC |
Destination MAC |
Destination MAC |
Destination MAC |
Source MAC |
Source MAC |
Source MAC |
Source MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Destination IP |
Destination MAC |
Destination MAC, destination IP |
Destination MAC, destination IP |
Source IP |
Source MAC |
Source MAC, source IP |
Source MAC, source IP |
Source and destination IP |
Source and destination MAC |
Source and destination MAC, source and destination IP |
Source and destination MAC, source and destination IP |
Destination TCP/UDP port |
Destination MAC |
Destination MAC, destination IP |
Destination MAC, destination IP, destination port |
Source TCP/UDP port |
Source MAC |
Source MAC, source IP |
Source MAC, source IP, source port |
Source and destination TCP/UDP port |
Source and destination MAC |
Source and destination MAC, source and destination IP |
Source and destination MAC, source and destination IP, source and destination port |
Fabric Extenders are not configurable individually. Fabric extender configurations are defined on the Cisco Nexus device. In the case of the port-channel load balancing protocol, the table below illustrates which port-channel load balancing option is automatically configured on the fabric extender modules as a result of the configuration performed on the Cisco Nexus device.
The following table shows the criteria used for each configuration:
Configuration |
Layer 2 Criteria |
Layer 3 Criteria |
Layer 4 Criteria |
---|---|---|---|
Destination MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Source MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Source and destination MAC |
Destination IP |
Source and destination MAC |
Source and destination MAC, and source and destination IP |
Source and destination MAC, and source and destination IP |
Source IP |
Source and destination MAC |
Source and destination MAC, and source and destination IP |
Source and destination MAC, and source and destination IP |
Source and destination IP |
Source and destination MAC |
Source and destination MAC, and source and destination IP |
Source and destination MAC, and source and destination IP |
Destination TCP/UDP port |
Source and destination MAC |
Source and destination MAC, and source and destination IP |
Source and destination MAC, source and destination IP , and source and destination port |
Source TCP/UDP port |
Source and destination MAC |
Source and destination MAC, and source and destination IP |
Source and destination MAC, source and destination IP , and source and destination port |
Source and destination TCP/UDP port |
Source and destination MAC |
Source and destination MAC, source and destination IP |
Source and destination MAC, source and destination IP , and source and destination port |
Use the option that provides the balance criteria with the greatest variety in your configuration. For example, if the traffic on a port channel is going only to a single MAC address and you use the destination MAC address as the basis of port-channel load balancing, the port channel always chooses the same link in that port channel; using source addresses or IP addresses might result in better load balancing.
Understanding LACP
Note |
You must enable the LACP feature before you can configure and use LACP functions. |
The following figure shows how individual links can be combined into LACP port channels and channel groups as well as function as individual links.
With LACP, just like with static port-channels, you can bundle up to 16 interfaces in a channel group.
Note |
When you delete the port channel, Cisco NX-OS automatically deletes the associated channel group. All member interfaces revert to their previous configuration. |
You cannot disable LACP while any LACP configurations are present.
LACP uses the following parameters:
Note |
The LACP system ID is the combination of the LACP system priority value and the MAC address. |
Individual interfaces in port channels are configured with channel modes. When you run static port channels, with no protocol, the channel mode is always set to on. After you enable LACP globally on the device, you enable LACP for each channel by setting the channel mode for each interface to active or passive. You can configure either channel mode for individual links in the LACP channel group.
Note |
You must enable LACP globally before you can configure an interface in either the active or passive channel mode. |
The following table describes the channel modes.
Channel Mode |
Description |
---|---|
passive |
LACP mode that places a port into a passive negotiating state, in which the port responds to LACP packets that it receives but does not initiate LACP negotiation. |
active |
LACP mode that places a port into an active negotiating state, in which the port initiates negotiations with other ports by sending LACP packets. |
on |
All static port channels, that is, that are not running LACP, remain in this mode. If you attempt to change the channel mode to active or passive before enabling LACP, the device returns an error message. You enable LACP on each channel by configuring the interface in that channel for the channel mode as either active or passive. When an LACP attempts to negotiate with an interface in the on state, it does not receive any LACP packets and becomes an individual link with that interface; it does not join the LACP channel group. |
Both the passive and active modes allow LACP to negotiate between ports to determine if they can form a port channel, based on criteria such as the port speed and the trunking state. The passive mode is useful when you do not know whether the remote system, or partner, supports LACP.
Ports can form an LACP port channel when they are in different LACP modes as long as the modes are compatible as in the following examples:
Using port channels, data traffic may be dynamically redistributed due to either a link failure or load balancing. LACP uses the Marker Protocol to ensure that frames are not duplicated or reordered because of this redistribution. Cisco NX-OS supports only Marker Responders.
The following table provides a brief summary of major differences between port channels with LACP enabled and static port channels. For information about the maximum configuration limits, see the Verified Scalability document for your device.
Configuring Port Channels
Parameters |
Default |
---|---|
Port channel |
Admin up |
Load balancing method for Layer 3 interfaces |
Source and destination IP address |
Load balancing method for Layer 2 interfaces |
Source and destination MAC address |
Load balancing per module |
Disabled |
RBH modulo mode |
Disabled |
LACP |
Disabled |
Channel mode |
on |
LACP system priority |
32768 |
LACP port priority |
32678 |
Minimum links for LACP |
1 |
Minimum links for FEX fabric port channel |
1 |
A port channel aggregates similar ports to provide increased bandwidth in a single manageable interface.
The introduction of the minimum links feature further refines LACP port-channel operation and provides increased bandwidth in one manageable interface.
The LACP port-channel minimum links feature does the following:
Note |
The minimum links feature works only with LACP port channels. However, the device allows you to configure this feature in non-LACP port channels, but the feature is not operational. |
You can create a port channel before creating a channel group. Cisco NX-OS automatically creates the associated channel group.
Note |
If you want LACP-based port channels, you need to enable LACP. |
This example shows how to create a port channel:
switch# configure terminal switch (config)# interface port-channel 1
You can add a port to a new channel group or to a channel group that already contains ports. Cisco NX-OS creates the port channel associated with this channel group if the port channel does not already exist.
Note |
If you want LACP-based port channels, you need to enable LACP. |
This example shows how to add an Ethernet interface 1/4 to channel group 1:
switch# configure terminal switch (config)# interface ethernet 1/4 switch(config-if)# switchport mode trunk switch(config-if)# channel-group 1
You can configure the load-balancing algorithm for port channels that applies to the entire device.
Note |
If you want LACP-based port channels, you need to enable LACP. |
Command or Action | Purpose | |
---|---|---|
Step 1 | switch# configure terminal | Enters configuration mode. |
Step 2 | switch(config)# port-channel load-balance ethernet {[destination-ip | destination-mac | destination-port | source-dest-ip | source-dest-mac | source-dest-port | source-ip | source-mac | source-port] crc-poly} | Specifies the load-balancing algorithm for the device. The range depends on the device. The default is source-dest-mac. |
Step 3 | switch(config)# no port-channel load-balance ethernet | (Optional) Restores the default load-balancing algorithm of source-dest-mac. |
Step 4 | switch# show port-channel load-balance | (Optional) Displays the port-channel load-balancing algorithm. |
This example shows how to configure source IP load balancing for port channels:
switch# configure terminal switch (config)# port-channel load-balance ethernet source-ip
By default, ingress multicast traffic on any port in the switch selects a particular port channel member to egress the traffic. You can configure hardware hashing for multicast traffic to reduce potential bandwidth issues and to provide effective load balancing of the ingress multicast traffic. Use the hardware multicast hw-hash command to enable hardware hashing. To restore the default, use the no hardware multicast hw-hash command.
Command or Action | Purpose | |
---|---|---|
Step 1 | switch# configure terminal |
Enters configuration mode. |
Step 2 | switch(config)# interface port-channel channel-number |
Selects the port channel and enters the interface configuration mode. |
Step 3 | switch(config-if)# hardware multicast hw-hash |
Configures hardware hashing for the specified port channel. |
This example shows how to configure hardware hashing on a port channel:
switch# configure terminal
switch (config)# interface port-channel 21
switch(config-if)# hardware multicast hw-hash
This example shows how to remove hardware hashing from a port channel:
switch# configure terminal
switch (config)# interface port-channel 21
switch(config-if)# no hardware multicast hw-hash
LACP is disabled by default; you must enable LACP before you begin LACP configuration. You cannot disable LACP while any LACP configuration is present.
LACP learns the capabilities of LAN port groups dynamically and informs the other LAN ports. Once LACP identifies correctly matched Ethernet links, it facilitates grouping the links into an port channel. The port channel is then added to the spanning tree as a single bridge port.
Command or Action | Purpose | |
---|---|---|
Step 1 | switch# configure terminal |
Enters configuration mode. |
Step 2 | switch(config)# feature lacp |
Enables LACP on the switch. |
Step 3 | switch(config)# show feature |
(Optional) Displays enabled features. |
This example shows how to enable LACP:
switch# configure terminal
switch(config)# feature lacp
You can configure the channel mode for each individual link in the LACP port channel as active or passive. This channel configuration mode allows the link to operate with LACP.
When you configure port channels with no associated protocol, all interfaces on both sides of the link remain in the on channel mode.
Ensure that you have enabled the LACP feature.
Command or Action | Purpose | |||
---|---|---|---|---|
Step 1 | switch# configure terminal | Enters global configuration mode. |
||
Step 2 | switch(config)# interface type slot/port |
Specifies the interface to configure, and enters the interface configuration mode.
|
||
Step 3 | switch(config-if)# channel-group channel-number [force] [mode {on | active | passive}] |
Specifies the port mode for the link in a port channel. After LACP is enabled, you configure each link or the entire channel as active or passive. force—Specifies that the LAN port be forcefully added to the channel group. mode—Specifies the port channel mode of the interface. active—Specifies that when you enable LACP, this command enables LACP on the specified interface. The interface is in an active negotiating state in which the port initiates negotiations with other ports by sending LACP packets. on—(Default mode) Specifies that all port channels that are not running LACP remain in this mode. passive—Enables LACP only if an LACP device is detected. The interface is in a passive negotiation state in which the port responds to LACP packets that it receives but does not initiate LACP negotiation. When you run port channels with no associated protocol, the channel mode is always on. |
||
Step 4 | switch(config-if)# no channel-group number mode |
Returns the port mode to on for the specified interface. |
This example shows how to set the LACP-enabled interface to active port-channel mode for Ethernet interface 1/4 in channel group 5:
switch# configure terminal switch (config)# interface ethernet 1/4 switch(config-if)# channel-group 5 mode active
This example shows how to forcefully add an interface to the channel group 5:
switch(config)# interface ethernet 1/1 switch(config-if)# channel-group 5 force switch(config-if)#
You can configure the LACP minimum links feature. Although minimum links work only in LACP, you can enter the CLI commands for this feature for non-LACP port channels, but these commands are nonoperational.
Ensure that you are in the correct port-channel interface.
Command or Action | Purpose | |
---|---|---|
Step 1 | switch# configure terminal | Enters global configuration mode. |
Step 2 | switch(config)# interface port-channel number | Specifies the interface to configure, and enters the interface configuration mode. |
Step 3 | switch(config-if)# lacp min-links number | Specifies the port-channel interface to configure the number of minimum links and enters the interface configuration mode. The range is from 1 to 16. |
Step 4 | switch(config-if)# show running-config interface port-channel number | (Optional) Displays the port-channel minimum links configuration. |
The following example shows how to configure LACP port-channel minimum links.
switch# configure terminal switch(config)# interface port-channel 3 switch(config-if)# lacp min-links 3 switch(config-if)# show running-config interface port-channel 3 interface port-channel 3 lacp min-links 3
You can change the LACP timer rate to modify the duration of the LACP timeout. Use the lacp rate command to set the rate at which LACP control packets are sent to an LACP-supported interface. You can change the timeout rate from the default rate (30 seconds) to the fast rate (1 second). This command is supported only on LACP-enabled interfaces.
Ensure that you have enabled the LACP feature.
Command or Action | Purpose | |||
---|---|---|---|---|
Step 1 | switch# configure terminal | Enters global configuration mode. |
||
Step 2 | switch(config)# interface type slot/port |
Specifies the interface to configure and enters the interface configuration mode.
|
||
Step 3 | switch(config-if)# lacp rate fast |
Configures the fast rate (one second) at which LACP control packets are sent to an LACP-supported interface. |
This example shows how to configure the LACP fast rate on Ethernet interface 1/4:
switch# configure terminal switch(config)# interface ethernet 1/4 switch(config-if)# lacp rate fast
This example shows how to restore the LACP default rate (30 seconds) on Ethernet interface 1/4.
switch# configure terminal switch(config)# interface ethernet 1/4 switch(config-if)# no lacp rate fast
The LACP system ID is the combination of the LACP system priority value and the MAC address.
Ensure that you have enabled the LACP feature.
Command or Action | Purpose | |
---|---|---|
Step 1 | switch# configure terminal |
Enters configuration mode. |
Step 2 | switch(config)# lacp system-priority priority |
Configures the system priority for use with LACP. Valid values are 1 through 65535, and higher numbers have lower priority. The default value is 32768. |
Step 3 | switch# show lacp system-identifier |
(Optional) Displays the LACP system identifier. |
This example shows how to set the LACP system priority to 2500:
switch# configure terminal
switch(config)# lacp system-priority 2500
You can configure each link in the LACP port channel for the port priority.
Ensure that you have enabled the LACP feature.
Command or Action | Purpose | |||
---|---|---|---|---|
Step 1 | switch# configure terminal | Enters global configuration mode. |
||
Step 2 | switch(config)# interface type slot/port |
Specifies the interface to configure, and enters the interface configuration mode.
|
||
Step 3 | switch(config-if)# lacp port-priority priority |
Configures the port priority for use with LACP. Valid values are 1 through 65535, and higher numbers have lower priority. The default value is 32768. |
This example shows how to set the LACP port priority for Ethernet interface 1/4 to 40000:
switch# configure terminal switch (config)# interface ethernet 1/4 switch(config-if)# lacp port priority 40000
Command or Action | Purpose | |
---|---|---|
Step 1 | configure terminal Example: switch# configure terminal switch(config)# |
Enters global configuration mode. |
Step 2 | interface port-channel number Example: switch(config)# interface port-channel 1 switch(config) # |
Specifies the port channel interface to configure, and enters interface configuration mode. |
Step 3 | shutdown Example: switch(config-if)# shutdown switch(config-if) # |
Administratively shuts down the port channel. |
Step 4 | no lacp graceful-convergence Example: switch(config-if)# no lacp graceful-convergence switch(config-if) # |
Disables LACP graceful convergence on the specified port channel. |
Step 5 | no shutdown Example: switch(config-if)# no shutdown switch(config-if) # |
Administratively brings the port channel up. |
Step 6 | copy running-config startup-config Example: switch(config-if)# copy running-config startup-config |
(Optional) Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration. |
The following example disables LACP graceful convergence on a port channel:
switch# configure terminal switch(config) # interface port-channel 1 switch(config-if) # shutdown switch(config-if) # no lacp graceful-convergence switch(config-if) # no shutdown switch(config-if) #
Command or Action | Purpose | |
---|---|---|
Step 1 | configure terminal Example: switch# configure terminal switch(config)# |
Enters global configuration mode. |
Step 2 | interface port-channel number Example: switch(config)# interface port-channel 1 switch(config) # |
Specifies the port channel interface to configure, and enters interface configuration mode. |
Step 3 | shutdown Example: switch(config-if)# shutdown switch(config-if) # |
Administratively shuts down the port channel. |
Step 4 | lacp graceful-convergence Example: switch(config-if)# lacp graceful-convergence switch(config-if) # |
Enables LACP graceful convergence on the specified port channel. |
Step 5 | no shutdown Example: switch(config-if)# no shutdown switch(config-if) # |
Administratively brings the port channel up. |
Step 6 | copy running-config startup-config Example: switch(config-if)# copy running-config startup-config |
(Optional) Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration. |
The following example disables LACP graceful convergence on a port channel:
switch# configure terminal switch(config) # interface port-channel 1 switch(config-if) # shutdown switch(config-if) # lacp graceful-convergence switch(config-if) # no shutdown switch(config-if) #
To display port channel configuration information, perform one of the following tasks:
Command |
Purpose |
||
---|---|---|---|
switch# show interface port-channel channel-number | Displays the status of a port channel interface. |
||
switch# show feature | Displays enabled features. |
||
switch# show resource | Displays the number of resources currently available in the system. |
||
switch# show lacp {counters | interface type slot/port | neighbor | port-channel | system-identifier} | Displays LACP information.
|
||
switch# show port-channel compatibility-parameters | Displays the parameters that must be the same among the member ports in order to join a port channel. |
||
switch# show port-channel database [interface port-channel channel-number] | Displays the aggregation state for one or more port-channel interfaces. |
||
switch# show port-channel summary | Displays a summary for the port channel interfaces. |
||
switch# show port-channel traffic | Displays the traffic statistics for port channels. |
||
switch# show port-channel usage | Displays the range of used and unused channel numbers. |
||
switch# show port-channel database | Displays information on current running of the port channel feature. |
||
switch# show port-channel load-balance | Displays information about load-balancing using port channels. |
Note |
Certain traffic flows are not subject to hashing, for example when there is a single port in a port-channel. |
To display the load-balancing outgoing port ID, perform one of the tasks listed in the table below.
Command |
Purpose |
---|---|
switch# show port-channel load-balance forwarding-path interface port-channel port-channel-id vlan vlan-id dst-ip src-ip dst-mac src-mac l4-src-port port-id l4-dst-port port-id | Displays the outgoing port ID. |
This example shows the output of the short port-channel load-balance command.
switch# show port-channel load-balance forwarding-path interface port-channel 10 vlan 1 dst-ip 1.225.225.225 src-ip 1.1.10.10 src-mac aa:bb:cc:dd:ee:ff l4-src-port 0 l4-dst-port 1 Missing params will be substituted by 0's. Load-balance Algorithm on switch: source-dest-port crc8_hash:204 Outgoing port id: Ethernet 1/1 Param(s) used to calculate load balance: dst-port: 0 src-port: 0 dst-ip: 1.225.225.225 src-ip: 1.1.10.10 dst-mac: 0000.0000.0000 src-mac: aabb.ccdd.eeff
Feature Name |
Release |
Feature Information |
---|---|---|
Min Links |
6.0(2)N1(2) |
This feature was introduced. |