Configure EtherChannel Between Catalyst 4500/4000, 5500/5000, and 6500/6000 Switches That Run CatOS System Software
Available Languages
Contents
Introduction
EtherChannel allows multiple physical Fast Ethernet or Gigabit Ethernet links to be combined into one logical channel. A logical channel allows load sharing of traffic among the links in the channel as well as redundancy in the event that one or more links in the channel fail. EtherChannel can be used to interconnect LAN switches, routers, servers, and clients with unshielded twisted pair (UTP) wiring or single-mode and multimode fiber.
Prerequisites
Requirements
There are no specific requirements for this document.
Components Used
The information in this document is based on these software and hardware versions:
-
A console cable that is suitable for the Supervisor Engine in the switch
For more information, refer to the Components Used section of the document Connecting a Terminal to the Console Port on Catalyst Switches.
-
Two Catalyst 5505 switches in a lab environment with cleared configurations
The clear config all command was entered on the switch in order to ensure a default configuration.
-
A Fast Ethernet module that is capable of EtherChannel in each Catalyst 5505
-
Four RJ-45 Ethernet crossover cables to connect the EtherChannel
For a pinout of an Ethernet crossover cable, see Appendix A: Ethernet Crossover Cables.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Conventions
Refer to Cisco Technical Tips Conventions for more information on document conventions.
Background Information
EtherChannel is an easy way to aggregate bandwidth between critical networking devices. On the Catalyst 5500/5000, a channel can be created from two ports, which creates a 200-Mbps link (400-Mbps, full-duplex), or four ports, which creates a 400-Mbps link (800-Mbps, full-duplex). Some cards and platforms also support Gigabit EtherChannel and have the ability to use from two to eight ports in an EtherChannel. The concept is the same no matter the speeds or number of links that are involved. Normally, the Spanning Tree Protocol (STP) considers these redundant links between two devices to be loops and causes the redundant links to be in blocking mode. This effectively makes the links inactive (providing only backup capabilities if the main link fails). With use of Catalyst OS (CatOS) software version 3.1(1) or later, STP treats the channel as one large link, so all the ports in the channel can be active at the same time.
This document takes you through the steps to configure EtherChannel between two Catalyst 5500/5000 switches and shows you the results of the commands that you issue. You can use Catalyst 4500/4000 and 6500/6000 switches that run CatOS in the scenarios that this document presents in order to obtain the same results. For the Catalyst 2900XL and Catalyst 1900/2820, the command syntax differs, but the EtherChannel concepts are the same. For EtherChannel guidelines and configuration information for the Catalyst 6500/6000 series switches that run Cisco IOS® system software, refer to Sample Configuration: EtherChannel Between Catalyst Switches Running CatOS and Cisco IOS Software.
For an overview and comparison of the Catalyst 6500 CatOS and Cisco IOS Software platforms, refer to Comparison of the Cisco Catalyst and Cisco IOS Operating Systems for the Cisco Catalyst 6500 Series Switch.
You can manually configure EtherChannel if you execute the appropriate commands. Or, for automatic configuration, have the switch negotiate the channel with the other side with use of Port Aggregation Protocol (PAgP). Whenever possible, use the PAgP desirable mode in order to configure EtherChannel because the manual configuration of EtherChannel sometimes creates complications. This document provides examples of the manual configuration of EtherChannel and examples of EtherChannel configuration with use of PAgP. The document also includes how to troubleshoot EtherChannel and how to use trunking with EtherChannel. In this document, the terms EtherChannel, Fast EtherChannel, Gigabit EtherChannel, or channel all refer to EtherChannel.
Network Diagram
The network setup in this section illustrates the test environment.
After the configuration of the switches was cleared with the clear config all command, the prompt was changed with the set system name command. An IP address and mask were assigned to the switch for management purposes with use of the set interface sc0 172.16.84.6 255.255.255.0 command for Switch A and the set interface sc0 172.16.84.17 255.255.255.0 command for Switch B. A default gateway was assigned to both switches with the set ip route default 172.16.84.1 command.
The switch configurations were cleared in order to start from the default conditions. The switches were given names for identification from the prompt on the command line. In order to ping between the switches for testing, the IP addresses were assigned. The default gateway was not used.
Many of the commands display more output than is needed for this discussion. Extraneous output is suppressed in this document.
Manually Configure EtherChannel
Step-by-Step
Complete these steps in order to manually configure EtherChannel:
-
Issue the show version command and the show module command.
The show version command displays the software version that the switch runs. The show module command lists which modules are installed in the switch.
Switch-A> show version WS-C5505 Software, Version McpSW: 4.5(1) NmpSW: 4.5(1) !--- This is the software version that runs on the switch. Copyright (c) 1995-1999 by Cisco Systems NMP S/W compiled on Mar 29 1999, 16:09:01 MCP S/W compiled on Mar 29 1999, 16:06:50 System Bootstrap Version: 3.1.2 Hardware Version: 1.0 Model: WS-C5505 Serial #: 066507453 Mod Port Model Serial # Versions --- ---- ---------- --------- ---------------------------------------- 1 0 WS-X5530 006841805 Hw : 1.3 Fw : 3.1.2 Fw1: 3.1(2) Sw : 4.5(1) 2 24 WS-X5225R 012785227 Hw : 3.2 Fw : 4.3(1) SW : 4.5(1) DRAM FLASH NVRAM Module Total Used Free Total Used Free Total Used Free ------ ------- ------- ------- ------- ------- ------- ----- ----- ----- 1 32640K 13650K 18990K 8192K 4118K 4074K 512K 108K 404K Uptime is 0 day, 3 hours, 32 minutes Switch-A> show module Mod Module-Name Ports Module-Type Model Serial-Num Status --- ------------------- ----- --------------------- --------- --------- ------- 1 0 Supervisor III WS-X5530 006841805 ok !--- These are the modules that are installed on the switch. 2 24 10/100BaseTX Ethernet WS-X5225R 012785227 OK Mod MAC-Address(es) Hw Fw SW --- -------------------------------------- ------ ---------- ----------------- 1 00-90-92-b0-84-00 to 00-90-92-b0-87-ff 1.3 3.1.2 4.5(1) 2 00-50-0f-b2-e2-60 to 00-50-0f-b2-e2-77 3.2 4.3(1) 4.5(1) Mod Sub-Type Sub-Model Sub-Serial Sub-Hw --- -------- --------- ---------- ------ 1 NFFC WS-F5521 0008728786 1.0 Switch-B> show version WS-C5505 Software, Version McpSW: 4.5(1) NmpSW: 4.5(1) !--- This is the software version that runs on the switch. Copyright (c) 1995-1999 by Cisco Systems NMP S/W compiled on Mar 29 1999, 16:09:01 MCP S/W compiled on Mar 29 1999, 16:06:50 System Bootstrap Version: 5.1(2) Hardware Version: 1.0 Model: WS-C5505 Serial #: 066509957 Mod Port Model Serial # Versions --- ---- ---------- --------- ---------------------------------------- 1 0 WS-X5530 008592453 Hw : 2.3 Fw : 5.1(2) Fw1: 4.4(1) SW : 4.5(1) 2 24 WS-X5234 015388641 Hw : 1.0 Fw : 4.5(2) SW : 4.5(1) DRAM FLASH NVRAM Module Total Used Free Total Used Free Total Used Free ------ ------- ------- ------- ------- ------- ------- ----- ----- ----- 1 32640K 13548K 19092K 8192K 7300K 892K 512K 119K 393K Uptime is 0 day, 3 hours, 36 minutes Switch-B> show module Mod Module-Name Ports Module-Type Model Serial-Num Status --- ------------------- ----- --------------------- --------- --------- ------- 1 0 Supervisor III WS-X5530 008592453 OK !--- These are the modules that are installed on the switch. 2 24 10/100BaseTX Ethernet WS-X5234 015388641 OK Mod MAC-Address(Es) Hw Fw SW --- -------------------------------------- ------ ---------- ----------------- 1 00-10-0d-b2-8c-00 to 00-10-0d-b2-8f-ff 2.3 5.1(2) 4.5(1) 2 00-d0-bc-03-58-98 to 00-d0-bc-03-58-af 1.0 4.5(2) 4.5(1) Mod Subtype Sub-Model Sub-Serial Sub-Hw --- -------- --------- ---------- ------ 1 EARL 1+ WS-F5520 0011591025 1.1 -
Verify that EtherChannel is supported on the ports.
Note: The show port capabilities command is available in CatOS software versions 4.x and later. If you have a software version that is earlier than 4.x, you must skip this step. Not every Fast Ethernet module supports EtherChannel. Some of the original EtherChannel modules have "Fast EtherChannel" printed on the bottom left corner of the module (as you face the module in the switch), which tells you that the feature is supported. But this convention was abandoned on later modules. The modules in this test do not have "Fast EtherChannel" printed on them, but they do support the feature.
Switch-A> show port capabilities 2/1 Model WS-X5225R Port 2/1 Type 10/100BaseTX Speed auto,10,100 Duplex half,full Trunk encap type 802.1Q,ISL Trunk mode on,off,desirable,auto,nonegotiate Channel 2/1-2,2/1-4 !--- This indicates that EtherChannel can be configured on port 2/1 !--- with two or four contiguous ports. Broadcast suppression percentage(0-100) Flow control receive-(off,on),send-(off,on) Security yes Membership static,dynamic Fast start yes Rewrite yes Switch-B> show port capabilities 2/1 Model WS-X5234 Port 2/1 Type 10/100BaseTX Speed auto,10,100 Duplex half,full Trunk encap type 802.1Q,ISL Trunk mode on,off,desirable,auto,nonegotiate !--- This indicates that EtherChannel can be configured on port 2/1 !--- with two or four contiguous ports. Channel 2/1-2,2/1-4 Broadcast suppression percentage(0-100) Flow control receive-(off,on),send-(off,on) Security yes Membership static,dynamic Fast start yes Rewrite no
A port that does not support EtherChannel looks like this:
Switch> show port capabilities 2/1 Model WS-X5213A Port 2/1 Type 10/100BaseTX Speed 10,100,auto Duplex half,full Trunk encap type ISL Trunk mode on,off,desirable,auto,nonegotiate Channel no !--- This indicates that EtherChannel is not supported on this port !--- or module. Broadcast suppression pps(0-150000) Flow control no Security yes Membership static,dynamic Fast start yes
-
Verify that the ports are connected and operational.
Before connection of the cables, the port status is:
Switch-A> show port Port Name Status Vlan Level Duplex Speed Type ----- ------------------ ---------- ---------- ------ ------ ----- ------------ 2/1 notconnect 1 normal auto auto 10/100BaseTX 2/2 notconnect 1 normal auto auto 10/100BaseTX 2/3 notconnect 1 normal auto auto 10/100BaseTX 2/4 notconnect 1 normal auto auto 10/100BaseTX
After connection of the cables between the two switches, the status is:
1999 Dec 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1 1999 DEC 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/2 1999 DEC 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/3 1999 DEC 14 20:32:44 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/4 Switch-A> show port Port Name Status Vlan Level Duplex Speed Type ----- ------------------ ---------- ---------- ------ ------ ----- ------------ 2/1 connected 1 normal a-full a-100 10/100BaseTX 2/2 connected 1 normal a-full a-100 10/100BaseTX 2/3 connected 1 normal a-full a-100 10/100BaseTX 2/4 connected 1 normal a-full a-100 10/100BaseTX Switch-B> show port Port Name Status Vlan Level Duplex Speed Type ----- ------------------ ---------- ---------- ------ ------ ----- ------------ 2/1 connected 1 normal a-full a-100 10/100BaseTX 2/2 connected 1 normal a-full a-100 10/100BaseTX 2/3 connected 1 normal a-full a-100 10/100BaseTX 2/4 connected 1 normal a-full a-100 10/100BaseTX
Because the switch configurations were cleared before the start of this test, the ports are in their default conditions. The ports are all in VLAN 1, and their speed and duplex are set to auto. After connection of the cables, the ports negotiate to a speed of 100 Mbps and full duplex. The status is connected. You are now able to ping the other switch.
Switch-A> ping 172.16.84.17 172.16.84.17 is alive
In your network, you can set the speeds manually to 100 Mbps and full duplex if you want your ports to always run at the fastest speed. Then you do not need to rely on autonegotiation. For a discussion of autonegotiation, refer to Configuring and Troubleshooting Ethernet 10/100/1000Mb Half/Full Duplex Auto-Negotiation..
-
Verify that the ports to be grouped have the same settings.
This verification is an important step that the Troubleshoot EtherChannel section covers in more detail. If the command to set up EtherChannel does not work, the cause is typically that the ports that are involved in the channel have configurations that differ from each other. These ports include the ports on the other side of the link as well as the local ports. In this case, because the switch configurations were cleared before this test, the ports are in their default conditions. The ports are all in VLAN 1, their speed and duplex are set to auto, and all spanning tree parameters for each port are set to be the same. After connection of the cables in Step 3, you saw that the ports negotiate to a speed of 100 Mbps and full duplex. Because STP runs for each VLAN, a simple configuration of the channel and response to error messages is easier than an attempt to check every STP field for consistency for each port and VLAN in the channel.
-
Identify valid port groups.
On the Catalyst 5500/5000, you can only put certain ports together in a channel. These restrictive dependencies do not apply to all platforms. The ports in a channel on a Catalyst 5500/5000 must be contiguous. If you issue the show port capabilities command for port 2/1, the output shows the possible combinations:
Switch-A> show port capabilities 2/1 Model WS-X5225R Port 2/1 ... Channel 2/1-2,2/1-4
Notice that this port can be a part of a group of two (2/1-2) or part of a group of four (2/1-4). An Ethernet Bundling Controller (EBC) on the module causes these configuration limitations. Here is an example in which the show port capabilities command is issued for another port:
Switch-A> show port capabilities 2/3 Model WS-X5225R Port 2/3 ... Channel 2/3-4,2/1-4
This port can be part of a group of two ports (2/3-4) or a group of four ports (2/1-4).
Note: There can be additional restrictions, which depends on the hardware. On certain modules (WS-X5201 and WS-X5203), you cannot form an EtherChannel with the last two ports in a port group unless the first two ports in the group already form an EtherChannel. A port group is a group of ports that are allowed to form an EtherChannel. In the example above, 2/1-4 is a port group.
For example, if you want to create separate EtherChannels with only two ports in a channel, you cannot assign ports 2/3-4 to a channel until you have first configured ports 2/1-2 to a channel. This is true only for the modules that have this restriction. Similarly, before you configure ports 2/6-7, you must configure ports 2/5-6. This restriction does not occur on the modules that this document uses (WS-X5225R and WS-X5234).
Because you are configuring a group of four ports (2/1-4), the group is within the approved grouping. You cannot assign a group of four to ports 2/3-6. This is a group of contiguous ports, but the ports do not start on the approved boundary, as the show port capabilities command shows. Valid groups are:
-
Ports 1-4
-
Ports 5-8
-
Ports 9-12
-
Ports 13-16
-
Ports 17-20
-
Ports 21-24
-
-
Create the channel.
In order to create the channel manually, use the set port channel mod/port on command for each switch. Turn the ports off on one side of the channel with use of the set port disable command before you turn EtherChannel on manually. This avoids possible problems with STP during the configuration process. STP can shut down some ports (with a port status of errdisable) if one side is configured as a channel before the other side can be configured as a channel. Because of this possibility, the creation of EtherChannels with use of PAgP is much easier. The Use PAgP to Configure EtherChannel (Recommended) section of this document covers the procedure. In order to avoid this situation when you configure EtherChannel manually, you disable the ports on Switch A, configure the channel on Switch A, configure the channel on Switch B, and then reenable the ports on Switch A.
-
Verify that channelling is off.
Switch-A> (enable) show port channel No ports channelling Switch-B> (enable) show port channel No ports channelling
-
Disable the ports on Switch A until both switches have been configured for EtherChannel.
Switch-A> (enable) set port disable 2/1-4 Ports 2/1-4 disabled. [output from Switch A upon disabling ports] 1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/1 left bridg1 1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2 1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3 1999 DEC 15 00:06:40 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
Now, STP does not generate errors and shut down the ports.
-
Turn the channel mode to on for Switch A.
Switch-A> (enable) set port channel 2/1-4 on Port(s) 2/1-4 channel mode set to on.
Note: In this case, ports 2/1 to 2/4 are configured for EtherChannel with a single command. If you configure the EtherChannel for every port independently without use of the port range, remember to mention the same admin group for all the ports that need to be part of the same EtherChannel. If you do not specify the admin group, each port belongs to a different EtherChannel group and the desired EtherChannel bundle is never formed.
-
Check the status of the channel.
Switch-A> (enable) show port channel Port Status Channel Channel Neighbor Neighbor mode status device port ----- ---------- --------- ----------- ------------------------- ---------- 2/1 disabled on channel 2/2 disabled on channel 2/3 disabled on channel 2/4 disabled on channel ----- ---------- --------- ----------- ------------------------- ----------Notice that the channel mode has been set to on, but the status of the ports is disabled (because you disabled the ports earlier). The channel is not operational at this point, but the channel becomes operational when the ports are enabled.
Because the Switch A ports were (temporarily) disabled, the Switch B ports no longer have a connection. This message is displayed on the Switch B console when Switch A ports are disabled:
Switch-B> (enable) 2000 Jan 13 22:30:03 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1 2000 Jan 13 22:30:04 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2 2000 Jan 13 22:30:04 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3 2000 Jan 13 22:30:04 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
-
Turn on the channel for Switch B.
Switch-B> (enable) set port channel 2/1-4 on Port(s) 2/1-4 channel mode set to on.
-
Verify that the channel mode is on for Switch B.
Switch-B> (enable) show port channel Port Status Channel Channel Neighbor Neighbor mode status device port ----- ---------- --------- ----------- ------------------------- ---------- 2/1 notconnect on channel 2/2 notconnect on channel 2/3 notconnect on channel 2/4 notconnect on channel ----- ---------- --------- ----------- ------------------------- ----------Notice that the channel mode for Switch B is on, but the status of the ports is notconnect. This is the case because the Switch A ports are still disabled.
-
Enable the ports on Switch A.
Switch-A> (enable) set port enable 2/1-4 Ports 2/1-4 enabled. 1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4 1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4 1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4 1999 DEC 15 00:08:40 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
-
Verify the EtherChannel Configuration
In order to verify that the channel is set up properly, issue the show port channel command.
Switch-A> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected on channel WS-C5505 066509957(SW 2/1
2/2 connected on channel WS-C5505 066509957(SW 2/2
2/3 connected on channel WS-C5505 066509957(SW 2/3
2/4 connected on channel WS-C5505 066509957(SW 2/4
----- ---------- --------- ----------- ------------------------- ----------
Switch-B> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected on channel WS-C5505 066507453(SW 2/1
2/2 connected on channel WS-C5505 066507453(SW 2/2
2/3 connected on channel WS-C5505 066507453(SW 2/3
2/4 connected on channel WS-C5505 066507453(SW 2/4
----- ---------- --------- ----------- ------------------------- ----------
If you have the output of a show port channel command from your Cisco device, you can use the Output Interpreter Tool 
(registered customers only) , which allows you to view an analysis of show command output.
The show spantree command shows that STP treats the ports as one logical port. This output lists the port as 2/1-4, which means that STP treats ports 2/1, 2/2, 2/3, and 2/4 as one port.
Switch-A> (enable) show spantree VLAN 1 Spanning tree enabled Spanning tree type ieee Designated Root 00-10-0d-b2-8c-00 Designated Root Priority 32768 Designated Root Cost 8 Designated Root Port 2/1-4 Root Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Bridge ID MAC ADDR 00-90-92-b0-84-00 Bridge ID Priority 32768 Bridge Max Age 20 sec Hello Time 2 sec Forward Delay 15 sec Port Vlan Port-State Cost Priority Fast-Start Group-Method --------- ---- ------------- ----- -------- ---------- ------------ 2/1-4 1 forwarding 8 32 disabled channel
If you have the output of a show spantree command from your Cisco device, you can use the Output Interpreter Tool (registered customers only) , which allows you to view an analysis of show command output.
EtherChannel can be implemented with different methods of traffic distribution across the ports in a channel. The EtherChannel specification does not dictate how the traffic distribution must occur across the links in a channel. The Catalyst 5500/5000 uses the last bit or the last two bits (which depends on how many links are in the channel) of the source and destination MAC addresses in the frame in order to determine which port in the channel to use. You should see a similar amount of traffic on each of the ports in the channel, if that traffic is generated by a normal distribution of MAC addresses on one side of the channel or the other. In order to verify that traffic goes over all the ports in the channel, you can use the show mac command. If your ports were active before the configuration of EtherChannel, you can reset the traffic counters to 0 with the clear counters command. The traffic values then represent how EtherChannel has distributed the traffic.
In this test environment, a real-world distribution is not achieved because there are no workstations, servers, or routers that generate traffic. The only devices that generate traffic are the switches themselves. Pings were issued from Switch A to Switch B. The unicast traffic uses the first port in the channel, as the output below shows. The receive information (Rcv-Unicast) in this case shows how Switch B distributed the traffic across the channel to Switch A. Also in the output, the transmit information (Xmit-Unicast) shows how Switch A distributed the traffic across the channel to Switch B. You also see that a small amount of switch-generated multicast traffic (Dynamic Inter-Switch Link Protocol [ISL], Cisco Discovery Protocol [CDP]) goes out all four ports. The broadcast packets are Address Resolution Protocol (ARP) queries (for the default gateway which does not exist in this lab). If you had workstations that send packets through the switch to a destination on the other side of the channel, you would expect to see that traffic goes over each of the four links in the channel. You can monitor the packet distribution in your network with use of the show mac command.
Switch-A> (enable) clear counters This command will reset all MAC and port counters reported in CLI and SNMP. Do you want to continue (y/n) [n]? y MAC and Port counters cleared. Switch-A> (enable) show mac Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast -------- -------------------- -------------------- -------------------- 2/1 9 320 183 2/2 0 51 0 2/3 0 47 0 2/4 0 47 0 (...) Port Xmit-Unicast Xmit-Multicast Xmit-Broadcast -------- -------------------- -------------------- -------------------- 2/1 8 47 184 2/2 0 47 0 2/3 0 47 0 2/4 0 47 0 (...) Port Rcv-Octet Xmit-Octet -------- -------------------- -------------------- 2/1 35176 17443 2/2 5304 4851 2/3 5048 4851 2/4 5048 4851 (...) Last-Time-Cleared -------------------------- Wed DEC 15 1999, 01:05:33
If you have the output of a show mac command from your Cisco device, you can use the Output Interpreter Tool (registered customers only) , which allows you to view an analysis of show command output.
Use PAgP to Configure EtherChannel (Recommended)
PAgP facilitates the automatic creation of EtherChannel links through the exchange of packets between channel-capable ports. The protocol learns the capabilities of port groups dynamically and informs the neighboring ports.
After PAgP identifies correctly paired channel-capable links, PAgP groups the ports into a channel. The channel is then added to the spanning tree as a single bridge port. A given outbound broadcast or multicast packet is transmitted out one port in the channel only, not out every port in the channel. In addition, outbound broadcast and multicast packets that are transmitted on one port in a channel are blocked so that the packets cannot return on any other port of the channel.
There are four user-configurable channel modes:
-
on
-
off
-
auto
-
desirable
PAgP packets are exchanged only between ports in auto and desirable mode. Ports that are configured in on or off mode do not exchange PAgP packets. For switches to which you want to form an EtherChannel, have both switches set to desirable mode. This setting gives the most robust behavior if one side or the other encounters error situations or is reset. The default mode of the channel is auto.
Both the auto and desirable modes allow ports to negotiate with connected ports in order to determine if the ports can form a channel. The determination is based on criteria such as port speed, trunking state, and native VLAN.
Ports can form an EtherChannel when they are in different channel modes as long as the modes are compatible. This list provides examples:
-
A port in desirable mode can successfully form an EtherChannel with another port that is in desirable or auto mode.
-
A port in auto mode can form an EtherChannel with another port in desirable mode.
-
A port in auto mode cannot form an EtherChannel with another port that is also in auto mode because neither port initiates negotiation.
-
A port in on mode can form a channel only with a port in on mode because ports in on mode do not exchange PAgP packets.
-
A port in off mode cannot form a channel with any port.
If this message (or a similar syslog message) is displayed when you use EtherChannel, the message indicates a mismatch of EtherChannel modes on the connected ports:
SPANTREE-2: Channel misconfig - x/x-x will be disabled
Issue the set port enable command in order to correct the configuration and reenable the ports. Valid EtherChannel configurations include:
| Port Channel Mode | Valid Neighbor Port Channel Modes |
|---|---|
| desirable | desirable or auto |
| auto (default) | desirable or auto1 |
| on | on |
| off | off |
1 If both the local and neighbor ports are in auto mode, an EtherChannel bundle does not form.
The next table provides a summary of all the possible channelling mode scenarios. Some of these combinations can cause STP to put the ports on the channelling side in errdisable state. In other words, some of the combinations shut down the ports on the channelling side.
| Switch A Channel Mode | Switch B Channel Mode | Switch A Channel State | Switch B Channel State |
|---|---|---|---|
| on | on | Channel (non-PAgP) | Channel (non-PAgP) |
| on | off | Not channel (errdisable) | Not channel |
| on | auto | Not channel (errdisable) | Not channel |
| on | desirable | Not channel (errdisable) | Not channel |
| off | on | Not channel | Not channel (errdisable) |
| off | off | Not channel | Not channel |
| off | auto | Not channel | Not channel |
| off | desirable | Not channel | Not channel |
| auto | on | Not channel | Not channel (errdisable) |
| auto | off | Not channel | Not channel |
| auto | auto | Not channel | Not channel |
| auto | desirable | Channel (PAgP) | Channel (PAgP) |
| desirable | on | Not channel | Not channel (errdisable) |
| desirable | off | Not channel | Not channel |
| desirable | auto | Channel (PAgP) | Channel (PAgP) |
| desirable | desirable | Channel (PAgP) | Channel (PAgP) |
You turn off the channel from the example in Step 6b of the section Manually Configure EtherChannel if you issue this command on Switch A and Switch B:
Switch-A> (enable) set port channel 2/1-4 auto Port(s) 2/1-4 channel mode set to auto.
The default channel mode for a port that is able to channel is auto. In order to verify this, issue this command:
Switch-A> (enable) show port channel 2/1
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected auto not channel
The show port channel port command also shows that the ports currently are not channelling. This command provides another way to verify the channel state:
Switch-A> (enable) show port channel No ports channelling Switch-B> (enable) show port channel No ports channelling
You can easily make the channel work with PAgP. At this point, both switches are set to auto mode, which means that they channel if a connected port sends a PAgP request to channel. If you set Switch A to desirable, Switch A sends PAgP packets to the other switch, asking it to channel.
Switch-A> (enable) set port channel 2/1-4 desirable Port(s) 2/1-4 channel mode set to desirable. 1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/1 left bridg1 1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2 1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3 1999 DEC 15 22:03:18 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4 1999 DEC 15 22:03:19 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2 1999 DEC 15 22:03:19 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3 1999 DEC 15 22:03:20 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4 1999 DEC 15 22:03:23 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4 1999 DEC 15 22:03:23 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4 1999 DEC 15 22:03:23 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4 1999 DEC 15 22:03:24 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
In order to view the channel, issue this command:
Switch-A> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected desirable channel WS-C5505 066509957(SW 2/1
2/2 connected desirable channel WS-C5505 066509957(SW 2/2
2/3 connected desirable channel WS-C5505 066509957(SW 2/3
2/4 connected desirable channel WS-C5505 066509957(SW 2/4
----- ---------- --------- ----------- ------------------------- ----------
Switch-A> (enable)
Because Switch B is in auto mode, Switch B responds to the PAgP packets and creates a channel with Switch A.
Switch-B> (enable)
2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/1 left bridg1
2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
2000 Jan 14 20:26:41 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
2000 Jan 14 20:26:45 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
2000 Jan 14 20:26:45 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
2000 Jan 14 20:26:45 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
2000 Jan 14 20:26:47 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
2000 Jan 14 20:26:47 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
2000 Jan 14 20:26:47 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
2000 Jan 14 20:26:48 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
Switch-B> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected auto channel WS-C5505 066507453(SW 2/1
2/2 connected auto channel WS-C5505 066507453(SW 2/2
2/3 connected auto channel WS-C5505 066507453(SW 2/3
2/4 connected auto channel WS-C5505 066507453(SW 2/4
----- ---------- --------- ----------- ------------------------- ----------
Switch-B> (enable)
Note: It is best to set both sides of the channel to desirable so that both sides try to initiate the channel if one side drops out. If you set the EtherChannel ports on Switch B to desirable mode, even though the channel is currently active and in auto mode, it poses no problem. The command is:
Switch-B> (enable) set port channel 2/1-4 desirable Port(s) 2/1-4 channel mode set to desirable.
Note: In this case, ports 2/1 to 2/4 are configured for EtherChannel with a single command. If you configure the EtherChannel for every port independently without use of the port range, remember to mention the same admin group for all the ports that need to be part of the same EtherChannel. If you do not specify the admin group, each port belongs to a different EtherChannel group and the desired EtherChannel bundle is never formed.
Switch-B> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected desirable channel WS-C5505 066507453(SW 2/1
2/2 connected desirable channel WS-C5505 066507453(SW 2/2
2/3 connected desirable channel WS-C5505 066507453(SW 2/3
2/4 connected desirable channel WS-C5505 066507453(SW 2/4
----- ---------- --------- ----------- ------------------------- ----------
Switch-B> (enable)
If Switch A drops out for some reason, or if new hardware replaces Switch A, Switch B tries to reestablish the channel. If the new equipment cannot channel, Switch B treats its ports 2/1-4 as normal nonchannelling ports. This is one of the benefits of use of the desirable mode. If you configure the channel with use of the PAgP on mode and one side of the connection has an error of some kind or a reset, an errdisable state (shutdown) results on the other side. With PAgP set in desirable mode on each side, the channel stabilizes and renegotiates the EtherChannel connection.
Silent/Non-Silent Mode
When you deal with fiber connections, there is a possibility that, even if a receive (Rx) transceiver dies, the transmit (Tx) transceiver on the other end is still up. In a similar scenario, packets can get black holed.
It is important for the switch that transmits to remove this port from the EtherChannel bundle. In order to do so on the Catalyst 5500/5000, you set PAgP in non-silent mode. Non-silent mode means that, if the Rx does not receive traffic, the port is not put into the channel. However, use of non-silent mode is not enough because this detection happens only when the channel is formed.
In order to prevent the black holing of traffic when the channel is already formed, this occurs:
-
PAgP detects that the Rx port does not receive any traffic.
-
PAgP resets the Tx transceiver of the port that does not receive traffic. PAgP resets it for 1.6 seconds so that the switch on the other end also resets the port.
-
The faulty port does not join the channel anymore because no traffic is received on that port.
On the Catalyst 5500/5000, set non-silent mode on fiber strands and silent mode on copper strands. This is both the default and recommended setting because, on fiber connections on the Catalyst 5500/5000, the negotiation is usually not available, so there is no way to detect the problem at a physical layer.
Default PAgP Settings on the Catalyst 4500/4000 and 5500/5000
By default, PAgP is auto for a plug-and-play implementation. Disable PAgP manually from the ports where there is no need to have it.
By default, the silent mode is on. Non-silent is acceptable as well. However, because a port can be connected to a device that does not send traffic (for example, a sniffer), it is more general to have silent enabled.
Recommendations
-
Use the non-silent keyword when you connect to a device that transmits bridge protocol data units (BPDUs) or other traffic. Use this keyword with the auto or desirable mode. PAgP non-silent adds an extra level of link state detection because it listens for BPDUs or other traffic in order to determine if the link functions properly. This adds a form of UniDirectional Link Detection (UDLD) capability that is not available when you use the default silent PAgP mode.
-
Use the silent keyword when you connect to a silent partner (which is a device that does not generate BPDUs or other traffic). An example of a silent partner is a traffic generator that does not transmit packets. Use the silent keyword with auto or desirable mode. If you do not specify silent or non-silent, silent is assumed.
-
The silent mode does not disable the PAgP ability to detect unidirectional links. However, when you configure a channel, non-silent prevents a unidirectional port from even joining the link.
-
A PAgP configuration (the set port channel {desirable | auto} command) is safer than a non-PAgP configuration (the set port channel on command). A PAgP configuration provides protection for unidirectional links and also avoids misconfigurations that can arise when there are ports channeling on one side of the link and not on the other side.
-
Refer to Understanding and Configuring the Unidirectional Link Detection Protocol Feature for more information on UDLD.
Trunking and EtherChannel
EtherChannel is independent of trunking. You can turn trunking on or you can leave trunking off. Also, you can turn trunking on for all the ports before you create the channel, or you can turn trunking on after you create the channel (as in this example). In terms of EtherChannel, because trunking and EtherChannel are completely separate features, it does not matter when you turn trunking on. What does matter is that all the ports that are involved are in the same mode:
-
The ports are all trunking before you configure the channel
or
-
The ports are all not trunking before you configure the channel
All the ports must be in the same trunking state before you create the channel.
After a channel is formed, whatever is changed on one port is also changed for the other ports in the channel. The modules that are used in this test bed can do ISL or IEEE 802.1Q trunking. By default, the modules are set to auto trunking and negotiate mode. This means that the ports trunk if the other side asks them to trunk, and they negotiate whether to use the ISL or 802.1Q method for trunking. If they are not asked to trunk, the ports work as normal nontrunking ports.
Switch-A> (enable) show trunk 2 Port Mode Encapsulation Status Native vlan -------- ----------- ------------- ------------ ----------- 2/1 auto negotiate not-trunking 1 2/2 auto negotiate not-trunking 1 2/3 auto negotiate not-trunking 1 2/4 auto negotiate not-trunking 1
There are a number of different ways to turn on trunking. For this example, Switch A is set to desirable. Switch A is already set to negotiate. The combination of desirable/negotiate causes Switch A to ask Switch B to trunk and to negotiate the type of trunking to perform (ISL or 802.1Q). Since Switch B defaults to autonegotiate, Switch B responds to the Switch A request. These are the results:
Switch-A> (enable) set trunk 2/1 desirable Port(s) 2/1-4 trunk mode set to desirable. Switch-A> (enable) 1999 DEC 18 20:46:25 %DTP-5-TRUNKPORTON:Port 2/1 has become isl trunk 1999 DEC 18 20:46:25 %DTP-5-TRUNKPORTON:Port 2/2 has become isl trunk 1999 DEC 18 20:46:25 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4 1999 DEC 18 20:46:25 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-4 1999 DEC 18 20:46:25 %DTP-5-TRUNKPORTON:Port 2/3 has become isl trunk 1999 DEC 18 20:46:26 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/1-4 1999 DEC 18 20:46:26 %DTP-5-TRUNKPORTON:Port 2/4 has become isl trunk 1999 DEC 18 20:46:26 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/1-4 1999 DEC 18 20:46:28 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4 1999 DEC 18 20:46:29 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4 1999 DEC 18 20:46:29 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4 1999 DEC 18 20:46:29 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4 Switch-A> (enable) show trunk 2 Port Mode Encapsulation Status Native vlan -------- ----------- ------------- ------------ ----------- 2/1 desirable n-isl trunking 1 2/2 desirable n-isl trunking 1 2/3 desirable n-isl trunking 1 2/4 desirable n-isl trunking 1
The trunk mode was set to desirable. The result was that trunking mode was negotiated with the neighbor switch, and the switches decided on ISL (n-isl). The current status now is trunking. This output shows what happened on Switch B because of the command that was issued on Switch A:
Switch-B> (enable) 2000 Jan 17 19:09:52 %DTP-5-TRUNKPORTON:Port 2/1 has become isl trunk 2000 Jan 17 19:09:52 %DTP-5-TRUNKPORTON:Port 2/2 has become isl trunk 2000 Jan 17 19:09:52 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4 2000 Jan 17 19:09:52 %DTP-5-TRUNKPORTON:Port 2/3 has become isl trunk 2000 Jan 17 19:09:52 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-4 2000 Jan 17 19:09:53 %DTP-5-TRUNKPORTON:Port 2/4 has become isl trunk 2000 Jan 17 19:09:53 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/1-4 2000 Jan 17 19:09:53 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/1-4 2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4 2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4 2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4 2000 Jan 17 19:09:55 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4 Switch-B> (enable) show trunk 2 Port Mode Encapsulation Status Native vlan -------- ----------- ------------- ------------ ----------- 2/1 auto n-isl trunking 1 2/2 auto n-isl trunking 1 2/3 auto n-isl trunking 1 2/4 auto n-isl trunking 1
Notice that all four ports (2/1-4) became trunking, even though you only specifically changed one port (2/1) to desirable. This is an example of how a change of one port in the channel affects all the ports.
Note: You must undestand that EtherChannel combines or bundles multiple links into single logical link, so it is not possible to send data through a dedicated link.
Disable EtherChannel
If you want to disable an EtherChannel or you do not want ports to participate in the EtherChannel negotiation, you can turn off the EtherChannel. Here is an example:
Switch-A> (enable) set port channel 2/1-4 off Port(s) 2/1-4 channel mode set to off.
If the ports of Switch B are configured in either the auto mode or desirable mode, the channel is not formed. If the ports of Switch B are configured as on, the ports go into the errdisable state after a few minutes. See the Waiting Too Long Before You Configure the Other Side section of this document in order to recover the ports from this state. For more information about the errdisable state, refer to Recovering From errDisable Port State on the CatOS Platforms.
The default port channel mode for the switch ports is auto. If you turn off the EtherChannel on any ports, you see the set port channel 2/1-4 off command in the switch configuration. Here is sample output that shows this command in the switch configuration:
Switch-A> (enable)show config !--- Output suppressed. #module 2 : 24-port 10/100BaseTX Ethernet set port channel 2/1-4 off
If you want to reset the port channel configuration to the default settings, you can configure the port channel mode to auto. Here is an example:
Switch-A> (enable) set port channel 2/1-4 auto Port(s) 2/1-4 channel mode set to auto.
Now the set port channel command does not appear in the switch configuration.
Troubleshoot EtherChannel
The challenges for EtherChannel can be divided into two main areas:
-
Troubleshooting during the configuration phase
-
Troubleshooting during the execution phase
Configuration errors usually occur because of mismatched parameters on the ports that are involved (for example, different speeds, different duplex, or different STP port values). However, you can also generate errors during the configuration if you set the channel on one side to on and wait too long before you configure the channel on the other side. This causes STP loops which generate an error and shut down the port.
When you encounter an error during the configuration of EtherChannel, be sure to check the status of the ports after you correct the EtherChannel error situation. If the port status is errdisable, this status indicates that the software has shut down the ports. The ports do not come on again until you issue the set port enable command.
Note: If the port status becomes errdisable, you must specifically enable the ports with use of the set port enable command in order for the ports to become active. Currently, you can correct all the EtherChannel issues, but the ports do not come up or form a channel until the ports are enabled again. Later versions of the operating system may periodically check in order to determine if errdisable ports should be enabled.
These tests are covered in this section. For the tests, trunking and EtherChannel are turned off:
- Mismatched Parameters
- Waiting Too Long Before You Configure the Other Side
- Correct the errdisable State
- Show What Happens When a Link Breaks and Is Restored
-
Bandwidth Is Limited to 1 Gbps When WS-X6148-GE-TX Ports Are Used in the Channel
Mismatched Parameters
Here is an example of mismatched parameters. Port 2/4 is set in VLAN 2 while the other ports are still in VLAN 1. In order to create a new VLAN, you must assign a VLAN Trunk Protocol (VTP) domain for the switch and then create the VLAN.
Switch-A> (enable) show port channel
No ports channelling
Switch-A> (enable) show port
Port Name Status Vlan Level Duplex Speed Type
----- ------------------ ---------- ---------- ------ ------ ----- ------------
2/1 connected 1 normal a-full a-100 10/100BaseTX
2/2 connected 1 normal a-full a-100 10/100BaseTX
2/3 connected 1 normal a-full a-100 10/100BaseTX
2/4 connected 1 normal a-full a-100 10/100BaseTX
Switch-A> (enable) set vlan 2
Cannot add/modify VLANs on a VTP server without a domain name.
Switch-A> (enable) set vtp domain testDomain
VTP domain testDomain modified
Switch-A> (enable) set vlan 2 name vlan2
Vlan 2 configuration successful
Switch-A> (enable) set vlan 2 2/4
VLAN 2 modified.
VLAN 1 modified.
VLAN Mod/Ports
---- -----------------------
2 2/4
Switch-A> (enable)
1999 DEC 19 00:19:34 %PAGP-5-PORTFROMSTP:Port 2/4 left bridg4
Switch-A> (enable) show port
Port Name Status Vlan Level Duplex Speed Type
----- ------------------ ---------- ---------- ------ ------ ----- ------------
2/1 connected 1 normal a-full a-100 10/100BaseTX
2/2 connected 1 normal a-full a-100 10/100BaseTX
2/3 connected 1 normal a-full a-100 10/100BaseTX
2/4 connected 2 normal a-full a-100 10/100BaseTX
Switch-A> (enable) set port channel 2/1-4 desirable
Port(s) 2/1-4 channel mode set to desirable.
Switch-A> (enable)
1999 DEC 19 00:20:19 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1
1999 DEC 19 00:20:19 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
1999 DEC 19 00:20:19 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
1999 DEC 19 00:20:20 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
1999 DEC 19 00:20:20 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
1999 DEC 19 00:20:22 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
1999 DEC 19 00:20:22 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
1999 DEC 19 00:20:24 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-2
1999 DEC 19 00:20:25 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-2
1999 DEC 19 00:20:25 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/3
1999 DEC 19 00:20:25 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/4
Switch-A> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected desirable channel WS-C5505 066509957(SW 2/1
2/2 connected desirable channel WS-C5505 066509957(SW 2/2
----- ---------- --------- ----------- ------------------------- ----------
Switch-A> (enable)
Notice that the channel formed only between ports 2/1-2. Ports 2/3-4 were left out because port 2/4 is in a different VLAN. There was no error message; PAgP just did what it could do to make the channel work. Watch the results when you create the channel to be sure that the results are what you expected.
Now, set the channel manually to on with port 2/4 in a different VLAN and see what happens. First, set the channel mode back to auto. This tears down the existing channel. Then, manually set the channel to on.
Switch-A> (enable) set port channel 2/1-4 auto Port(s) 2/1-4 channel mode set to auto. Switch-A> (enable) 1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-2 1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-2 1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3 1999 DEC 19 00:26:08 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4 1999 DEC 19 00:26:18 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1 1999 DEC 19 00:26:19 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/2 1999 DEC 19 00:26:19 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/3 1999 DEC 19 00:26:19 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/4 Switch-A> (enable) show port channel No ports channelling Switch-A> (enable) set port channel 2/1-4 on Mismatch in vlan number. Failed to set port(s) 2/1-4 channel mode to on. Switch-A> (enable) show port channel No ports channelling
On Switch B, when you turn the channel on, it indicates that the ports are channelling fine. You know, however, that Switch A is not configured correctly.
Switch-B> (enable) show port channel
No ports channelling
Switch-B> (enable) show port
Port Name Status Vlan Level Duplex Speed Type
----- ------------------ ---------- ---------- ------ ------ ----- ------------
2/1 connected 1 normal a-full a-100 10/100BaseTX
2/2 connected 1 normal a-full a-100 10/100BaseTX
2/3 connected 1 normal a-full a-100 10/100BaseTX
2/4 connected 1 normal a-full a-100 10/100BaseTX
Switch-B> (enable) set port channel 2/1-4 on
Port(s) 2/1-4 channel mode set to on.
Switch-B> (enable)
2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1
2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/2
2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/3
2000 Jan 17 22:54:59 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/4
2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4
2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
2000 Jan 17 22:55:00 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
Switch-B> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected on channel WS-C5505 066507453(SW 2/1
2/2 connected on channel WS-C5505 066507453(SW 2/2
2/3 connected on channel WS-C5505 066507453(SW 2/3
2/4 connected on channel WS-C5505 066507453(SW 2/4
----- ---------- --------- ----------- ------------------------- ----------
You must check both sides of the channel when you manually configure the channel in order to ensure that both sides, not just one side, are up. The above output shows that Switch B is set for a channel, but Switch A is not channelling because Switch A has one port that is in the wrong VLAN.
Waiting Too Long Before You Configure the Other Side
In this situation, Switch B has EtherChannel turned on, but Switch A does not have EtherChannel turned on because the switch has a VLAN configuration error. Ports 2/1-3 are in VLAN 1, and port 2/4 is in VLAN 2. When one side of an EtherChannel is set to on while the other side is still in auto mode, these events occur:
-
After a few minutes, Switch B shuts down its ports because of a spanning loop detection. This occurs because Switch B ports 2/1-4 all act as one large port while Switch A ports 2/1-4 are all completely independent ports.
-
A broadcast that is sent from Switch B to Switch A on port 2/1 is sent back to Switch B on ports 2/2, 2/3, and 2/4 because Switch A treats these ports as independent ports.
-
Switch B interprets this as a spanning tree loop. Notice that the ports on Switch B are now disabled and have a status of errdisable:
Switch-B> (enable) 2000 Jan 17 22:55:48 %SPANTREE-2-CHNMISCFG: STP loop - channel 2/1-4 is disabled in vlan 1. 2000 Jan 17 22:55:49 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4 2000 Jan 17 22:56:01 %PAGP-5-PORTFROMSTP:Port 2/2 left bridge port 2/1-4 2000 Jan 17 22:56:13 %PAGP-5-PORTFROMSTP:Port 2/3 left bridge port 2/1-4 2000 Jan 17 22:56:36 %PAGP-5-PORTFROMSTP:Port 2/4 left bridge port 2/1-4 Switch-B> (enable) show port channel Port Status Channel Channel Neighbor Neighbor mode status device port ----- ---------- --------- ----------- ------------------------- ---------- 2/1 errdisable on channel 2/2 errdisable on channel 2/3 errdisable on channel 2/4 errdisable on channel ----- ---------- --------- ----------- ------------------------- ---------- Switch-B> (enable) show port Port Name Status Vlan Level Duplex Speed Type ----- ------------------ ---------- ---------- ------ ------ ----- ------------ 2/1 errdisable 1 normal auto auto 10/100BaseTX 2/2 errdisable 1 normal auto auto 10/100BaseTX 2/3 errdisable 1 normal auto auto 10/100BaseTX 2/4 errdisable 1 normal auto auto 10/100BaseTX
Correct the errdisable State
Sometimes, when you try to configure EtherChannel but the ports are not configured in the same way, the ports on one side of the channel shut down. The link lights are yellow on the port. The console indicates this in the show port command output, in which the ports are listed as errdisable. In order to recover, fix the mismatched parameters on the ports that are involved, and then reenable the ports.
Note: The reenablement of the ports is a separate step that you must do if the ports are to become functional again.
In this example, you know that Switch A had a VLAN mismatch. On Switch A, put port 2/4 back into VLAN 1. Then turn on the channel for ports 2/1-4. Switch A does not show that it is connected until you reenable the Switch B ports. After you have fixed Switch A and put it in channelling mode, go back to Switch B and reenable the ports.
Switch-A> (enable) set vlan 1 2/4
VLAN 1 modified.
VLAN 2 modified.
VLAN Mod/Ports
---- -----------------------
1 2/1-24
Switch-A> (enable) set port channel 2/1-4 on
Port(s) 2/1-4 channel mode set to on.
Switch-A> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 notconnect on channel
2/2 notconnect on channel
2/3 notconnect on channel
2/4 notconnect on channel
----- ---------- --------- ----------- ------------------------- ----------
Switch-B> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 errdisable on channel
2/2 errdisable on channel
2/3 errdisable on channel
2/4 errdisable on channel
----- ---------- --------- ----------- ------------------------- ----------
Switch-B> (enable) set port enable 2/1-4
Ports 2/1-4 enabled.
Switch-B> (enable)
2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/1 joined bridg4
2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/1-4
2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/3 joined bridge port 2/1-4
2000 Jan 17 23:15:22 %PAGP-5-PORTTOSTP:Port 2/4 joined bridge port 2/1-4
Switch-B> (enable) show port channel
Port Status Channel Channel Neighbor Neighbor
mode status device port
----- ---------- --------- ----------- ------------------------- ----------
2/1 connected on channel
2/2 connected on channel
2/3 connected on channel
2/4 connected on channel
----- ---------- --------- ----------- ------------------------- ----------
Show What Happens When a Link Breaks and Is Restored
When a port in the channel goes down, any packets that are normally sent on that port are shifted over to the next port in the channel. You can issue the show mac command in order to verify that this occurs. In this test bed, Switch A sends ping packets to Switch B in order to determine which link the traffic uses. The procedure is:
-
Clear the counters.
-
Issue the show mac command.
-
Send three pings.
-
Issue the show mac command again in order to determine on which channel the ping responses were received.
Switch-A> (enable) clear counters This command will reset all MAC and port counters reported in CLI and SNMP. Do you want to continue (y/n) [n]? y MAC and Port counters cleared. Switch-A> (enable) show port channel Port Status Channel Channel Neighbor Neighbor mode status device port ----- ---------- --------- ----------- ------------------------- ---------- 2/1 connected on channel WS-C5505 066509957(SW 2/1 2/2 connected on channel WS-C5505 066509957(SW 2/2 2/3 connected on channel WS-C5505 066509957(SW 2/3 2/4 connected on channel WS-C5505 066509957(SW 2/4 ----- ---------- --------- ----------- ------------------------- ---------- Switch-A> (enable) show mac Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast -------- -------------------- -------------------- -------------------- 2/1 0 18 0 2/2 0 2 0 2/3 0 2 0 2/4 0 2 0 Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive Switch-A> (enable) show mac Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast -------- -------------------- -------------------- -------------------- 2/1 3 24 0 2/2 0 2 0 2/3 0 2 0 2/4 0 2 0At this point, ping responses are received on port 3/1. When the Switch B console sends a response to Switch A, the EtherChannel uses port 2/1.
-
Shut down port 2/1 on Switch B.
-
From Switch A, issue another ping and determine on which channel the response comes back.
Note: Switch A sends on the same port to which Switch B is connected. Only the received packets from Switch B are shown because the transmit packets appear later in the show mac command output.
1999 DEC 19 01:30:23 %PAGP-5-PORTFROMSTP:Port 2/1 left bridge port 2/1-4 Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive Switch-A> (enable) show mac Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast -------- -------------------- -------------------- -------------------- 2/1 3 37 0 2/2 1 27 0 2/3 0 7 0 2/4 0 7 0
Now that port 2/1 is disabled, EtherChannel automatically uses the next port in the channel, 2/2.
-
Reenable port 2/1, and wait for it to join the bridge group.
-
Issue two more pings.
1999 DEC 19 01:31:33 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1-4 Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive Switch-A> (enable) ping 172.16.84.17 172.16.84.17 is alive Switch-A> (enable) show mac Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast -------- -------------------- -------------------- -------------------- 2/1 5 50 0 2/2 1 49 0 2/3 0 12 0 2/4 0 12 0
Note: These pings are sent from port 2/1. When the link comes back up, EtherChannel again adds it to the bundle and uses it. All this is done transparently to the user.
Connectivity Issue with Channel Down after Supervisor Replacement
The EtherChannel can go down if the correct procedure is not followed while you replace a Supervisor module and the connected device has errdisable enabled. This usually happens when cables are connected to the new Supervisor module before it is configured for EtherChannel. Therefore, the connected device configured for errdisable detects the port channel misconfiguration and puts its ports in the errdisable state. This causes the connectivity issue. The channel does not come on again until you issue the set port enable command on the connected device.
In order to avoid the port channel misconfiguration, always follow these steps when you replace a Supervisor module that has EtherChannel configurations:
-
Unplug all the cables from the Supervisor that you want to replace.
-
Replace the Supervisor with the new Supervisor.
-
Configure the new Supervisor module for EtherChannel.
-
Connect the cables.
Bandwidth Is Limited to 1 Gbps When WS-X6148-GE-TX Ports Are Used in the Channel
The WS-X6148-GE-TX and WS-X6148V-GE-TX modules do not support more than 1 Gbps of traffic per EtherChannel. On these modules, there is a single 1-Gigabit Ethernet uplink from the port application-specific integrated circuit (ASIC) that supports eight ports. For EtherChannel, the data from all the links in a bundle go to the port ASIC, even though the data are destined for another link. These data consume bandwidth in the 1-Gigabit Ethernet link. For these modules, the sum total of all data on an EtherChannel cannot exceed 1 Gbps. As a result, they should only be used in port channels for link redundancy purposes. If they are included in any Gigabit EtherChannels, the entire channel is limited to 1 Gbps of bandwidth. You also see a warning message that is similar to this one:
Adding a WS-X6148-GE-TX port to a channel limits the channel's bandwidth to a maximum of 1Gig throughput
Commands Used in This Document
Commands to Set the Configuration
-
set port channel on—Turns on the EtherChannel feature.
-
set port channel auto—Resets the ports to their default mode of auto.
-
set port channel desirable—Sends PAgP packets to the other side that request that a channel be created.
-
set port enable—Enables the ports after the set port disable command is issued or after an errdisable state.
-
set port disable—Disables a port during other configuration settings.
-
set trunk desirable—Turns on trunking by causing this port to send to the other switch a request that this be a trunk link. Also, if the port is set to negotiate (the default setting), it requests to negotiate the type of trunking to use on the link (ISL or 802.1Q).
Commands to Verify the Configuration
-
show version—Displays the version of software that the switch runs.
-
show module—Displays the modules that are installed in the switch.
-
show port capabilities—Determines if the ports that you want to use have EtherChannel capabilities.
-
show port—Determines the status of the port (notconnect or connected) as well as the speed and duplex settings.
-
ping—Tests connectivity to the other switch.
-
show port channel—Shows the current status of the EtherChannel bundle.
-
show port channel mod/port —Provides a more detailed view of the channel status of a single port.
-
show spantree—Verifies that STP viewed the channel as one link.
-
show trunk—Shows the trunking status of ports.
Commands to Troubleshoot the Configuration
-
show port channel—Shows the current status of the EtherChannel bundle.
-
show port—Determines the status of the port (notconnect or connected) as well as the speed and duplex settings.
-
clear counters—Resets the switch packet counters to 0. The counters are visible with the show mac command.
-
show mac—Shows packets that the switch receives and sends.
-
ping—Tests connectivity to the other switch and generates traffic that appears in the show mac command output.
Commands to Help Create the Troubleshooting Scenarios
-
set vtp domain testDomain—Gives the switch a VTP domain, which is required in order to add VLANs on the switch.
-
set vlan 2 name vlan2—Creates VLAN 2 with a name of "vlan2".
-
set vlan 2 2/4—Moves port 2/4 into VLAN 2.
-
set port channel 2/1-4 desirable—Sends PAgP packets to the other side that request the creation of a channel.
-
set port channel 2/1-4 auto—Resets the ports to their default mode of auto.
-
set port channel 2/1-4 on—Sets the channel mode of these ports to on. No PAgP packets are sent to the other side. This side simply assumes that the other side has formed a channel as well.
-
set vlan 1 2/4—Moves port 2/4 into VLAN 1.
Command Summary
Because this document uses CatOS software version 4.5, the command syntax is taken from the Command Reference for Cisco Catalyst 5000 Series Switches.
| Syntax: | show version |
|---|---|
| As used in this document: | show version |
| Syntax: | show module [mod_num] |
| As used in this document: | show module |
| Syntax: | show port capabilities [mod_num[/port_num]] |
| As used in this document: | show port capabilities |
| Syntax: | show port [mod_num[/port_num]] |
| As used in this document: | show port |
| Syntax: | ping [-s] host[packet_size][packet_count] |
| As used in this document: | ping 172.16.84.17 |
| Syntax: | show port channel [mod] [info | statistics] show port channel mod/port [info | statistics] |
| As used in this document: | show port channel show port channel 2/1 |
| Syntax: | set port disable mod_num/port_num |
| As used in this document: | set port disable 2/1-4 |
| Syntax: | set port channel mod/ports... [on | off | desirable | auto] set port channel admin_group [on | off | desirable | auto] set port channel admin_group mod/ports..[on | off | desirable | auto] |
| As used in this document: | set port channel 2/1-4 on set port channel 2/1-4 auto set port channel 2/1-4 desirable |
| Syntax: | set port enable mod_num/port_num |
| As used in this document: | set port enable 2/1-4 |
| Syntax: | show spantree [vlan | mod_num/port_num] [active] |
| As used in this document: | show spantree |
| Syntax: | show trunk [mod_num[/port_num]] [detail] |
| As used in this document: | show trunk 2 |
| Syntax: | set trunk mod_num/port_num [on | off | desirable | auto | nonegotiate] [vlan_range] [isl | dot1q | dot10 | lane | negotiate] |
| As used in this document: | set trunk 2/1 desirable |
| Syntax: | set vtp [domain domain_name] [mode {client | server | transparent}] [passwd passwd][pruning {enable | disable}] [v2 {enable | disable}] |
| As used in this document: | set vtp domain testDomain |
| Syntax: | set vlan vlan_num mod_num/port_list set vlan vlan_num [name name] [type {ethernet | fddi | fddinet | trcrf | trbrf}] [state {active | suspend}] [said said] [mtu mtu] [ring hex_ring_number ] [decring decimal_ring_number ] [bridge bridge_num] [parent vlan_num] [mode {srt | srb}] [stp {IEEE | ibm | auto}] [translation vlan_num] [backupcrf {off | on}] [aremaxhop hop_count] [stemaxhop hop_count] |
| As used in this document: | set vlan 2 name vlan2 set vlan 2 2/4 |
| Syntax: | clear counters |
| As used in this document: | clear counters |
| Syntax: | show mac [mod_num[/port_num]] |
| As used in this document: | show mac |
Appendix A: Ethernet Crossover Cables
These cables are available from most computer stores. Also, you can make your own. These two images show the pinouts that are required for a switch-to-switch crossover cable:
Related Information
- Configuring Fast EtherChannel and Gigabit EtherChannel
- Understanding EtherChannel Load Balancing and Redundancy on Catalyst Switches
- Best Practices for Catalyst 4500/4000, 5500/5000, and 6500/6000 Series Switches Running CatOS Configuration and Management
- Switches Product Support
- LAN Switching Technology Support
- Technical Support & Documentation - Cisco Systems
Feedback