Table of Contents
Introduction
The Cisco Catalyst 6500 Series Virtual Switching System (VSS) 1440 is a new and exciting innovation on the Cisco® Catalyst® 6500 Series switches that effectively allows the clustering of two or more physical chassis together into a single, logical entity. This technology allows for new enhancements in all areas of network design, including high availability, scalability, management, and maintenance.
This paper analyzes the Cisco VSS technology, including its benefits and requirements, and highlights potential deployment caveats you should consider before deploying Cisco Virtual Switching System.
Cisco Catalyst 6500 Series Virtual Switching System 1440: An Overview
The initial release of Cisco Catalyst 6500 Series Virtual Switching System (VSS) 1440 allows for the merging of two physical Cisco Catalyst 6500 Series switches together into a single, logically managed entity. Figure 1 graphically represents this concept, where you can manage two Cisco Catalyst 6509 chassis as a single, 18-slot chassis after enabling Cisco Virtual Switching System.
Figure 1. Cisco Virtual Switching System
The key enabler of the Cisco Virtual Switching System technology is a special link that binds the two chassis together, called a Virtual Switch Link (VSL). The VSL carries special control information and encapsulates every frame with a header that passes across this link.
Cisco Catalyst 6500 Series Virtual Switching System 1440 Architecture
The Cisco Catalyst 6500 Series Virtual Switching System 1440 allows for the combination of two switches into a single, logical network entity from the network control-plane and management perspectives. To the neighboring devices, the Cisco Virtual Switching System appears as a single, logical switch or router.
Within the Cisco Virtual Switching System, one chassis is designated as the active virtual switch and the other is designated as the standby virtual switch. All control-plane functions, including management (Simple Network Management Protocol [SNMP], Telnet, Secure Shell [SSH] Protocol, etc.), Layer 2 protocols (bridge protocol data units [BPDUs], protocol data units [PDUs], Link Aggregation Control Protocol [LACP], etc.), Layer 3 protocols (routing protocols, etc.), and software data path are centrally managed by the active supervisor engine of the active virtual switch chassis. The supervisor engine on the active virtual switch is also responsible for programming the hardware forwarding information onto all the distributed forwarding cards (DFCs) across the entire Cisco Virtual Switching System as well as the policy feature card (PFC) on the standby virtual switch supervisor engine.
Figure 2. Components of Cisco Virtual Switching System
From data-plane and traffic-forwarding perspectives, both switches in the Cisco Catalyst 6500 Series Virtual Switching System 1440 actively forward traffic. The PFC on the active virtual switch supervisor engine performs central forwarding lookups for all traffic that ingresses the active virtual switch, whereas the PFC on the standby virtual switch supervisor engine performs central forwarding lookups for all traffic that ingresses the standby virtual switch.
Additionally, all DFCs across the entire Cisco Virtual Switching System can also simultaneously perform packet lookups. As a result, the Cisco Virtual Switching System in aggregate offers greater than 800 Mpps of IPv4 lookup performance. Because the switch fabrics of both switches are also in an active state, the Cisco Virtual Switching System in aggregate has the switch fabric capacity of 1440 Gbps, or 1.44 Tbps.
Centralized Management
The fundamental design of a Cisco Catalyst 6500 Series Virtual Switching System 1440 allows for the centralized management of all network and device resources, including Layer 3 protocols (Open Shortest Path First [OSPF], Enhanced Interior Gateway Routing Protocol [EIGRP], Border Gateway Protocol [BGP], etc.) and Layer 2 protocols (Spanning Tree Protocol, Unidirectional Link Detection Protocol [UDLD], Flow Control, LACP, etc.). A single supervisor engine in the Cisco Virtual Switching System is elected as the central management point for the entire system.
The chassis containing the supervisor engine acting as the single management point is referred to as the active virtual switch. The peer chassis is referred to as the standby virtual switch. The single supervisor engine acting as the single management point is referred to as the active supervisor engine, and the peer supervisor engine in the standby virtual switch chassis is referred to as the hot-standby supervisor engine. You can verify this setup with the following commands:
vss#show switch virtual
Switch mode : Virtual Switch
Virtual switch domain number : 200
Local switch number : 1
Local switch operational role: Virtual Switch Active
Peer switch number : 2
Peer switch operational role : Virtual Switch Standby
vss#show switch virtual redundancy
My Switch Id = 1
Peer Switch Id = 2
Last switchover reason = none
Configured Redundancy Mode = sso
Operating Redundancy Mode = sso
Switch 1 Slot 5 Processor Information :
-----------------------------------------------
Current Software state = ACTIVE
Uptime in current state = 3 weeks, 4 days, 9 minutes
Image Version = Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN_DBG-M), Version 12.2(SIERRA_INTEG_070502) INTERIM SOFTWARE
Synced to V122_32_8_11, 12.2(32.8.11)SR on rainier, Weekly 12.2(32.8.11)SX76
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 03-May-07 09:46 by kchristi
BOOT = sup-bootdisk:s72033-adventerprisek9_wan_dbg-mz.SIERRA_INTEG_070502,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2102
Fabric State = ACTIVE
Control Plane State = ACTIVE
Switch 2 Slot 5 Processor Information :
-----------------------------------------------
Current Software state = STANDBY HOT (switchover target)
Uptime in current state = 3 weeks, 4 days, 8 minutes
Image Version = Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN_DBG-M), Version 12.2(SIERRA_INTEG_070502) INTERIM SOFTWARE
Synced to V122_32_8_11, 12.2(32.8.11)SR on rainier, Weekly 12.2(32.8.11)SX76
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 03-May-07 09:46 by kchristi
BOOT = sup-bootdisk:s72033-adventerprisek9_wan_dbg-mz.SIERRA_INTEG_070502,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2102
Fabric State = ACTIVE
Control Plane State = STANDBY
Router MAC Addresses
Router MAC addresses are assigned to Layer 3 interfaces (physical interfaces or VLANs). They are used primarily to address the Layer 2 fields of the interface for communications, but are also fundamental for the device to perform a Layer 3 lookup; Layer 3 lookups are initiated only if the destination MAC address of the frame is equal to the router MAC address of the interface.
In a standalone Cisco Catalyst 6500, the router MAC address is derived from the MAC erasable electronic programmable read only memory (EEPROM) that is embedded in each Cisco Catalyst 6500 chassis. In a Cisco Virtual Switching System environment, because two physical chassis form the single, logical device, the router MAC addresses must be consistent across both physical chassis. Therefore, the assignment of the router MAC address varies in a Cisco Virtual Switching System environment.
In a Cisco Virtual Switching System environment, the router MAC address assigned to the entire Cisco Virtual Switching System is the router MAC address derived from the MAC EEPROM of the active virtual switch chassis upon initial system activation. When the virtual switch transitions to active state, it assigns all its Layer 3 interfaces with its own router MAC address local to its MAC EEPROM. When the standby virtual switch is brought online after VSL activation, it also derives its router MAC addresses from the MAC EEPROM of the active virtual switch. From this point onward, even if a switchover occurs between the virtual switches (causing a role change), the MAC address remains consistent (Figure 3).
Figure 3. MAC Address Synchronization Across Cisco Virtual Switching System
If the entire Cisco Virtual Switching System is restarted and brought online again but the peer switch assumes the active virtual switch role on activation, the router MAC address changes. In most environments, this change does not represent a problem because gratuitous Address Resolution Protocol (ARP) frames advertising the new router MAC addresses are transmitted upon interface initialization. If you have devices that do not interpret gratuitous ARP frames in your network, you should configure a static router MAC address on the interface:
vss#conf t
Enter configuration commands, one per line. End with CNTL/Z.
vss(config)#int te2/1/2
vss(config-if)#mac-address 000f.f8aa.9c00
vss(config-if)#^Z
vss-demo-1#
*Jul 12 06:49:56.441: %SYS-5-CONFIG_I: Configured from console by console
vss#sh int ten 2/1/2 | include address
Hardware is C6k 10000Mb 802.3, address is 000f.f8aa.9c00 (bia 000f.f8aa.9c00)
If you issue this command and save the configuration, the Cisco Virtual Switching System will always use the manually configured address, regardless of the role it assumes.
Virtual Switch Link
The Cisco Catalyst 6500 Series Virtual Switching System 1440 consists initially of two Cisco Catalyst 6500 chassis. In order to bond the two chassis together into a single, logical node, special signaling and control information must be exchanged between the two chassis in a timely manner. To facilitate this information exchange, you need a special link to transfer both data and control traffic between the peer chassis. This link is referred to as the virtual switch link (VSL). (Refer to Figure 4.)
The VSL, formed as a Cisco EtherChannel® interface, can comprise links ranging from 1 to 8 physical member ports. These links carry two types of traffic: the Cisco Virtual Switching System control traffic and normal data traffic. In order to ensure that control traffic gets highest priority across the VSL, a special bit is set on all VSL control frames to help ensure that these frames always get priority service from both ingress and egress hardware queues. From a data-plane perspective, the VSL is used to extend the internal chassis data path to the neighboring chassis. Data traffic sent on the VSL is load balanced using the regular Cisco EtherChannel load-balancing algorithms.
All frames that are sent across the VSL are encapsulated with a virtual switch header (VSH), which is appended to the frame by the egress port application-specific integrated circuit (ASIC) and striped off on the other side of the VSL by the ingress port ASIC. The VSH carries information such as the ingress port index, destination port index, VLAN, class of service (CoS), etc. The size of the VSH is the same as that of the internal compact header used by the Cisco Catalyst 6500; it is 32 bytes long. This header is placed after the Ethernet preamble and directly before the Layer 2 header.
Figure 4. Virtual Switch Header
Because a VSH is appended onto every frame that is sent across the VSL, there are also new port ASIC requirements to be able to form the VSL. The interfaces that can currently form the VSL are the 10 Gigabit Ethernet uplink interfaces on the Cisco Catalyst 6500 Series Supervisor Engine 720-10G VSS as well as those on the 8-port 10 Gigabit Ethernet line card (WS-X6708-10G-3C/XL).
The VSL is both the enabling technology of the Cisco Virtual Switching System and a critical link of the system. Internal Cisco Catalyst 6500 control information that is usually retained within the chassis must now be exchanged across the VSL to the peer switch, extending the backplane between the two switches. You should, therefore, always provision the VSL to contain at least two member links, which should be deployed along different physical paths or conduits. You should minimize data traffic use of the VSL wherever possible. Refer to the section "Cisco EtherChannel Concepts" for more details.
Control-Plane Communication
The VSL is crucial for both CPUs in each supervisor engine to communicate with each other. It is also used to determine which virtual switch becomes the active virtual switch and which becomes the standby virtual switch. Because this determination affects the behavior of each switch, the roles must be negotiated very early during the chassis bootup cycle. As a result, the system must bring the VSL and its associated ports online before initializing the rest of the system.
Communication between the two chassis is facilitated with internal messaging that is sent across the VSL. Because the VSL is implemented as a Cisco EtherChannel interface, it is resilient to single-link failures. However, realistically only a single link of the VSL is chosen as the control link at any given time because the hash algorithm of the Cisco EtherChannel interface is based on the source and destination MAC addresses, which are always the same for each CPU.
Virtual Switch Link Initialization
The system must bring the VSL online before activating the Cisco Virtual Switching System. The initialization sequence consists of the following steps:
1. Link initialization
The VSL link initialization occurs very early in the system boot cycle-it actually occurs even before the configuration file is parsed and the system is initialized. To determine which ports form members of the VSL, the configuration is preparsed to extract the appropriate VSL commands and their associated interfaces, so that the modules containing these interfaces can be powered up, diagnostics run, and VSL interfaces brought online.
The Link Management Protocol (LMP) operates on each link of the VSL and is part of the Virtual Switch Link Protocol (VSLP). It performs the following functions:
• Identifies and rejects unidirectional links
• Exchanges switch IDs between the two chassis
• Exchanges other information required to establish communication between the two chassis
2. Role resolution
The role of each physical chassis is resolved by another protocol that forms part of VSLP-the Role Resolution Protocol (RRP), which performs the following functions:
• Determines whether the hardware and software versions allow a Cisco Virtual Switching System to be formed
• Determines which chassis will become the active virtual switch and which will become the standby virtual switch chassis from a control-plane perspective.
3. High-availability role determination
After the role resolution, the active and standby image versions and configurations are checked for compatibility, helping ensure that the hardware and software versions are the same on both chassis supervisor engines. The configuration check helps ensure that the VSL-related configurations on the two switches are compatible. If either of the two checks fails, then the standby chassis comes up in route-processor redundancy (RPR), mode where all modules are powered down, as opposed to Nonstop Forwarding/Stateful Switchover (NSF/SSO) mode, where the standby chassis is fully initialized and can forward traffic.
An example of how configuration checking may force the system into RPR mode is provided in the following output:
*Jun 29 14:05:44.731: %VSLP-SW2_SP-5-RRP_ROLE_RESOLVED: Role resolved as ACTIVE by VSLP
*Jun 29 14:05:44.735: %VSL-SW2_SP-5-VSL_CNTRL_LINK: vsl_new_control_link NEW VSL Control Link 5/4
*Jun 29 14:05:44.735: %VSL-SW2_SP-2-VSL_STATUS: -======== VSL is UP ========-
*Jun 29 14:08:22.294: %VS_PARSE-3-CONFIG_MISMATCH: The system:/running-config VSL config comparison failed
Switch 2 has the following configs that mismatch with Switch 1:
Interface TenGigabitEthernet1/5/4 shutdown
*Jun 29 14:08:22.210: SW2_SP: VS_PARSE_DBG_ERR: vs_redun_send_check_vs_config: icc_req_resp_timeout_and_success: Failed
*Jun 29 14:08:22.210: SW2_SP: VS_PARSE_DBG: vs_redun_check_vs_config: running config check on rp not ok
*Jun 29 14:08:22.218: %PFREDUN-SW2_SP-6-ACTIVE: Standby initializing for RPR mode
This output shows that the configuration consistency check failed because of a mismatch in the VSL configuration between switch 1 and switch 2. In this case, switch 2 has an extra "shutdown" statement under one of its VSL members, whereas switch 1 does not, forcing the standby virtual switch (in this case switch 1) into RPR mode.
In order to recover from this situation, make any necessary changes to the configuration, save the configuration, and reload the standby chassis:
vss#conf t
Enter configuration commands, one per line. End with CNTL/Z.
vss(config)#int te1/5/4
vss(config-if)#no shut
vss(config-if)#^Z
vss#wr
Building configuration...
*Jun 29 14:28:53.906: %SYS-5-CONFIG_I: Configured from console by console
*Jun 29 14:29:04.834: %PFINIT-SW2_SP-5-CONFIG_SYNC: Sync'ing the startup configuration to the standby Router. [OK]
vss#redundancy reload shelf 1
Reload the entire remote shelf[confirm]
Preparing to reload remote shelf
Upon reload of switch 1, you should be able to observe that configurations are not synchronized and both switches can enter into NSF/SSO mode:
*Jun 29 14:40:46.101: VS_PARSE_DBG: vsl_mgr_parse_config_file: vsl_mgr_parse_config_file:Open Succeeded for running config system:/running-config
*Jun 29 14:40:46.029: SW2_SP: VS_PARSE_DBG: vs_redun_check_vs_config: running config check on rp ok
*Jun 29 14:40:46.037: %PFREDUN-SW2_SP-6-ACTIVE: Standby initializing for SSO mode
*Jun 29 14:40:49.874: %PFINIT-SW2_SP-5-CONFIG_SYNC: Sync'ing the startup configuration to the standby Router.
Cisco Catalyst 6500 Series Virtual Switching System 1440 Hardware Requirements
Specific hardware is required to enable the Cisco Virtual Switching System feature; it exists in the form of the supervisor engine and system forwarding engines, the VSL-capable modules, and modules that may exist in a Cisco Virtual Switching System-enabled system.
Forwarding Engine
The forwarding engine of the Cisco Catalyst 6500 may exist in the form of a PFC on the supervisor engine or a DFC that is installed on the individual line cards. These forwarding engines perform lookup functions for every frame that enters into the system and can determine the ultimate destination of the packet as well as providing value-added services such as security access control list (ACL) and quality of service (QoS) lookups.
A standalone Cisco Catalyst 6500 and a Cisco Virtual Switching System-enabled Cisco Catalyst 6500 have two notable differences:
• Both the active and the hot-standby supervisor engine PFCs are active and are used to perform packet lookups for centralized lookup on each chassis.
• All forwarding engines are required to cater for an increased amount of port index information to be able to address a fully populated Cisco Virtual Switching System-enabled chassis. In the initial release of software, this requirement generally indicates a requirement to address twice the number of physical ports.
As a result, a new system forwarding engine mode is required to enable the Cisco Virtual Switching System capabilities. This new mode is the Policy Feature Card 3C (PFC3C) or Policy Feature Card 3CXL (PFC3CXL) mode. You can configure the operating mode of the system with the following command:
vss#sh platform hardware pfc mode
PFC operating mode : PFC3C
Configured PFC operating mode : None
Because the system also operates in a lowest-common-denominator mode, it is important to ensure that all other forwarding engines in the chassis are either at PFC3C mode or above (PFC3CXL). If a lower-mode module was previously inserted into the chassis that forced the system mode of operation to PFC3A, PFC3B, or PFC3BXL mode, then the Cisco Virtual Switching System function will not be enabled on the system. Likewise, if a module with a lower-mode DFC is inserted into the chassis after conversion to Cisco Virtual Switching System mode, the system will not grant power to the module.
Additionally, the supervisor engines of both chassis may prenegotiate their modes to be in PFC3CXL mode during VSL initiation. If modules are installed in a system that contains a DFC3C, then the module may never be allowed to power up. This example is further highlighted by the case where the VSL module itself is a DFC3C, but the supervisor engine has a PFC3CXL. In this case, the VSL-enabled module may never power up. To avoid this situation, configure the following global command if installation of a mix of PFC3C and PFC3CXL modules is possible:
vss(config)#platform hardware vsl pfc mode pfc3c
System EARL mode will be forced to PFC3C on next bootup
vss#sh platform hardware pfc mode
PFC operating mode : PFC3C
Configured PFC operating mode : PFC3C
Virtual Switch Link-Capable Interfaces
VSL-capable interfaces are required to create a VSL port channel. The current modules that support the formation of a VSL port channel are the 10 Gigabit Ethernet uplink ports on the Supervisor Engine 720-10G VSS (VS-S720-10G-3C/XL) and the Cisco 8-Port 10 Gigabit Ethernet Switching Module (WS-X6708-10G-3C/XL). (Refer to Figure 5.)
Figure 5. VSL-capable Modules
These interfaces contain new port ASICs that allow the VSH to be encapsulated on each frame forwarded out of the port and also support the ability to de-encapsulate VSH-tagged frames.
Supported Chassis
From a chassis perspective, both E-Series chassis and non E-Series chassis are supported within a Cisco Virtual Switching System environment, with the exception of the Cisco Catalyst 6503 (non E-Series) and Cisco Catalyst 6509-NEB (non E-Series). Table 1 gives a complete list of the chassis supported with the initial release of Cisco Virtual Switching System:
Table 1. Chassis Supported by Cisco Virtual Switching System
It should be noted that there is no requirement that the two members of the Cisco Virtual Switching System use the same chassis type. The members consisting of the Cisco Virtual Switching System can be different chassis with varying slot counts.
Additionally, note that no Cisco 7600 Series chassis will be supported after the system is converted to Cisco Virtual Switching System mode.
Supervisor Engine
The supervisor engine that supports the Cisco Virtual Switching System feature is the Supervisor Engine 720-10G VSS (VS-S720-10G-3C/XL). This supervisor engine has both the PFC3C/XL- as well as the VSL-capable interfaces integrated on the module (Figure 6).
Figure 6. Supervisor Engine 720-10G VSS
The initial release of the Cisco Virtual Switching System supports only one Supervisor Engine 720-10G VSS per chassis. A subsequent software release will allow for redundant supervisors to exist in a single physical chassis within the VSS.
Other Supported Modules
You can use all other modules in the chassis to connect to other network nodes or devices, including interface modules or integrated services modules.
Supported interface modules that can coexist within a Cisco Virtual Switching System-enabled chassis include all CEF720 modules (WS-X6700 series). These modules can also support either a centralized forwarding card (CFC) or a DFC. If a DFC is installed, it must either be DFC3C or DFC3CXL. A lower-mode DFC inserted in the module will be denied system power until the system-wide PFC mode has been configured and the system is reloaded.
Cisco Catalyst 6500 Series Virtual Switching System Hardware Deployment Recommendations
You can deploy Cisco Virtual Switching System in your network in numerous ways. In order to maximize system availability and capacity, note the following recommendations with their associated benefits and caveats.
Two-Port VSL Using Supervisor-Engine Uplinks
In this scenario, the two members of the Cisco Virtual Switching System are connected through a 2-port VSL bundle. Figure 7 shows the VSL being formed out of the two 10 Gigabit Ethernet uplink ports on the Supervisor Engine 720-10G VSS.
Figure 7. VSL Formed out of Two 10 Gigabit Ethernet Uplink Ports
This deployment scenario allows for a redundant VSL connection and also depends on fewer hardware components. However, if a VSL module fails (in this case, it is the supervisor engine), the entire VSL also will fail, potentially resulting in a dual-active scenario. The effect is mitigated, however, because redundant supervisor engines are not supported in the initial software release of Cisco Virtual Switching System. Note that this scenario does not allow for future scaling of VSL bandwidth because no extra VSL-capable 10 Gigabit Ethernet interfaces are available.
Two-Port VSL Using Line Cards Only
Figure 8 shows the two members of the Cisco Virtual Switching System connected through a 2-port VSL bundle, but takes full advantage of line cards instead of the supervisor-engine uplinks.
Figure 8. VSL Comprising of Interfaces on the Cisco 8-Port 10 Gigabit Ethernet Switching Module
This deployment scenario allows for a redundant VSL connection as well, but uses the two Cisco 8-port 10 Gigabit Ethernet modules if the uplinks on the supervisor engine are already in use. As a result, the scenario depicted in Figure 8 involves more hardware, but it is highly redundant: the VSL can survive both a physical link failure as well as a complete VSL module failure. This solution also provides for future VSL bandwidth scalability.
Two-Port VSL Using Supervisor Engine and Line Card
Figure 9 shows the two members of the VSS connected through a 2-port VSL bundle; this scenario uses both supervisor-engine uplinks and a single 8-port 10 Gigabit Ethernet line card for bandwidth and redundancy.
Figure 9. VSL Members Across Supervisor Engine Ports and 8-Port 10 Gigabit Ethernet Switching Module
This deployment scenario, which allows for a redundant VSL connection, may require more hardware than using a single supervisor engine, but offers both link and line-card redundancy as well as the added benefit of future scaling requirements if you need extra VSL bandwidth in the future.
Multiple Cisco Virtual Switching System Domains
Multiple deployments of Cisco Virtual Switching System can exist in a given network design, adding to the availability and scalability of the network. As a result, Cisco recommends that you use unique virtual switch domain identifiers for each pair of VSS switches.
Figure 10 shows an example of multiple Cisco Virtual Switching System domains in a network design. The figure shows three unique VSS domains, each with a unique domain ID. You can also deploy multichassis Cisco EtherChannel links across other Cisco Virtual Switching Systems, removing the reliance on protocols such as Spanning Tree Protocol.
Figure 10. Multiple Cisco Virtual Switching System Domains
Another example of multiple Cisco Virtual Switching System domains is in the area of Layer 2 adjacent WAN deployments. It may be a requirement for the business or the applications that a routed Layer 3 WAN connection may not be possible requiring a Layer 2 connection between two disparate geographic sites, yet still providing link redundancy at the same time. This will ultimately require some form of Layer 2 redundancy protocol be implemented (such as Spanning Tree Protocol), resulting in complex topologies as well as inefficient bandwidth utilization across network links. By leveraging Cisco Virtual Switching System, such inefficiencies will be mitigated through the formation of multichassis Cisco EtherChannel connections.
Cisco EtherChannel Concepts
Cisco EtherChannel interfaces on the Cisco Catalyst 6500 platform represent a grouping of one or more physical ports into a single, logical port from the perspective of either a Layer 2 switching or Layer 3 routing environment. Cisco EtherChannel interfaces allow for individual link resiliency as well as providing added bandwidth without the necessity of complex protocols.
There are generally no restrictions with regard to which ports or modules can form members of a Cisco EtherChannel link, except that the member interfaces need to be of the same speed and no more than 8 members can belong to a single Cisco EtherChannel grouping. You can, therefore, extend members of the Cisco EtherChannel interface across switching modules to allow for the maximum availability of the Cisco EtherChannel interface if either a single link or module fails.
Traffic Distribution and Hashing
The distribution of traffic across the various members of the Cisco EtherChannel link is accomplished through different hash schemes, each using a fixed set of fields within the frame to determine which Cisco EtherChannel member is used to forward a particular traffic flow. With the PFC3C running Cisco IOS® Software Release 12.2(33)SXH, you can choose from 13 possible different hash schemes:
vss(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
dst-mixed-ip-port Dst IP Addr and TCP/UDP Port
dst-port Dst TCP/UDP Port
mpls Load Balancing for MPLS packets
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-dst-mixed-ip-port Src XOR Dst IP Addr and TCP/UDP Port
src-dst-port Src XOR Dst TCP/UDP Port
src-ip Src IP Addr
src-mac Src Mac Addr
src-mixed-ip-port Src IP Addr and TCP/UDP Port
src-port Src TCP/UDP Port
Selection of the hash scheme of choice largely depends on the traffic mix through the Cisco EtherChannel interface, noting that these hash schemes may be selected only on a global basis.
Determination of Hash Result
With the release of Cisco Virtual Switching System, a new mechanism has been implemented to allow you to determine which physical link a given flow of traffic uses within a port-channel group. You provide inputs to the command and the hashing algorithm computes the physical link that is selected for the traffic mix and algorithm.
vss#sh etherchannel load-balance hash-result ?
interface Port-channel interface
ip IP address
ipv6 IPv6
l4port Layer 4 port number
mac Mac address
mixed Mixed mode: IP address and Layer 4 port number
mpls MPLS
vss#sh etherchannel load-balance hash-result interface port-channel 120 ip 192.168.220.10 192.168.10.10
Computed RBH: 0x4
Would select Gi1/2/1 of Po120
Adaptive Load Balancing
The addition or removal of a member port from a Cisco EtherChannel interface has always led to a varied amount of traffic loss for customers. The current generation of port ASICs uses a 3-bit Result Bundle Hash (RBH) value from the PFC or DFC result to index into a load register to allow a packet to be transmitted if the corresponding bit is set.
When a new port is added or deleted, the load value is reset on all the ports. A new load is then distributed on all the ports in the Cisco EtherChannel interface, including the new member, and reprogrammed into the port ASIC for each port. This process causes packets to be dropped during the short outage window (approximately 200 to 300 ms), an undesirable result for higher-speed interfaces such as 10 Gigabit Ethernet connections where a large amount of traffic may be lost during this brief outage window.
This problem has led to the development of an enhanced load-distribution mechanism such that when ports are added or removed from a Cisco EtherChannel interface, the load result does not need to be reset on existing member ports, resulting in better traffic recovery times.
You can implement this new algorithm either globally or on a per-port channel basis, where fixed is the current default mode and adaptive uses the enhanced mode:
vss(config)#port-channel hash-distribution ?
adaptive selective distribution of the bndl_hash among port-channel members
fixed fixed distribution of the bndl_hash among port-channel members
vss(config)#int port-channel 4
vss(config-if)#port-channel port hash-distribution ?
adaptive selective distribution of the bndl_hash among port-channel members
fixed fixed distribution of the bndl_hash among port-channel members
The algorithm selected with these commands is applied only at the next hash-distribution instance, which usually occurs on a port-channel member link transition event.
vss#sh etherchannel 4 summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use N - not in use, no aggregation
f - failed to allocate aggregator
M - not in use, no aggregation due to minimum links not met
m - not in use, port not aggregated due to minimum links not
met
u - unsuitable for bundling
d - default port