Guest

Cisco IOS Software Releases 12.0 S

Link Bundling on Cisco 12000 Series Internet Routers

Table Of Contents

Link Bundling on Cisco 12000 Series Internet Routers

Feature Overview

Using EtherChannel Link Bundles

Using POS Channel Link Bundles

Link Bundling in a Cisco 12000 Series Internet Router

Engine 5 Ingress Support

Load-balancing in Cisco 12000 Series Link Bundles

VLAN ID-Based Load-balancing

Software Features Supported on a Link Bundle

Using an EtherChannel for High-Speed Provider Edge Support

Any Transport over MPLS—Ethernet over MPLS

Stacked VLAN Processing—802.1Q-in-Q

Feature Parity on VLAN and Stacked VLAN Member Interfaces

MAC Address Accounting

Multicast and Multicast VPN

Policy-Based Routing

Unicast Reverse Path Forwarding

VLAN Scalability

Quality of Service

Feature Parity on POS Channel Bundles

Route Processor Redundancy Plus

Stateful Switchover

Benefits of Link Bundling

Restrictions for Link Bundling

Related Features and Technologies

Related Documents

Supported Platforms

Supported Standards, MIBs, and RFCs

Configuration Tasks

Configuring a Port-Channel Interface

Configuring a POS-Channel Interface

Adding a Fast Ethernet or Gigabit Ethernet Interface

Adding a Packet-over-SONET Interface

Maximizing VLAN TCAM to Support VLAN Scalability

Removing an Ethernet or Packet-over-SONET Interface from a Link Bundle

Configuring Out-of-Service Support

Configuring Bandwidth Propagation

Verifying an EtherChannel

Verifying a POS Channel

Configuration Examples

Configuring an EtherChannel

Configuring a VLAN EtherChannel

Removing an EtherChannel

Configuring a POS Channel

Removing a POS Channel

Configuring Out-of-Service Support

Configuring Bandwidth Propagation

Command Reference

channel-group

hw-module slot linkbundle-tcam-scalability

interface port-channel

interface pos-channel

show interfaces port-channel

show interfaces pos-channel

Glossary


Link Bundling on Cisco 12000 Series Internet Routers


Part Number OL-8696-01, May 30, 2008

Feature History

Release
Modification

12.0(23)S

This feature was introduced on Engine 0, Engine1, and Engine 2 line cards on the Cisco 12000 series Internet router.

12.0(26)S

This feature was introduced on IP Service Engine (ISE/E3) and Engine 4 Plus (E4+) line cards on the Cisco 12000 series Internet router.

The following new features were introduced:

Out-of-service support

Bandwidth propagation support

12.0(32)S

Support was added for the following features in a link bundle that consists only of ISE/E3 or E4+ Gigabit Ethernet (GE) member interfaces in a
Cisco 12000 series Internet router deployed on the provider edge:

Any Transport over MPLS (AToM: Layer 2 Quality of Service, Ethernet over MPLS—VLAN mode only, and Layer 2 Local Switching)

Stacked VLAN (802.1Q-in-Q) Processing

MAC Address Accounting

Multicast and Multicast VPN (Gigabit Ethernet ISE interfaces only)

Policy-Based Routing

Quality of Service (QoS)

Feature parity on VLAN and Stacked VLAN subinterfaces

Unicast Reverse Path Forwarding (Loose Mode)

VLAN scalability

Engine 5 Ingress support was added.
VLAN ID-based mode of load-balancing was added.
route processor redundancy plus (RPR+) support in link bundles was added.

12.0(33)S

Support was added for the following features in an ISE/E3 link bundle that consists of POS or GE member interfaces:

Multicast Egress QoS

QoS hierarchical policing

QoS hierarchical shaping

Unicast Reverse Path Forwarding (Strict Mode)

Support was added for Engine 5 (E5) Fast Ethernet member interfaces in an EtherChannel link bundle.

Engine 5 Egress support was added so that the following features are now supported in an EtherChannel link bundle that consists of E5 Fast Ethernet or Gigabit Ethernet member interfaces:

Any Transport over MPLS (AToM: Layer 2 Quality of Service, Ethernet over MPLS—VLAN mode only, and Layer 2 Local Switching)

Border Gateway Protocol (BGP) policy accounting

Feature parity on VLAN and Stacked VLAN subinterfaces

Hot Standby Router Protocol (HSRP)

MAC Address Accounting

MPLS VPN

Multicast and Multicast VPN

Policy-Based Routing

Quality of Service (QoS), including:

Multicast Egress QoS

QoS hierarchical policing

QoS hierarchical shaping

Route processor redundancy plus (RPR+)

Sampled and Aggregate NetFlow on input and output interfaces

Stacked VLAN (802.1Q-in-Q) Processing

Unicast Reverse Path Forwarding (Loose and Strict Mode)

VLAN ID-based mode of load-balancing

VLAN scalability

Cisco IOS software feature parity between a POS channel that consists of ISE, E4+, E5, or E6 member interfaces and EtherChannel bundles was added. Table 4 lists supported features.

Stateful Switchover (SSO) support in link bundles was added.


This feature module describes how to configure and use the Link Bundling feature on Cisco 12000 series Internet routers. This document includes the following sections:

Feature Overview

Supported Platforms

Supported Standards, MIBs, and RFCs

Configuration Tasks

Configuration Examples

Glossary

Feature Overview

The Link Bundling feature allows you to group multiple point-to-point links together into one logical link to provide higher bidirectional bandwidth (a bigger pipe), redundancy, and load-balancing between two routers. The following types of link bundling are supported on Cisco 12000 series Internet routers:

Fast EtherChannel—Used to bundle multiple E5 Fast Ethernet (FE) interfaces.

Gigabit EtherChannel—Used to bundle multiple Gigabit Ethernet (GE) interfaces.

POS channel—Used to bundle multiple Packet-over-SONET (POS) interfaces.

Managing Bandwidth

All three link bundling methods on Cisco 12000 series Internet routers provide flexible and incremental bandwidth with link redundancy and greater layer transparency to network applications. You can use link bundles in multiple locations in the same network.

Use link bundling on Cisco 12000 series Internet routers in networks under the following conditions:

Faster links do not exist

The next step available for increasing link capacity is expensive

The operational costs to increase link capacity are high

EtherChannels and POS channels allow you to increase and decrease bandwidth by adding or removing an interface from the link bundle. Also, by incrementally increasing bandwidth, you are no longer dependent on the fixed increases in bandwidth (for example, 1 Gbps, 10 Gbps, and so on) determined by the physical layer technology.

Managing Link Failure

The failure of one link does not necessarily cause a network failure. Traffic is redirected to remaining links within the channel without user intervention. As a result, the availability of a FE, GE, or POS link is increased.

On Cisco 12000 series Internet routers, link bundling is implemented so that a virtual interface is created for each link bundle. You can dynamically add and delete links to the virtual interface. The virtual interface is treated as one interface on which you configure an IP address and other software features used by the link bundle, instead of configuring them on individual GE and POS interfaces.

Packets sent to the link bundle are forwarded on one of the links in the bundle. Load-balancing is supported on all links in a bundle using per-destination load-balancing based on a hash calculated using the source and destination IP addresses in the IP packet. Per-destination load-balancing ensures that packets are delivered in order.

For a description of how to use EtherChannel and POS channel link bundling on Cisco 12000 series Internet routers, see the "Using EtherChannel Link Bundles" section and the "Using POS Channel Link Bundles" section.

Link Bundling Features

Beginning in Cisco IOS Release 12.0(26)S, support for the following features is added to link bundling on Cisco 12000 series Internet routers:

Out-of-service support—If the minimum bandwidth is not available on a link bundle, the EtherChannel or POS channel are brought down. The link bundle is brought up again when the specified minimum number of links are active.

Bandwidth propagation—In releases earlier than Cisco IOS Release 12.0(33)S, when the bandwidth used on a EtherChannel or POS channel changes, each bandwidth change is propagated, by default, to the upper-layer protocols. You can now (optionally) control the propagation of bandwidth changes and specify a bandwidth threshold so that the total bandwidth of a link bundle is propagated at each change until it exceeds the threshold value.

For a description of how to configure out-of-service support and bandwidth propagation, see the "Configuring Out-of-Service Support" section and the "Configuring Bandwidth Propagation" section.

Using EtherChannel Link Bundles

EtherChannel link bundles are at Layer 2 and use one MAC address and one IP address for all FE or GE interfaces in the bundle. A Fast or Gigabit EtherChannel is used mostly in intra-POP applications to interconnect multiple routers. Figure 1 shows how to aggregate Gigabit Ethernet links into one Gigabit EtherChannel interface in an intra-POP application.

Figure 1 Gigabit EtherChannel Used in an Intra-POP Application

You can also use an EtherChannel in metropolitan area networks for Ethernet aggregation at the network edge. For example, you can use a Gigabit EtherChannel to connect multiple sites that are moving from using 1 Gbps to 10 Gbps throughput. Bandwidth demand grows at different rates in metropolitan network sites, depending on business needs and data traffic patterns. You can, therefore, use Gigabit EtherChannel to incrementally increase bandwidth in 1-Gbps segments.

Using POS Channel Link Bundles

POS channel link bundles are at Layer 2 and use one logical IP interface. You can use POS channel link bundling in intra-POP applications (Figure 2), and in backbone (Figure 3) and peering connections.

For example, if the fiber infrastructure required to support greater transmission speeds does not exist, bundling together multiple POS connections is useful. A POS channel link bundle provides an alternative way to support OC-48, OC-192, or OC-768 transmission speeds.

Figure 2 POS Channel Used in an Intra-POP Application

Figure 3 POS Channel Used in a Backbone Application

Link Bundling in a Cisco 12000 Series Internet Router

Link bundling is a capability that must be supported on a system-wide level in a Cisco 12000 series Internet router. The Link Bundling feature requires that each line card in the router recognize an EtherChannel or POS channel bundle as a virtual interface.

The successful flow of data packets into and out of a link bundle consists of the following steps:

1. An EtherChannel or POS channel link bundle is created on a line card or across multiple line cards.

2. An adjacency representing the new bundle is created in the forwarding information base (FIB) table on the route processor (RP) and is forwarded to all the line cards. This adjacency represents a virtual link and has pointers to individual links in the bundle.

3. As incoming data packets are received by the router, line cards route packets to the link bundle as a whole. The ability to route and load balance packets across the bundle depends on whether or not a line card recognizes the new virtual link adjacency. The line card that receives incoming packets and decides how to route them to the link bundle is called the ingress line card. The process of making the decision is called ingress decision support.

If the ingress line card:

Supports the ingress decision capability, the line card recognizes the virtual adjacency, and properly routes and load balances the packets across the subadjacencies represented by the virtual adjacency. Packets are properly routed and load balanced towards the bundle, and then properly transmitted across the bundle.

Does not support the ingress decision capability, the line card routes packets to the virtual adjacency, but does not perform load-balancing across the subadjacencies. Packets are sent across only one link in the bundle. As a result, this link can become saturated and packets can be dropped. For a list of the Cisco 12000 series line cards that support and do not support the ingress decision capability, see the "Restrictions for Link Bundling" section.

4. When incoming packets are received by the link bundle, the packets are properly routed to the destination interface in the router.

Engine 5 Ingress Support

Starting in Cisco IOS Release 12.0(32)S, EtherChannel bundles support load-balancing across member links for the following combinations of IP and MPLS data streams when an Engine 5 line card is the ingress interface and an EtherChannel port-channel interface is the egress interface in a Cisco 12000 series Internet router. An Engine 5 ingress interface:

Receives IP packets; a port-channel interface forwards the IP packets.

Receives IP packets; a port-channel interface encapsulates packets with an MPLS label and forwards the packets.

Encapsulates packets with an MPLS label; a port-channel interface swaps or replaces the MPLS label and forwards the MPLS packets.

Receives MPLS packets; a port-channel interface removes the MPLS labels and forwards IP packets.

Configured for AToM encapsulates incoming packets and transmits them across a pseudowire connection; a port-channel interface decapsulates the packets and transmits the Layer 2 packets.

In these conditions, IP and MPLS packets are load-balanced across the member links in an EtherChannel bundle. For more information about MPLS tag switching, refer to MPLS/Tag Switching.

Load-balancing in Cisco 12000 Series Link Bundles

On Cisco 12000 series Internet routers, EtherChannel, and POS channel link bundling provides load-balancing (equal cost) across all active links in the bundle. Load-balancing (also known as Layer 2 load-balancing) is supported on the links using per-destination load-balancing, based on a hash calculated using the source and destination IP addresses in the IP packet. Per-destination load-balancing ensures that packets are delivered in order.

In an EtherChannel or POS channel link bundle, you can only configure interfaces of the same engine and media type. (For more information, see "Line Card Restrictions in Routers on Each Side of a Link Bundle" section.) Each engine type uses a different hash algorithm to perform load-balancing across member interfaces in a link bundle.

Load-balancing logic is built into the CPU software of lower engine types and into the Layer 3 packet-switching ASIC (PSA) driver modules of higher engine types. Load-balancing logic is stored:

In the CPU software on Engine 0 and Engine 1 line cards

On the packet-switching ASIC (PSA) of Engine 2 line cards

On the ALPHA PSA of IP Service Engine (ISE) line cards

On the Rx Plus PSA of Engine 4 Plus (E4+) line cards

On the WAHOO PSAs of Engine 5 (E5) line cards


Note Layer 2 load-balancing is always performed on the ingress line card when a packet is switched.

Equal load-balancing across all links in a bundle occurs only when the source and destination IP addresses in packets vary across a wide range of IP addresses.


Starting in Cisco IOS Release 12.0(32)S, VLAN ID-based load-balancing, is supported. See "VLAN ID-Based Load-balancing" section for more information.

VLAN ID-Based Load-balancing

VLAN ID-based load-balancing is supported on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

E5 Fast Ethernet

ISE (E3) Gigabit Ethernet

E4+ Gigabit Ethernet

E5 Gigabit Ethernet

Outgoing traffic is normally equally balanced across all links in a bundle based on the source and destination IP address of transmitted packets. However, VLAN ID-based load-balancing in a link bundle introduces load-balancing based on the VLAN ID of transmitted packets.

When you configure VLAN ID-based load-balancing, all egress traffic with a specified VLAN ID is internally mapped to outgoing member interfaces through which the traffic is transmitted. To switch between VLAN ID-based and per-destination load-balancing, use the channel-group load-share vlan command in (port-channel) interface configuration mode.

Use VLAN ID-based load sharing in a link bundle in the following configurations:

A customer requires VLAN-specific Quality of Service on egress traffic, which cannot be achieved due to hardware limitations when source- and destination-based load-balancing is enabled.

For Layer 2 traffic or when AToM features are configured on a port-channel interface, Layer 3 parameters, such as source and destination IP address, are not supported. Therefore, we recommend load-balancing based on Layer 2 characteristics.


Note VLAN ID-based load-balancing is effective when there are a large number of (more than 10) VLANs configured on the port-channel interface. When only a few VLANs are configured in an EtherChannel, VLAN ID-based load-balancing is not effective for balancing outgoing traffic across member links.


The VLAN ID-to-interface mapping is performed by assigning the traffic for each VLAN ID according to the load supported on a member interface and the load required by the VLAN. The load required for VLAN traffic is calculated as the bandwidth allocated to the Modified Deficit Round Robin (MDRR) queue. New VLAN traffic is assigned to the member interface transmitting the smallest load. The total load of egress VLAN traffic in a link bundle is balanced among member interfaces. If a change in the number of active member links available for use occurs, the VLAN ID-to-interface mapping is recalculated.

Software Features Supported on a Link Bundle

Sometimes the software features supported on an Fast Ethernet, Gigabit Ethernet, or POS interface are not supported on the entire link bundle because of engineering restrictions. For example, IPv6 is not supported on a link bundle. Configuring an unsupported feature on an EtherChannel or POS channel link-bundle interface may result in unexpected behavior.

On Cisco 12000 series Engine 0 (E0), E1, and E2 line cards, the minimum set of software features supported across an entire link bundle (port-channel or pos-channel and member interfaces) is:

IP unicast

Multiprotocol Label Switching (MPLS)

MPLS virtual private networks (VPNs)

MPLS VPN Inter-Autonomous Systems (Inter-AS)

Weighted Random Early Detection (WRED) and Modified Deficit Round Robin (MDRR) on each member interface


Note Engine 1 and Engine 2 interfaces do not support VLAN configuration in an EtherChannel.


On Cisco 12000 series ISE/E3, E4+, and E5 line cards, the minimum set of software features supported on the port-channel or pos-channel and member interfaces is:

IP unicast

Equal-cost load-balancing of IP traffic is performed across all active links in a link bundle.

VLAN configuration on a link bundle—EtherChannels only

An EtherChannel interface supports the configuration of VLAN subinterfaces. However, the configuration of stacked (802.1Q-in-Q) processing is only supported on certain EtherChannel interfaces. For more detailed information, see "Stacked VLAN Processing—802.1Q-in-Q" section.

For an example of how to configure an EtherChannel that consists of Ethernet VLAN links, see the "Configuring a VLAN EtherChannel" section.

Multiprotocol Label Switching

Equal-cost load-balancing of MPLS traffic is performed across all active links in a link bundle.

MPLS virtual private networks

A Layer 3 MPLS VPN configuration is supported on an EtherChannel or POS channel interface that is used in any of the following Provider Edge (PE) router configurations:

PE-to-CE (customer edge) connection as a VPN routing and forwarding (VRF) instance

PE-to-P (provider core) connection as a core-facing interface

Quality of Service (QoS) features: shaping, policing, MDRR/WRED, and packet marking

Access Control Lists (ACLs)

Named ACLs

Sampled NetFlow on:

Input interfaces only

Output interfaces only

On both input and output interfaces

Border Gateway Protocol (BGP) policy accounting

Hot Standby Router Protocol (HSRP) and Virtual Router Redundancy Protocol (VRRP)—EtherChannels only


Note If the link bundle consists of Engine 5 line cards with different ASIC versions, software features may not be supported in an EtherChannel. For example, stacked VLAN (802.1Q-in-Q) processing is supported in an EtherChannel only if all member interfaces are on SPAs that use the Engine 5 Version 2 FUGU ASIC. The Version 1 GILA ASIC does not support stacked VLAN processing. For information about Engine 5 line card support on specific ASIC versions, refer to Supported Line Cards for the 12000 Series Routers in the Cross-Platform Release Notes for Cisco IOS Release 12.0S.


Using an EtherChannel for High-Speed Provider Edge Support

In addition to the minimum set of software features described in the "Software Features Supported on a Link Bundle" section, the following software features are also supported in an EtherChannel that consists of ISE (E3), Engine 4+ (E4+), or Engine 5 (E5) member interfaces in a Cisco 12000 series Internet router that is deployed as a provider edge (PE) router:

Any Transport over MPLS—Ethernet over MPLS

Ethernet over MPLS—VLAN Mode

Layer 2 Quality of Service

Layer 2 Local Switching

Stacked VLAN Processing—802.1Q-in-Q

Unicast Reverse Path Forwarding

VLAN ID-Based Load-balancing

Multicast and Multicast VPN

Policy-Based Routing

Quality of Service, including: traffic shaping, policing, Modified Deficit Round Robin (MDRR), Weighted Random Early Detection (WRED), packet classification (marking), filtering defined by access control lists (ACLs), and Class of Service (COS) bit mapping

Route Processor Redundancy Plus

Stateful Switchover

Any Transport over MPLS—Ethernet over MPLS

The following Any Transport over Multiprotocol Label Switching (AToM)—Ethernet over MPLS (EoMPLS) features are supported only on the port-channel (logical) interface of an EtherChannel bundle that is configured for EoMPLS:

Ethernet over MPLS—VLAN Mode

Layer 2 Quality of Service

Layer 2 Local Switching

Layer 2 Protocol Tunneling and PDU Filtering

EoMPLS provides the following benefits:

A tunneling mechanism for Ethernet traffic through an MPLS-enabled Layer 3 core. It encapsulates Ethernet protocol data units (PDUs) inside MPLS packets and using label stacking forwards them across the MPLS network.

The ability for service providers to offer customers a virtual Ethernet line service or VLAN service using the service provider's existing MPLS backbone.

For more information about how to configure and use EoMPLS, refer to Any Transport over MPLS.

Ethernet over MPLS—VLAN Mode

In an MPLS network, the Ethernet over MPLS (EoMPLS) feature enables you to connect two VLAN networks located in different locations, without using bridges, routers, or switches at the VLAN locations. You can enable the MPLS backbone network to accept Layer 2 VLAN traffic by configuring the label edge routers (LERs) at both ends of the MPLS backbone.

In the Link Bundling feature on the Cisco 12000 series Internet router, EoMPLS is supported on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces and only in VLAN mode:

ISE/Engine 3 Gigabit Ethernet

Engine 5 Fast Ethernet

Engine 5 Gigabit Ethernet

You can configure an EtherChannel with EoMPLS on the ingress interface of a PE router in a PE-CE (customer edge) connection or on the egress interface in a PE-P provider-core router connection.

VLAN mode transports Ethernet traffic from a source 802.1Q VLAN to a destination 802.1Q VLAN (including stacked VLAN 802.1Q-in-Q traffic) over a core MPLS network. You must configure Ethernet over MPLS (VLAN mode) on the subinterfaces of member interfaces.


Note EoMPLS is not supported in port mode on an EtherChannel port-channel interface. Port mode allows a frame traveling into an interface to be packed into an MPLS packet and transported over the MPLS backbone to an egress interface. The entire Ethernet frame is transported without the preamble or Frame Check Sequence (FCS) as one packet.


When you configure EoMPLS on a port-channel interface, AToM virtual circuits (VCs) are transparently distributed on the interfaces of the member links in PE-to-P router connections by the AToM control plane, which views the link bundle as one virtual interface. A VC label is allocated on a per-VLAN basis. Because all Ethernet packets are forwarded on the same path for a given VLAN, the egress VLAN ID is used to obtain the Layer 2 adjacency.

On the port-channel interface, EoMPLS also supports the VLAN ID Rewrite feature, which enables you to use VLAN interfaces with different VLAN IDs at both ends of the tunnel.

In an EoMPLS configuration on a port-channel interface, only VLAN ID-based load-balancing is supported.

Layer 2 Quality of Service

On the Cisco 12000 series Internet router, Layer 2 QoS features that are normally supported on Fast or Gigabit Ethernet interfaces (see the "Quality of Service" section) are supported on the port-channel (Layer 3) interface of a link bundle that is configured for EoMPLS and consists of the following member interfaces:

ISE/Engine 3 Gigabit Ethernet

Engine 5 Fast Ethernet

Engine 5 Gigabit Ethernet

Supported Layer 2 QoS features include:

Per-class traffic shaping on egress traffic (disposition)

2-rate, 3-color Color-blind policer (supported on Cisco 12000 series ISE and E5 Ethernet line cards, based on RFC 2698)


Note The QoS: Color-Aware Policer feature is not supported on a port-channel interface. You can configure policing for EtherChannel traffic only in color-blind mode. The conform-color command is not supported.


Policing and bit marking on ingress traffic (imposition)

Marking MPLS experimental bits as a policing action, in addition to setting the 802.1p User Priority field (P-bits)

Mapping and copying the Layer 2 class of service (COS) P-bits to MPLS experimental bits at the ingress interface

Mapping and copying MPLS experimental bits to Layer 2 CoS P-bits at the egress interface

Setting Layer 2 CoS P-bits based on the VLAN ID at the egress interface

Policing and shaping Layer 2 VPN traffic at the MPLS imposition and disposition interfaces allows a service provider to offer service level agreements (SLAs) to customers in terms that include bandwidth, delay, jitter, and packet-loss guarantees in an EtherChannel connection. At imposition, Ethernet QoS markers are mapped to MPLS experimental bits. The traffic can be classified by the MPLS experimental bit then policed and shaped on provider interfaces.

For traffic traversing an EtherChannel, Modified Deficit Round Robin (MDRR) Congestion management and Weighted Random Early Detection (WRED) congestion avoidance are supported for MPLS packets with Layer 2 VPN payloads. Because packet queuing characteristics vary among Cisco 12000 series line cards, MDRR and WRED configurations can vary with the line card combinations used for MPLS imposition and disposition interfaces.

The following restrictions apply to Layer 2 QoS features configured on a port-channel interface:

Per-class traffic shaping is not supported in a QoS policy attached to an ingress port-channel interface that performs EoMPLS label imposition and disposition.

MDRR and WRED on ToFab queues are not supported on a port-channel interface that performs EoMPLS label imposition.

For a complete description and configuration information of the Layer 2 QoS features supported on Cisco 12000 series ISE and E5 interfaces, refer to Any Transport over MPLS (AToM): Layer 2 QoS (Quality of Service).

Layer 2 Local Switching

The Layer 2 Local Switching feature allows you to switch Layer 2 data between two interfaces of the same type (for example, Ethernet to Ethernet VLAN) or between interfaces of different types (for example, ATM to Ethernet) on the same router.

In the Link Bundling feature on the Cisco 12000 series Internet router, Layer 2 Local Switching is supported on the port-channel interface of an EtherChannel bundle that is configured for EoMPLS and consists of the following member interfaces:

ISE/Engine 3 Gigabit Ethernet

Engine 5 Fast Ethernet

Engine 5 Gigabit Ethernet

In addition, Layer 2 local switching is only supported for "like-to-like" switching between two member interfaces or subinterfaces of the same type as follows:

Gigabit Ethernet to Gigabit Ethernet interface

802.1Q VLAN to 802.1Q VLAN subinterface

Stacked VLAN (802.1Q-in-Q) to 802.1Q-in-Q subinterface

Apart from "like-to-like" switching, ATM-Ethernet interworking mode (bridged mode) for link bundling is also supported.


Note Only bridged mode is supported for link bundling. Routed mode is not supported.


The interfaces can reside on the same line card or on two different cards. During these kinds of like-to-like switching, the Layer 2 address is used, not any Layer 3 address. Additionally, same-port local switching allows you to switch Layer 2 data between two circuits on the same interface. For more information, refer to Layer 2 Local Switching.

In an EtherChannel, local switching involves switching Ethernet packets between customer edge (CE) routers that are on the same side of the connection as the Cisco 12000 series Internet (PE) router. Because the packet switching is performed using fake VC labels in a way similar to AToM label imposition, the VC label cannot be used for load-balancing. For load-balancing, the hash index is calculated using the VLAN ID in EoMPLS VLAN mode. (EoMPLS port mode is not supported for Layer 2 local switching.) The hash index is used to select the adjacency and is programmed into the hardware.

Layer 2 Protocol Tunneling and PDU Filtering

In the Link Bundling feature on the Cisco 12000 series Internet router, EoMPLS supports Layer 2 protocol tunneling (L2PT) and Layer 2 PDU filtering on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

ISE/Engine 3 Gigabit Ethernet

Engine 5 Fast Ethernet

Engine 5 Gigabit Ethernet

Layer 2 protocol tunneling allows service providers to carry traffic from multiple customers across a core network, and maintain the VLAN and Layer 2 protocol configurations of each customer without impacting the traffic of other customers. The Transparent Layer 2 Protocol Tunneling feature allows Layer 2 protocol data units (PDUs) to be tunneled across the core network without being interpreted and processed by intermediary network devices. Layer 2 PDU filtering allows a service provider to specify which Layer 2 PDUs are to be dropped at an ingress interface on a provider edge (PE) router. Transparent Layer 2 Protocol Tunneling and PDU filtering provide an enhanced feature set for service providers that transmit customer VLAN traffic from metro Ethernet VPNs across an MPLS core network.

Transparent Layer 2 protocol tunneling and PDU filtering are intended for use on provider edge (PE) routers in an MPLS-enabled service-provider core network.

For more information, refer to Transparent Layer 2 Protocol Tunneling and PDU Filtering.

Stacked VLAN Processing—802.1Q-in-Q

Stacked VLAN processing is supported only on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

ISE (E3) 4-port Gigabit Ethernet

E5 (FUGU ASIC-based) Fast Ethernet

E5 (FUGU ASIC-based) Gigabit Ethernet

The Stacked VLAN Processing feature supports the encapsulation of IEEE 802.1Q VLAN tags within a second layer of 802.1Q tag on provider edge (PE) routers to allow service providers to use a single VLAN to support customers who have multiple VLANs. The core service-provider network carries traffic with double-tagged, stacked VLAN (802.1Q-in-Q) headers of multiple customers while maintaining the VLAN and Layer 2 protocol configurations of each customer and without impacting the traffic of other customers. The Stacked VLAN Processing feature preserves VLAN IDs and keeps traffic in different customer VLANs segregated.

To allow a subinterface in an EtherChannel bundle on a PE-POP Cisco 12000 series Internet router to connect to a third-party PE-CLE switch that supports an EtherType value that is different from the default Cisco EtherType value (0x8100), enter the dot1q tunneling ethertype command on the port-channel interface. (The EtherType value supported on Cisco routers and switches is 0x8100. The networking devices of some other vendors support EtherType 0x9100.)

After you enter this command, only stacked VLAN packets with the specified EtherType value in the SP-VLAN tag of the packet header are received on subinterfaces in the link bundle. The new EtherType value is used in the SP-VLAN tag of all stacked VLAN packets sent from subinterfaces in the EtherChannel bundle to the PE-CLE switch, except if EoMPLS is configured on the port-channel interface. If EoMPLS is configured, all traffic received on an EoMPLS member interface (or subinterface) is sent to the PE-CLE switch as it was received from the peer PE router, without the new EtherType value.

For more information about how to configure and use stacked VLAN processing, refer to Stacked VLAN Processing.

Feature Parity on VLAN and Stacked VLAN Member Interfaces

VLAN and Stacked VLAN subinterfaces on member interfaces support all software features that are supported on a port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

E5 Fast Ethernet

ISE (E3) Gigabit Ethernet

E4+ Gigabit Ethernet

E5 Gigabit Ethernet

Some of the software features supported both on the EtherChannel interface and member VLAN and stacked VLAN subinterfaces include:

Access control lists (ACLs)

BGP policy-map accounting (only on ingress port-channel and subinterfaces)

Hot Standby Router Protocol (HSRP) and Virtual Router Redundancy Protocol (VRRP)

MPLS VPN Interautonomous Systems

Multiprotocol Label Switching (MPLS) Virtual Private Network (VPN)

MAC Address Accounting

MAC address accounting is supported only on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

E5 Fast Ethernet

ISE (E3) Gigabit Ethernet

E4+, Gigabit Ethernet

E5 Gigabit Ethernet

The MAC address accounting feature provides accounting information for IP traffic based on the source and destination MAC addresses on LAN interfaces. This feature calculates the total packet and byte counts for the EtherChannel that receives or sends IP packets to or from a unique MAC address. It also records a timestamp for the last packet received or sent.

To configure a port-channel interface for IP accounting based on the MAC address, enter the ip accounting mac-address {input | output} command in interface configuration mode, where:

input performs accounting based on the source MAC address on received packets.

output performs accounting based on the destination MAC address on received packets.

To display the MAC accounting information for an EtherChannel, use the show interfaces mac command in privileged EXEC mode.


Note An EtherChannel that consists of Modular Gigabit Ethernet (E4+) member interfaces supports either byte or packet accounting, but not both.


Multicast and Multicast VPN

Multicast routing protocols and multicast VPN are supported on the 802.1Q VLAN and stacked VLAN 802.1Q-in-Q subinterfaces and on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

E5 Fast Ethernet

ISE 4-port Gigabit Ethernet

E5 Gigabit Ethernet

Multicast Routing Protocols

You can enable multicast routing protocols on the port-channel interface and 802.1Q VLAN and 802.1Q-in-Q subinterfaces of an EtherChannel bundle, including Internet Group Management Protocol (IGMP), Protocol-Independent Multicast (PIM), Protocol Independent Multicast dense mode (PIM-DM), PIM sparse mode (PIM-SM), source specific multicast (SSM), Multicast Distributed Switching (MDS), Distance Vector Multicast Routing Protocol (DVMRP), and Cisco Group Management Protocol (CGMP).

For information about how to use and configure multicast routing protocols, refer to:

P1C: Cisco IOS IP and IP Routing Configuration Guide

FC: Cisco IOS Release 12.0 Configuration Fundamentals Configuration Guide

FR: Cisco IOS Release 12.0 Configuration Fundamentals Command Reference

Multicast Distributed Switching

Multicast Source Discovery Protocol

Multicast Fast-path Forwarding

Multicast fast-path forwarding allows hardware-based fast forwarding of IPv4 multicast traffic, resulting in high throughputs and performance. Packet forwarding is performed in the hardware switching engine instead of on the line card CPU. For more information, refer to Fast-Path Multicast Forwarding on Cisco 12000 Series Engine 2 and ISE Line Cards.

Multicast VPN

Multicast VPN provides the ability to support the multicast feature over a Layer 3 VPN. A multicast VPN allows an enterprise to transparently interconnect its private network across the network backbone of a service provider. Because MPLS VPNs support only unicast traffic connectivity, deploying the Multicast VPN feature in conjunction with MPLS VPN allows service providers to offer both unicast and multicast connectivity to MPLS VPN customers.

The Multicast VPN—IP Multicast Support for MPLS VPNs feature allows a service provider to configure and support multicast traffic in an MPLS Virtual Private Network (VPN) environment. This feature supports routing and forwarding of multicast packets for each individual VPN routing and forwarding (VRF) instance, and it also provides a mechanism to transport VPN multicast packets across the service provider backbone. For more information about this feature, refer to Multicast VPN—IP Multicast Support for MPLS VPNs.

Multicast QoS

Quality-of-service (QoS) features, which are supported on Fast Ethernet or Gigabit member interfaces and VLAN subinterfaces, are also supported in the egress direction on an EtherChannel interface configured for multicast traffic. For more information about this feature, refer to Multicast Egress QoS Support on Cisco 12000 Series Router, Integrated Services Engine (ISE) Line Cards. For more information on specific QoS features and the EtherChannels on which they are supported, see the "Quality of Service" section.

Multicast Configuration Commands

You must configure all multicast and multicast VPN commands on the port-channel interface associated with an EtherChannel after you enter the interface port-channel command. The configuration of multicast commands is not supported on:

Member interfaces

Port-channel interfaces of bundles that have one or more non-ISE or non-E5 member interfaces

The multicast and multicast VPN control plane and protocols that you configure on a port-channel interface are not aware of the member interfaces. All packets transmitted through member interfaces appear to the multicast control plane as packets sent through the logical EtherChannel interface. All multicast show commands that you enter on the control-plane level display multicast information for the port-channel (logical) interface.

Load-balancing for multicast traffic is performed using a hash algorithm that takes group information in (*,G) entries and (S,G) entries. The outgoing member link is selected using the source and destination IP addresses. If you enable VLAN ID-based load-balancing, multicast traffic is forwarded based on the VLAN ID of transmitted packets.

Policy-Based Routing

Policy-based routing (PBR) is supported on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

E5 Fast Ethernet

ISE Gigabit Ethernet

E4+ Gigabit Ethernet

E5 Gigabit Ethernet

You can also configure a routing policy for individual Fast Ethernet and Gigabit Ethernet member subinterfaces.

Policy-based routing provides a flexible means of routing packets by allowing you to configure a defined policy for traffic flows, lessening reliance on routes derived from routing protocols. You can set up PBR as a way to route packets based on configured policies. For example, you can implement routing policies to allow or deny paths based on the identity of a particular end system or an application protocol.

You can configure a unique routing policy for any of the following:

Port-channel interface of an EtherChannel bundle

An individual subinterface configured on the port-channel interface

Multiple subinterfaces configured on an EtherChannel port-channel interface

A policy contains one or more route maps. Each route map has one or more statements that specify the required matching criteria and actions. For example, a matching statement can be an access control list (ACL). If the matching criteria are met, the action statements are executed. An action statement can contain a routing action or a quality of service (QoS) action. The routing action determines the output interfaces for a packet or the next hop. The QoS action may be to set the type of service (ToS) or IP precedence parameter to a specific value.

For information about configuring policy-based routing, refer to Configuring Policy-Based Routing.

Unicast Reverse Path Forwarding

The Unicast Reverse Path Forwarding (Unicast RPF) feature is supported in an EtherChannel bundle as follows:

In an EtherChannel bundle that consists only of E5 Fast Ethernet, ISE Gigabit Ethernet, or E5 Gigabit Ethernet member interfaces, Unicast RPF is supported in loose and strict modes on the main port-channel interface, VLAN subinterfaces, and stacked VLAN subinterfaces.

In an EtherChannel bundle that consists only of E4+ Modular Gigabit Ethernet member interfaces, Unicast RPF is supported only in loose mode on the main port-channel interface.

The Unicast Reverse Path Forwarding Loose Mode feature provides a scalable anti-spoofing mechanism for use in multihome network configuration. This mechanism is effective in service-provider networks on routers that have multiple links to multiple Internet service providers (ISPs), on an ISP-to-ISP network edge.

Unicast RPF (loose or strict mode) provides a quick reaction mechanism for dropping network traffic on the basis of either the source or destination IP address. Network administrators can use this configuration as a tool for mitigating denial of service (DoS) and distributed denial of service (DDoS) attacks.

Verifying Source IPv4 Addresses

The difference between loose and strict checking modes is as follows:

Loose (exist-only) checking mode only verifies that a source IPv4 address exists in the routing table and that a valid return path through any interface exists in the Forwarding Information Base (FIB).

Strict checking mode verifies that the source IPv4 address of an IPv4 packet exists in the routing table and that the source IPv4 address is reachable by a path through the input interface (the interface on which the packet enters the router), such as a link-bundle member interface.

To provide ISPs with a DDoS resistance tool on the ISP-to-ISP edge of a network, Unicast RPF is enhanced from its original strict mode implementation to check the source addresses of each ingress packet without regard for the specific interface on which it was received.

You can enable Unicast RPF in strict or loose mode on an EtherChannel interface or subinterface. For information about enabling Unicast RPF, refer to Unicast Reverse Path Forwarding Loose Mode or Unicast Reverse Path Forwarding in Strict Mode on the Cisco 12000 Series Internet Router.

VLAN Scalability

Enhanced VLAN scalability is only supported on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

E5 Fast Ethernet

ISE 4-port Gigabit Ethernet

E5 Gigabit Ethernet

In Cisco IOS Release 12.0(31)S and earlier releases, when you create a subinterface on a member interface of an EtherChannel bundle, a subinterface is also created on all other member interfaces in the bundle. The following equation shows the total number of software data structures (known as IDBs) used for tracking Cisco IOS interfaces and interface status when you configure N number of subinterfaces in a link bundle:

Number of IDBs created = (1 + M) x N

Where:

M is the total number of member interfaces in the bundle

1 is added for the IDB used for the subinterface created for the port-channel (logical) interface

N is the number of subinterfaces created

For example, if a link bundle consists of 4 member interfaces and you configure 2 subinterfaces on member interfaces, the total number of IDBs created is 10.


Note In the forwarding information base (FIB) table, the total number of IDBs created is included in the total number of IDBs created.


The equation shows how the number of subinterfaces supported on an EtherChannel bundle, and therefore the number of supported VLANs, is limited due to the increased amount of route processor memory used by the IDBs.

For this reason, in Cisco IOS Release 12.0(31)S and earlier releases, the number of VLANs supported in an EtherChannel bundle that consists of 4-port Gigabit Ethernet ISE member interfaces is 300. However, starting in Cisco IOS Release 12.0(32)S and later releases, a software enhancement allows you to configure up to 1000 VLANs on the supported Gigabit Ethernet member interfaces in a link bundle. Engine 5 Fast Ethernet member interfaces are supported starting in Cisco IOS Release 12.0(33)S.

In addition, when you increase the number of VLANs on the member interfaces of an EtherChannel, the performance of VLAN load-balancing and data transmission depends on the percentage of ternary content addressable memory (TCAM) allocated for VLANs on each line card, whose interfaces belong to the link bundle. You can maximize the VLAN TCAM size allocated to each ISE/E3 Gigabit Ethernet subinterface by using the hw-module slot linkbundle-tcam-scalability command (see the "Maximizing VLAN TCAM to Support VLAN Scalability" section). When you enter this command, the Layer 2 TCAM regions on an ISE (E3) line card are recarved to support 1K of VLAN traffic on each subinterface in a link bundle.


Note The hw-module slot linkbundle-tcam-scalability command is a required command for an ISE line card.


Quality of Service

The Quality-of-Service (QoS) features described in this section are supported only on the port-channel interface of an EtherChannel bundle that consists of the following member interfaces:

E5 Fast Ethernet

ISE 4-port Gigabit Ethernet

E4+ Modular Gigabit Ethernet

E5 Gigabit Ethernet

Certain QoS features supported on Ethernet interfaces are also supported when you assign these interfaces to an EtherChannel bundle. See Table 1, Table 2, and Table 3 for a list of supported QoS features.

The following restrictions apply when you use QoS features in an EtherChannel bundle:

On ISE 4-port Gigabit Ethernet member links, QoS features are configured on the port-channel interface and supported on main Gigabit Ethernet interfaces and on subinterfaces when VLAN ID-based load-balancing is enabled. Main interfaces support only QoS packet classification and marking. QoS packet classification and marking are supported only in the ingress direction on main and subinterfaces. QoS features are supported on a port-channel interface only in the ingress and egress direction for multicast traffic.

On E4+ Modular Gigabit Ethernet member links, QoS features are configured on the port-channel interface and supported only on main Gigabit Ethernet interfaces. QoS features are not supported on subinterfaces.

On E5 Fast Ethernet and Gigabit Ethernet member links, QoS features are configured on the port-channel interface and supported on main Gigabit Ethernet interfaces and on subterfaces when VLAN ID-based load-balancing is enabled. Main interfaces support only QoS packet classification and marking. QoS packet classification and marking are supported only in the ingress direction on main and subinterfaces.

The match VLAN statement in class-map configuration mode is not supported.

In an EtherChannel bundle, traffic policing configured on ingress interfaces works properly only if incoming traffic is transmitted from another network device that uses VLAN ID-based load-balancing.

To configure QoS features, you must use the modular Quality-of-Service command-line (MQC) interface. For information, refer to:

Cisco IOS Quality of Service Solutions Configuration Guide, Release 12.3

Modular Quality of Service Command-Line Interface

Table 1 QoS Features Supported on 4-port Gigabit Ethernet ISE Interfaces in an EtherChannel

QoS Command or Feature in Ingress and Egress Directions
4-port Gigabit Ethernet ISE Interface
4-port Gigabit Ethernet ISE VLAN 802.1Q Subinterface
Stacked VLAN
(802.1Q-in-Q)
Subinterface

Committed access rate (CAR) using rate-limit

Not supported

Supported

Supported

Modified Deficit Round Robin (MDRR)

Not supported

Not supported: ingress Supported: egress

Not supported: ingress Supported: egress

Weighted Random Early Detection (WRED)

Not supported: ingress Not supported: egress

Not supported: ingress Supported: egress

Not supported: ingress Supported: egress

set cos (egress only)

Not supported

Supported

Supported

set discard-class (ingress only)

Supported

Supported

Supported

set dscp

Supported

Supported

Supported

set ip precedence

Supported

Supported

Supported

set mpls experimental

Supported

Supported

Supported

set qos-group (ingress only)

Supported

Supported

Supported

Shaping

Not supported: ingress Not supported: egress

Not supported: ingress Supported: egress

Not supported: ingress Supported: egress

Hierarchical shaping

Not supported: ingress Not supported: egress

Supported: egress only

Supported: egress only

Hierarchical policing

Not supported: ingress Not supported: egress

Supported: ingress Supported: egress

Supported: ingress Supported: egress


Table 2 QoS Features Supported on Modular Gigabit Ethernet Interfaces in an EtherChannel

QoS Command or Feature in Ingress and Egress Directions
Modular Gigabit Ethernet Interface
Modular Gigabit Ethernet VLAN Subinterface1

Committed access rate (CAR) using rate-limit

Not supported

Not supported

Modified Deficit Round Robin (MDRR)

Not supported

Not supported

Weighted Random Early Detection (WRED)

Not supported

Not supported

set cos (egress only)

Not supported

Not supported

set discard-class (ingress only)

Supported

Not supported

set dscp

Supported

Not supported

set ip precedence

Supported

Not supported

set mpls experimental

Supported

Not supported

set qos-group (ingress only)

Supported

Not supported

Shaping

Not supported

Not supported

Hierarchical shaping

Not supported

Not supported

Hierarchical policing

Not supported

Not supported

1 In an EtherChannel bundle, modular Gigabit Ethernet (E4+) member subinterfaces do not support any of the QoS features normally supported on a modular Gigabit Ethernet (E4+) subinterface.


Table 3 QoS Features Supported on Engine 5 Fast and Gigabit Ethernet Interfaces in an EtherChannel

QoS Command or Feature in Ingress and Egress Directions
E5 Fast Ethernet or Gigabit Ethernet Interface
E5 Fast Ethernet or Gigabit Ethernet VLAN 802.1Q Subinterface
E5 Stacked VLAN (802.1Q-in-Q) Subinterface

Committed access rate (CAR) using rate-limit

Not supported

Supported

Supported

Modified Deficit Round Robin (MDRR)

Not supported

Not supported: ingress Supported: egress

Not supported: ingress Supported: egress

Weighted Random Early Detection (WRED)

Not supported: ingress Not supported: egress

Not supported: ingress Supported: egress

Not supported: ingress Supported: egress

set cos (egress only)

Not supported

Supported

Supported

set discard-class (ingress only)

Supported: ingress

Supported

Supported

set dscp

Supported

Supported

Supported

set ip precedence

Supported

Supported

Supported

set mpls experimental

Supported

Supported

Supported

set qos-group (ingress only)

Supported

Supported

Supported

Shaping

Not supported: ingress Not supported: egress

Not supported: ingress Supported: egress

Not supported: ingress Supported: egress

Hierarchical shaping

Not supported: ingress Not supported: egress

Supported: egress only

Supported: egress only

Hierarchical policing

Not supported: ingress Not supported: egress

Supported: ingress Supported: egress

Supported: ingress Supported: egress


Feature Parity on POS Channel Bundles

Starting in Cisco IOS Release 12.0(33)S, the same level of support for IOS software features exists on a POS channel bundle that consists of ISE, E4+, E5, and E6 member interfaces as on an EtherChannel link bundle. For a description of the supported software features, see the "Software Features Supported on a Link Bundle" section and the "Using an EtherChannel for High-Speed Provider Edge Support" section.

Table 4 summarizes the support for each Cisco IOS software feature in a POS channel link-bundle interface according to the engine type of member interfaces.

Table 4 IOS Software Features Supported on a POS Channel Interface by Engine Type

IOS Software Feature or QoS Command
ISE/E3 POS Channel Main Interface
E4+ POS Channel Main Interface
E5 POS Channel Main Interface
E6 POS Channel Main Interface

IPv4 forwarding

Supported

Supported

Supported

Supported

MPLS forwarding

Supported

Supported

Supported

Supported

MPLS VPNs

Supported

Supported

Supported

Supported

MPLS VPN Inter-Autonomous Systems (Inter-AS)

Supported

Supported

Supported

Supported

Multicast forwarding

Supported

Not supported

Supported

Not supported

Multicast QoS

Supported

Not supported

Supported

Not supported

Multicast VPN

Supported

Not supported

Supported

Not supported

Policy-based routing

Supported

Supported

Supported

Supported

set cos (egress only)

Supported

Supported

Supported

Supported

set discard-class (ingress only)

Supported

Supported

Supported

Supported

set dscp

Supported

Supported

Supported

Supported

set ip precedence