Interfaces and Hardware Component Configuration Guide for Cisco 8000 Series Routers, Cisco IOS XR Releases

PDF

Interfaces and Hardware Component Configuration Guide for Cisco 8000 Series Routers, Cisco IOS XR Releases

Link bundling

Updated: July 31, 2026

Want to summarize with AI?

Log in

This topic describes how link bundling groups multiple point-to-point links into a single logical interface to provide higher bandwidth, redundancy, and load balancing between two routers.

Link bundling is a network configuration that

  • aggregates multiple ports into a single logical interface

  • utilizes a shared MAC and IP address, and

  • supports a unified configuration set.

Table 1. Feature History Table

Feature Name

Release

Description

Configure link bundling

 Release 26.1.1

Introduced in this release on: Centralized Systems (8400 [ASIC:K100]) )(select variants only*)

*This feature is now supported on Cisco 8404-SYS-D routers.

Configure link bundling

Release 25.4.1

Introduced in this release on: Fixed Systems (8700 [ASIC: K100], 8010 [ASIC: A100])(select variants only*)

*This feature is now supported on:

  • 8711-48Z-M

  • 8011-32Y8L2H2FH

  • 8011-12G12X4Y-A/D

Configure link bundling

Release 25.1.1

Introduced in this release on: Fixed Systems (8010 [ASIC: A100])(select variants only*)

*This feature is supported on Cisco 8011-4G24Y4H-I routers.

Configure link bundling

Release 24.4.1

Introduced in this release on: Fixed Systems (8200 [ASIC: P100], 8700 [ASIC: P100, K100])(select variants only*); Modular Systems (8800 [LC ASIC: P100])(select variants only*)

You can now enhance network resilience and efficiency by aggregating multiple physical interfaces into a single logical link. This bundling improves redundancy by ensuring continued data transmission even if individual links fail. It supports load balancing across bundled links, optimizing data flow and minimizing congestion. The feature is easily configurable and integrates seamlessly with existing network setups, providing a scalable solution for managing increased traffic demands. By utilizing link bundling, you achieve a more robust and flexible network infrastructure, promoting continuous connectivity and performance.

*Previously this feature was supported on Q200 and Q100. It is now extended to:

  • 8212-48FH-M

  • 8711-32FH-M

  • 8712-MOD-M

  • 88-LC1-12TH24FH-E

  • 88-LC1-36EH

  • 88-LC1-52Y8H-EM

1023 Ethernet Bundle Interfaces Support

Release 7.3.15

With the introduction of this enhancement, the maximum system-wide bundle interface scale has increased from 512 to 1023 bundle interfaces. The default value remains at 64-member links for each bundle.

64-bit Bandwidth Support

Release 7.3.15

With this release, the Cisco 8000 Series Router supports 64-bit bandwidth field, as opposed to the previous 32-bit bandwidth field. 64-bit bandwidth enables the system to support interface bandwidths greater than 4.2T.

A link bundle is a group of one or more ports that the system aggregates and treats as a single link.

Each bundle has

  • a single MAC address

  • a single IP address, and

  • a single configuration set (such as ACLs).

The virtual interface is treated as a single interface on which you can configure an IP address and other software features that the link bundle uses. Packets sent to the link bundle are forwarded to one of the links in the bundle.

Operational specifications for link bundling

These are the operational specifications for configuring link bundling:

  • The router supports both Layer 2 and Layer 3 link bundles. If the link bundle is a Layer 3 interface, the system requires an IP address. If the link bundle is a Layer 2 interface, the system does not require an IP address. A link bundle on the router may contain Layer 2 and Layer 3 subinterfaces within it. In that case, the Layer 3 subinterfaces require IP addresses, but the link bundle interface does not require an IP address.

  • The router supports bundling for Ethernet interfaces.

  • All the individual links within a single bundle must be of the same type.

  • Cisco IOS XR software supports IEEE 802.3ad— A standard technology that employs a Link Aggregation Control Protocol (LACP) to ensure that all the member links in a bundle are compatible. The system automatically removes the links from a bundle that are incompatible or have failed.

  • Prior to Cisco IOS XR Software Release 7.3.15, the interface bandwidth was stored and processed as a 32-bit value, which supported bundles with an aggregate bandwidth of up to 4.2 Gbps (the sum of its members). Starting with Cisco IOS XR Software Release 7.3.15, the interface bandwidth is stored and processed as a 64-bit value. The 64-bit value supports significantly larger aggregate bandwidths, accommodating bundles with high-bandwidth members whose combined bandwidth can exceed 4.2 Tbps.

Benefits of link bundling

These are the advantages of using link bundles:

  • Multiple links can span several line cards to form a single interface. The failure of a single link does not cause a loss of connectivity.

  • Bundled interfaces increase bandwidth availability, because traffic is forwarded over all available members of the bundle. Traffic can flow on the available links if one of the links within a bundle fails. You can add bandwidth without interrupting the packet flow.

IEEE 802.3ad standard

The IEEE 802.3ad standard typically defines a method of forming Ethernet link bundles. The IEEE 802.3ad standard is a method that

  • defines Ethernet link bundle formation

  • facilitates information exchange between systems, and

  • determines link compatibility via link aggregation group identifiers (LAG IDs).

For each link configured as a bundle member, The systems exchange the following information to form the LAG ID:

  • A globally unique local system identifier distinguishes one router from another.

  • An operational key identifies the bundle of which the link is a member.

  • An identifier , also known as port ID for the link.

  • The current aggregation status indicates the state of the link.

This information is used to form the link aggregation group identifier (LAG ID). Links that share a common LAG ID can be aggregated. Individual links have unique LAG IDs.

The system guarantees the uniqueness of the system identifier by using a MAC address from the router. The bundle and link identifiers maintain significance only to the router that assigns them, ensuring that no two links or bundles share the same identifier. The system combines information from the peer system with local system data to determine the compatibility of links configured as bundle members.

The router assigns bundle MAC addresses from a set of reserved addresses in the backplane. This MAC address remains with the bundle for the duration of the interface existence unless the user configures a different address. Member links utilize the bundle MAC address when passing bundle traffic. The system also applies any unicast or multicast addresses configured on the bundle to all member links.

Note

We recommend that you avoid modifying the MAC address, because changes in the MAC address can affect packet forwarding.

The optional Link Aggregation Control Protocol (LACP) is defined in the IEEE 802 standard. LACP communicates between two directly connected systems (or peers) to verify the compatibility of bundle members. For the router, the peer can be either another router or a switch.

LACP monitors the operational state of link bundles to ensure that

  • all links terminate on the same two systems

  • both systems consider the links to be part of the same bundle, and

  • all links have the appropriate settings on the peer.

LACP transmits frames containing the local port state and the local view of the partner system's state. The system analyzes these frames to ensure that both systems are in agreement.

LACP fallback

The LACP Fallback is a network feature thatThe LACP Fallback feature allows an active LACP interface to establish a Link Aggregation Group (LAG) port-channel before the port-channel receives the Link Aggregation and Control Protocol (LACP) protocol data units (PDU) from its peer.

  • allows an active Link Aggregation and Control Protocol (LACP) interface to establish a LAG port-channel before receiving LACP protocol data units (PDUs)

  • enables servers to connect to PXE servers for boot image downloads, and

  • maintains one active port until the server completes the boot process.

Benefits of LACP fallback

The LACP Fallback feature provides several operational benefits:

  • The router allows the server to bring up the LAG before the system receives any LACP PDUs.

  • The server establishes a connection to a PXE server over one Ethernet port to download its boot image.

  • The server fully forms an LACP port-channel once the boot process concludes.

LACP short period time intervals

The LACP short period time interval is a configuration setting that

  • enables faster detection of link failures

  • facilitates rapid recovery from link timeouts, and

  • verifies the stability of member links.

As packets are exchanged across member links of a bundled interface, some member links may slow down or time-out and fail. LACP packets are exchanged periodically across these links to verify the stability and reliability of the links over which they pass. The configuration of short period time intervals, in which LACP packets are sent, enables faster detection and recovery from link failures.

The system supports specific parameters for short period time intervals:

  • Configuration occurs in milliseconds.

  • Values increase in increments of 100 milliseconds.

  • The range spans from 100 to 1000 milliseconds.

  • The default setting equals 1000 milliseconds (1 second).

  • The system supports up to the maximum available member links.

  • The system processes up to 1280 packets per second (pps).

After 6 missed packets, the link is detached from the bundle.

When the short period time interval is not configured, LACP packets are transmitted over a member link every 30 seconds by default.

When the short period time interval is configured, LACP packets are transmitted over a member link once every 1000 milliseconds (1 second) by default. Optionally, both the transmit and receive intervals can be configured to less than 1000 milliseconds, independently or together, in increments of 100 milliseconds (100, 200, 300, and so on).

When you configure a custom LACP short period transmit interval at one end of a link, you must configure the same time period for the receive interval at the other end of the link.

Note

You must always configure the transmit interval at both ends of the connection before you configure the receive interval at either end of the connection. Failure to configure the transmit interval at both ends first results in route flapping (a route going up and down continuously). When you remove a custom LACP short period, you must do it in reverse order. You must remove the receive intervals first and then the transmit intervals.

Load balancing is a forwarding mechanism that

  • distributes traffic over multiple links based on certain parameters

  • enables bandwidth-based load-balancing for Layer 3 unicast flows, and

  • supports all links in a bundle using Layer 2, Layer 3, and Layer 4 routing information.

For more information about other forms of load balancing on the router, see the following:

  • Per-flow load balancing on non-bundle interfaces using Layer 3 and 4 routing information.

  • Pseudowire (PW) load balancing.

The Layer 3 egress load balancing mechanism is a network traffic distribution method that

  • enables global link bundle load balancing by default

  • utilizes consistent hashing between ingress and egress linecards, and

  • optimizes packet forwarding across network processors.

The ingress linecard performs bundle member selection and forwards the packet to the linecard and network processor that corresponds to the selected bundle member. The system uses the same hash value for both ingress and egress linecards. Consequently, the egress linecard selects the same bundle member as the ingress linecard.

Multicast IPv4 and IPv6 traffic

The system manages multicast traffic distribution through these processes:

  • The system predetermines a set of egress linecards for outbound multicast IPv4 and IPv6 traffic.

  • The system selects the bundle member for each outgoing interface based on the multicast group address.

  • The egress linecard performs network processor selection when bundle members reside across multiple processors within the egress linecard.

  • The system uses a 5-tuple hash to select a bundle member within a network processor when a packet arrives at the egress linecard. This provides better resiliency for bundle member state changes within a network processor.

When one member link in a bundle fails, the system redirects the traffic to the remaining operational member links and traffic flow remains uninterrupted.

The link bundle protection mechanism is a network configuration feature that

  • manages active link capacity

  • provides redundancy through standby links, and

  • facilitates rapid traffic switchover during link failures.

The system provides several configuration options for managing link bundle protection:

  • A bundle supports a maximum of 64 active links by default, and the system redirects traffic to remaining operational members if a link fails.

  • You can implement 1:1 link protection by setting the bundle maximum-active links command to 1, which designates one active link and one or more dedicated standby links.

  • The system performs a switchover to a standby link immediately upon the failure of an active link to ensure uninterrupted traffic.

  • If the active and standby links are running LACP, you can choose between an IEEE standard-based switchover (the default) or a faster proprietary optimized switchover. If the active and standby links are not running LACP, the proprietary optimized switchover option is used.

  • Regardless of the type of switchover you are using, you can disable the wait-while timer, which expedites the state negotiations of the standby link and causes a faster switchover from a failed active link to the standby link. To do so, you can use the lacp switchover suppress-flaps command.

Nonstop forwarding during card failover

Cisco IOS XR software supports nonstop forwarding during a failover between active and standby paired RP cards. Nonstop forwarding ensures that there is no change in the state of the link bundles when a failover occurs.

For example, if an active RP fails, the standby RP becomes operational. The system replicates the configuration, node state, and checkpoint data of the failed RP to the standby RP. The bundled interfaces are present when the standby RP becomes the active RP.

Note

Failover is always onto the standby RP. You do not need to configure anything to guarantee that the system maintains the standby interface configurations.

On the router, when the system applies QoS on the bundle for either the ingress or egress direction, QoS is applied at each member interface. For complete information on configuring QoS on link bundles on the router, refer to the Cisco 8000 Series Aggregation Services Router Modular Quality of Service Configuration Guide and the Cisco 8000 Series Aggregation Services Router Modular Quality of Service Command Reference.

You can configure 802.1Q VLAN subinterfaces on 802.3ad Ethernet link bundles.

Note

The memory requirement for bundle VLANs is slightly higher than for standard physical interfaces.

To create a VLAN subinterface on a bundle, include the VLAN subinterface instance with the interface Bundle-Ether command, as follows:

interface Bundle-Ether interface-bundle-id.subinterface

After you create a VLAN on an Ethernet link bundle, the system supports all VLAN subinterface configuration on that link bundle.

VLAN subinterfaces can support Ethernet Flow Points (EFPs) and Layer 3 services.

You can configure Layer 3 VLAN subinterfaces as follows:

interface bundle-ether instance.subinterface, encapsulation dot1q xxxxx

The link bundle is a network configuration that

  • aggregates multiple physical links into a single logical link, negotiates link aggregation through the Link Aggregation Control Protocol (LACP), and

  • manages the availability of member links for data transmission.

When a member link is added to a link bundle, Link Aggregation Control Protocol (LACP) communication is established with the peer to negotiate and control the link aggregation. LACP does not have any provision to incorporate a delay before letting data transmission over the link. Therefore, the peer starts using the link when the LACP communication is up. Occasionally, even though the link status is up and LACP communication is up, the hardware programming for data-path packet forwarding does not complete. In such scenarios, the transmitted data is lost without any notification or error message to the source or destination of the traffic.

Table 2. Feature History Table

Feature Name

Release

Description

Designate a member link as unviable

 Release 25.4.1

Introduced in this release on: Fixed Systems (8010 [ASIC: A100])(select variants only*)

*This feature is now supported on:

  • 8011-32Y8L2H2FH

  • 8011-12G12X4Y-A/D

Designate a member link as unviable

Release 25.1.1

Introduced in this release on: Fixed Systems (8700 [ASIC: K100], 8010 [ASIC: A100])(select variants only*)

*This feature is supported on:

  • 8712-MOD-M

  • 8011-4G24Y4H-I

Designate a member link as unviable

Release 24.4.1

Introduced in this release on: Fixed Systems (8200 [ASIC: P100], 8700 [ASIC: P100])(select variants only*); Modular Systems (8800 [LC ASIC: P100])(select variants only*)

*This feature is now supported on:

  • 8212-48FH-M

  • 8711-32FH-M

  • 88-LC1-12TH24FH-E

  • 88-LC1-36EH

  • 88-LC1-52Y8H-EM

Designate a member link as unviable

Release 7.10.1

Earlier, when a member link is added to an interface link bundle, the peer starts using the link as soon as the LACP communication is up. Sometimes, the hardware programming for the data-path does not get complete in this time resulting in packet loss without any notification to the source.

You can now mark a member link as unviable to introduce a delay during which the link is treated as standby. By delaying the usage of the member link for data transmission, you can ensure that the link configuration is fully established, which enables successful data transmission.

This feature introduces these changes:

You can manage the readiness of member links using these commands:

  • You can delay the use of such member links, which are not fully ready to handle data transmission, using the forwarding-unviable command. This command configures the link as forwarding-unviable and the member link is considered "standby" for bundle management. As standby member links of a bundle are not used for data transmission, the usage of forwarding-unviable member links is delayed.

  • When the member link is fully up—that is, the packet forwarding data-path is also completely programmed—you can disable forwarding-unviability of the link using the no forwarding-unviable command. This removes the forwarding-unviable configuration of the link. The link is then treated as an active member of the bundle and is used in data transmission and load balancing.

Note

We recommend that you wait for a few minutes before running the no forwarding-unviable command to ensure that the packet forwarding data-path is completely programmed.

Consider these guidelines and restrictions while designating a member link as unviable:

  • Forwarding-unviable is disabled on all Ethernet interfaces by default. Therefore, by default, all member links in a bundle are considered "active".

  • A link bundle is considered up only if at least one member link is active. Only the active member links in the link bundle are used for data transmission, load balancing, and redundancy.

  • If a link bundle has only one member link, which is forwarding-unviable, the bundle state is considered "down".

  • If all the member links in a bundle are forwarding-unviable, the bundle state is considered "down".

  • Other existing threshold parameters such as minimum-active links, maximum-active links, and maximum-active bandwidth, which are considered to determine the bundle state, continue to function along with the forwarding-unviable functionality.

  • The forwarding-unviable configuration has no effect on individual Ethernet interfaces that are not part of a link bundle. Irrespective of the configuration, such non-member interfaces continue to attempt data transmission and reception.

A link-flap err-disable is a network reliability feature that:

  • Automatically shuts down an interface if it frequently toggles between "up" and "down" states.

  • Ensures network reliability by allowing traffic to reroute to stable backup paths when a port becomes unstable.

  • Is optimized for short-distance links within data centers, such as those connecting servers and Top-of-Rack (ToR) switches.

Table 3. Feature History Table

Feature Name

Release

Description

Link flap err-disables

 Release 26.2.1

Introduced in this release on: Fixed Systems (8200 [ASIC: Q100, Q200, P100], 8700 [ASIC: P100, K100], 8010 [ASIC: A100]); Centralized Systems (8600 [ASIC:Q200]) ; Modular Systems (8800 [LC ASIC: K100)]

You can achieve reliable network performance and a stable system state by isolating unstable interfaces that flap excessively, preventing routing protocol disruptions and ensuring effective workload failover. With this feature, the system monitors link transitions and automatically moves physical, breakout, or bundle member interfaces into an error-disabled state if a defined flap-count threshold is breached within a specific time window. This automated mechanism guarantees traffic redirection to stable paths while providing the flexibility of manual restoration or a timer-based automatic recovery once the link stabilizes.

This feature introduces these changes:

CLI:

  • fault management link-flap error-disable

  • error-disable recovery cause

  • clear error-disable

  • show error-disable

Supported interface types:

  • Physical interfaces

  • Breakout interfaces

  • Bundle member interfaces

Syslog messages

When an interface is disabled due to excessive flapping, the system generates this syslog message:

%PKT_INFRA-ERRDIS-6-ERROR_DISABLE : GigabitEthernet0/0/0/0: Error disabling due to reason: link-flap-exceeded

Consider these limitations while configuring link-flap err-disable:

  • Only hardware-level link state changes are counted. Upper-layer protocol changes, such as Layer 2 protocols or line protocols, do not trigger a flap count.

  • The link-flap err-disable feature works independently of the interface dampening feature. If both are configured, the first feature to reach its threshold takes action.

  • To ensure optimal coexistence, review interactions with carrier-delay and object-tracking, as these may impact feature functionality.