Interfaces and Hardware Component Configuration Guide for Cisco 8000 Series Routers, Cisco IOS XR Releases

PDF

Interfaces and Hardware Component Configuration Guide for Cisco 8000 Series Routers, Cisco IOS XR Releases

Configure link bundling

Want to summarize with AI?

Log in

This topic describes how link bundle configuration involves creating a link bundle, assigning an IP address to the virtual interface, and adding member interfaces to the bundle.



Before configuring link bundling, ensure that you meet the following tasks and conditions:

  • You know the interface IP address (Layer 3 only).

  • You know the links that you must include in the bundle that you are configuring.

  • If you are configuring an Ethernet link bundle, you must install Ethernet line cards on the router.

Note

For more information about physical interfaces, PLIMs, and modular services cards, refer to the Cisco 8000 Series Router Hardware Installation Guide.


These guidelines and restrictions apply when you configure Ethernet link bundles:

Mixed-speed bundle support

Starting with Cisco IOS XR Release 7.2.1, the router supports mixed-speed bundles, allowing member links with different bandwidths to be included in the same bundle. Traffic distribution across bundle members is based on the bandwidth of each link. Mixed-speed bundles are subject to a maximum bandwidth ratio of 10:1 between the fastest and slowest member links.

For example, you can combine a 10 Gbps and a 100 Gbps link, or a 100 Gbps and a 400 Gbps link, in the same bundle; however, a 10 Gbps and a 400 Gbps link cannot be bundled together. Load balancing is performed in proportion to the bandwidth of each member link. Typical valid combinations include:

  • 400G, 100G

  • 400G, 40G

  • 400G, 100G, 40G

  • 100G, 40G

  • 100G, 10G

  • 100G, 40G, 10G

  • 40G and 10G

Bundle scale and interface support

Bundle scale and interface support properties apply to Ethernet link bundles on the router.

  • Any type of Ethernet interface can be bundled, with or without the use of Link Aggregation Control Protocol (LACP).

  • With Cisco IOS XR Release 7.3.15, a single router can support up to 1023 bundle interfaces.

    Each bundle can accommodate up to 64 member links.

    If adding a new line card causes these limits to be exceeded, the system will experience continuous Out of Resource (OOR) failures. To resolve these errors, you must either reduce the scale or disable the affected line card.

  • Physical-layer and link-layer configuration is performed on individual member links of a bundle.

  • Configuration of network-layer protocols and higher-layer applications is performed on the bundle itself.

  • IPv4 and IPv6 addressing is supported on Ethernet link bundles.

  • Only physical links can be bundle members.

Bundle operational characteristics

Operational characteristics apply to bundles and their member links.

  • A bundle can be administratively enabled or disabled.

  • Each individual link within a bundle can be administratively enabled or disabled.

  • Ethernet link bundles are created in the same way as Ethernet channels, where the user enters the same configuration on both end systems.

  • The MAC address that is set on the bundle becomes the MAC address of the links within that bundle.

  • Load balancing (the distribution of data between member links) is done by flow instead of by packet. Data is distributed to a link in proportion to the bandwidth of the link in relation to its bundle.

  • QoS is supported and is applied proportionally on each bundle member.

  • All links within a single bundle must terminate on the same two systems.

  • Bundled interfaces are point-to-point.

  • A link must be in the up state before it can be in the distributing state in a bundle.


Procedure

1.

Enter global configuration mode using the configure command.

Example:

Router# configure
2.

Create a new Ethernet link bundle with the specified bundle-id in the range 1 to 65535, and enter interface configuration submode.

Example:

Router(config)# interface Bundle-Ether 3
3.

Assign an IP address and subnet mask to the virtual interface.

Example:

Router(config-if)# ipv4 address 10.1.2.3 255.0.0.0

Only a Layer 3 bundle interface requires an IP address.

4.

(Optional) Set the minimum amount of bandwidth required before a user can bring up a bundle.

Example:

Router(config-if)# bundle minimum-active bandwidth 580000
5.

(Optional) Set the number of active links required before you can bring up a specific bundle.

Example:

Router(config-if)# bundle minimum-active links 2
6.

(Optional) Implement 1:1 link protection for the bundle.

Example:

Router(config-if)# bundle maximum-active links 1 hot-standby

This causes the highest-priority link in the bundle to become active and the second-highest-priority link to become the standby. It also specifies that a switchover between active and standby LACP-enabled links is implemented per a proprietary optimization.

Note

The priority of the active and standby links is based on the value of the bundle port-priority command.

7.

(Optional) If you enabled 1:1 link protection on a bundle with member links running LACP, disable the wait-while timer in the LACP state machine.

Example:

Router(config-if)# lacp fast-switchover

Disabling this timer causes a bundle member link in standby mode to expedite its normal state negotiations, thereby enabling a faster switchover from a failed active link to the standby link.

8.

Exit interface configuration submode.

Example:

Router(config-if)# exit

What to do next

Perform the Add Ethernet interface to bundle sub-task next to add member Ethernet interfaces to the bundle and commit the configuration on both connection endpoints.


Add Ethernet interface to bundle

Perform this task after you create the Bundle-Ether interface and set the required bundle parameters. Repeat this task on the remote end of the connection so that both endpoints share the same bundle configuration.

Procedure

1.

Enter interface configuration mode for the Ethernet interface to add to the bundle.

Example:

Router(config)# interface GigabitEthernet 1/0/0/0

Enter the GigabitEthernet or TenGigE keyword to specify the interface type. Replace the interface-path-id argument with the node-id in the rack/slot/module format.

2.

Add the link to the specified bundle using the bundle id bundle-id [mode {active | on | passive}] command.

Example:

Router(config-if)# bundle id 3

To enable active or passive LACP on the bundle, include the optional mode active or mode passive keywords in the command string. To add the link to the bundle without LACP support, include the optional mode on keywords.

Note

If you do not specify the mode keyword, the default mode is on (LACP is not run over the port).

3.

(Optional) If you set the bundle maximum-active links command to 1, set the priority of the active link to the highest priority (lowest value) and the standby link to the second-highest priority.

Example:

Router(config-if)# bundle port-priority 1
4.

(Optional) If a link is in the down state, bring it up.

Example:

Router(config-if)# no shutdown

The no shutdown command returns the link to an up or down state depending on the configuration and state of the link.

5.

(Optional) Repeat the interface addition steps to add more member links to the bundle.

Example:

Router(config-if)# exit
Router(config)# interface GigabitEthernet 1/0/2/1
Router(config-if)# bundle id 3
Router(config-if)# bundle port-priority 2
Router(config-if)# no shutdown
6.

Save the configuration changes using the end or commit command.

Example:

Router(config-if)# commit
7.

Perform all the previous steps on the remote end of the connection to bring up the other end of the link bundle.

8.

(Optional) Verify the specified Ethernet link bundle.

Example:

Router# show bundle Bundle-Ether 3
Router# show lacp bundle Bundle-Ether 3

Confirm that the bundle is operational, that all expected member links appear, and that LACP is operational on the ports and their peers.


Configure VLAN bundles

Configure 802.1Q VLAN subinterfaces on an Ethernet link bundle to support Layer 3 services on the bundle.

The creation of a VLAN bundle involves three main tasks:

  1. Create an Ethernet bundle.

  2. Create VLAN subinterfaces and assign them to the Ethernet bundle.

  3. Assign Ethernet links to the Ethernet bundle.

Note

For a VLAN bundle to be active, you must perform the same configuration on both ends of the bundle connection.

Procedure

Enter global configuration mode using the configure command.

Example:

Router# configure

What to do next

Perform the following sub-tasks in order to complete the VLAN bundle configuration:

  1. Create an Ethernet bundle

  2. Create VLAN subinterfaces and assign them to the Ethernet bundle

  3. Assign Ethernet links to the Ethernet bundle


Create an Ethernet bundle

Create an Ethernet link bundle to serve as the parent interface for the VLAN subinterfaces that you configure in the next sub-task.

Perform this sub-task from global configuration mode.

Procedure

1.

Create a new Ethernet link bundle using the interface Bundle-Ether bundle-id command.

Example:

Router(config)# interface Bundle-Ether 3
2.

Assign an IP address and subnet mask to the virtual interface.

Example:

Router(config-if)# ipv4 address 10.1.2.3 255.0.0.0
3.

(Optional) Set the number of active links required before you can bring up a specific bundle.

Example:

Router(config-if)# bundle minimum-active links 2
4.

Exit interface configuration submode.

Example:

Router(config-if)# exit

What to do next

Perform the Create VLAN subinterfaces and assign them to the Ethernet bundle sub-task next.


Create VLAN subinterfaces and assign them to the Ethernet bundle

Create one or more 802.1Q VLAN subinterfaces on the Ethernet bundle and configure the Layer 2 encapsulation and IP address for each subinterface.

Perform this sub-task from global configuration mode, after you create the Ethernet bundle.

Procedure

1.

Create a new VLAN subinterface and assign it to the Ethernet bundle you created earlier.

Example:

Router(config)# interface Bundle-Ether 3.1

Replace the bundle-id with the ID you created earlier. Replace the vlan-id with a subinterface identifier. The range is from 1 to 4094, inclusive (0 and 4095 are reserved).

Note

When you include the .vlan-id argument with the interface Bundle-Ether bundle-id command, you enter subinterface configuration mode.

2.

Set the Layer 2 encapsulation of the subinterface.

Example:

Router(config-subif)# encapsulation dot1q 100
3.

Assign an IP address and subnet mask to the subinterface.

Example:

Router(config-subif)# ipv4 address 10.1.2.3/24
4.

(Optional) If a link is in the down state, bring it up.

Example:

Router(config-subif)# no shutdown
5.

Exit subinterface configuration mode for the VLAN subinterface.

Example:

Router(config-subif)# exit
6.

(Optional) Repeat the VLAN subinterface creation steps to add more VLANs to the bundle.

What to do next

Perform the Assign Ethernet links to the Ethernet bundle sub-task next.


Procedure

1.

Enter interface configuration mode for the Ethernet interface to add to the bundle.

Example:

Router(config)# interface GigabitEthernet 1/0/0/0

Enter the GigabitEthernet or TenGigE keyword to specify the interface type. Replace the interface-path-id argument with the node-id in the rack/slot/module format.

2.

(Optional) Disable the wait-while timer in the LACP state machine for faster switchover.

Example:

Router(config-if)# lacp fast-switchover

Use this option if you enabled 1:1 link protection (you set the value of the bundle maximum-active links command to 1) on a bundle with member links running LACP.

3.

Save the configuration changes using the end or commit command.

Example:

Router(config-if)# commit

Configure LACP fallback

Enable LACP fallback so that an active LACP interface can establish a Link Aggregation Group (LAG) port-channel and keep one port active while the server completes its boot process over that port.

This task configures the LACP fallback timeout on a Bundle-Ether interface.

Procedure

1.

Enter global configuration mode using the configure command.

Example:

Router# configure
2.

Create and name a new Ethernet link bundle using the interface Bundle-Ether bundle-id command.

Example:

Router(config)# interface Bundle-Ether 3

The interface Bundle-Ether command enters interface configuration submode, where you can enter interface-specific configuration commands. Use the exit command to exit from the interface configuration submode back to global configuration mode.

3.

Specify a primary IPv4 address for the interface.

Example:

Router(config-if)# ipv4 address 192.168.1.27 255.0.0.0
4.

Enable the LACP Fallback feature using the bundle lacp-fallback timeout number command.

Example:

Router(config-if)# bundle lacp-fallback timeout 4
5.

Save the configuration changes using the end or commit command.

Example:

Router(config-if)# commit
6.

(Optional) Verify the MA information of the bundle manager for the configured bundle.

Example:

Router# show bundle infrastructure database ma bdl-info Bundle-e1010 | inc "fallback"
Router# show bundle infrastructure database ma bdl-info Bundle-e1015 | inc "fallback"

Confirm that the fallback configuration is reflected in the bundle manager output.


Configure the default LACP short period time interval

Enable the LACP short period using the default time interval to speed up detection and recovery from link failures on bundle members.

This task configures the default short period time interval for sending and receiving LACP packets on a Gigabit Ethernet interface. This procedure also enables the LACP short period.

Procedure

1.

Enter global configuration mode using the configure command.

Example:

Router# configure
2.

Create a Gigabit Ethernet interface and enter interface configuration mode.

Example:

Router(config)# interface HundredGigE 0/1/0/1
3.

Specify the bundle interface and put the member interface in active mode.

Example:

Router(config-if)# bundle id 1 mode active
4.

Configure the short period time interval for sending and receiving LACP packets using the default time period of 1000 milliseconds (1 second).

Example:

Router(config-if)# lacp period short
5.

Save the configuration changes using the end or commit command.

Example:

Router(config-if)# commit
When you issue the end command, the system prompts you to commit changes:
Uncommitted changes found, commit them before
exiting(yes/no/cancel)?
[cancel]:
  • yes - Entering yes saves configuration changes to the running configuration file, exits the configuration session, and returns the router to EXEC mode.
  • no - Entering no exits the configuration session and returns the router to EXEC mode without committing the configuration changes.
  • cancel - Entering cancel leaves the router in the current configuration session without exiting or committing the configuration changes.
  • commit - Use the commit command to save the configuration changes to the running configuration file and remain within the configuration session.

Configure custom LACP short period time intervals

Set a custom LACP short period time interval to tune the frequency of LACP packet exchanges for faster detection and recovery from link failures.

This task configures a custom short period time interval for sending and receiving LACP packets on a Gigabit Ethernet interface. The interval can be in the range 100 to 1000 milliseconds, in multiples of 100.

Procedure

1.

Enter global configuration mode using the configure command.

Example:

Router# configure
2.

Create a Gigabit Ethernet interface and enter interface configuration mode.

Example:

Router(config)# interface HundredGigE 0/1/0/1
3.

Specify the bundle interface and put the member interface in active mode.

Example:

Router(config-if)# bundle id 1 mode active
4.

Configure a custom period time interval for sending and receiving LACP packets using the lacp period time-interval command.

Example:

Router(config-if)# lacp period 300

The interval can be in the range 100 to 1000 milliseconds, in multiples of 100.

5.

Save the configuration changes using the end or commit command.

Example:

Router(config-if)# commit
When you issue the end command, the system prompts you to commit changes:
Uncommitted changes found, commit them before
exiting(yes/no/cancel)?
[cancel]:
  • yes - Entering yes saves configuration changes to the running configuration file, exits the configuration session, and returns the router to EXEC mode.
  • no - Entering no exits the configuration session and returns the router to EXEC mode without committing the configuration changes.
  • cancel - Entering cancel leaves the router in the current configuration session without exiting or committing the configuration changes.
  • commit - Use the commit command to save the configuration changes to the running configuration file and remain within the configuration session.

Procedure

1.

Enter interface configuration mode for the bundle member interface.

Example:

Router(config)# interface HundredGigE 0/0/0/34
2.

Designate the member link as unviable using the forwarding-unviable command.

Example:

Router(config-if)# forwarding-unviable
3.

Save the configuration changes using the commit command, and exit the configuration session.

Example:

Router(config-if)# commit
Router(config-if)# end
4.

Verify the running configuration on the interface.

Example:

Router# show running-config interface HundredGigE 0/0/0/34
interface HundredGigE0/0/0/34
 forwarding-unviable
!

Confirm that the forwarding-unviable command is present in the interface configuration.

5.

Verify the forwarding-viable status of LAG members using the show bundle command.

Example:

Router# show bundle

Bundle-Ether3
  Status:                                    Down
  Local links <active/standby/configured>:   0 / 0 / 1
  Local bandwidth <effective/available>:     0 (0) kbps
  MAC address (source):                      78c6.9991.3504 (Chassis pool)
  Inter-chassis link:                        No
  Minimum active links / bandwidth:          1 / 1 kbps
  Maximum active links:                      64
  Wait while timer:                          2000 ms

  Port                  Device           State        Port ID         B/W, kbps
  --------------------  ---------------  -----------  --------------  ----------
  Hu0/0/0/34            Local            Standby      0x8000, 0x0001   100000000
      Link is not forwarding viable and in standby state

Confirm that the member interface is in the standby state and that the output reports "Link is not forwarding viable and in standby state" for that interface.


Before you begin

  • Determine the target interface and the desired recovery method: manual or automatic.

Procedure

1.

Configure link-flap err-disable with a specific count and interval.

Example:

Router# configure
Router(config)# interface GigabitEthernet0/0/0/0
Router(config-if)# fault-management link-flap error-disable count 30 interval 420
Router(config-if)# commit
2.

Verify which interfaces are err-disabled.

Example:

Router# show error-disable
Interface            Error-Disable reason            Retry (s)  Time disabled
-------------------  ------------------------------  ---------  ---------------
Fo0/1/0/25           link-flap-exceeded                    ---         10:43:18
Fo0/1/0/26           link-flap-exceeded                    ---         10:43:18

Confirm the interfaces that are in the err-disabled state and their disable reasons.

3.

Recover an err-disabled interface using one of these methods.

  • Configure automatic recovery to bring up the interface after a specific time using the error-disable recovery cause command.
    Router(config)# error-disable recovery cause link-flap-exceeded interval 30

    The default timer is 300 seconds for the error-disable recovery cause link-flap-exceeded interval command.

  • Manually recover an interface by resetting the interface state.
    Router(config)# interface <interface-type> <interface-path>
    Router(config-if)# shut
    Router(config-if)# no shut
    Router(config-if)# commit
    Note

    Although shut and no shut resets the interface state, the recommended CLIs to clear err-disable are clear error-disable interface and clear error-disable location .

4.

Verify the err-disable state on the interface.

Example:

Router# show error-disable interface TenGigE 0/0/0/1
Interface            Error-Disable reason            Retry (s)  Time disabled
-------------------  ------------------------------  ---------  ---------------
Te0/0/0/1            link-flap-exceeded                    200         11:35:07

Confirm the recovery retry timer and the current disabled duration for the interface.

5.

Clear the err-disable state from an interface explicitly.

Example:

Router# clear error-disable interface Hu0/0/0/0
6.

Clear the err-disable state from all interfaces in a specific location.

Example:

Router# clear error-disable location interface TenGigE 0/0/0/1

Before you begin

Identify the bundle ID and the node that hosts the bundle members.

Procedure

1.

Verify the operational status of the Ethernet link bundle and its member interfaces using these commands.

Example:

Router# show bundle brief
Router# show bundle bundle-ether bundle-id
Router# show interfaces bundle-ether bundle-id

Confirm that the configured-member count is 128. Confirm that the active-member count matches the number of operational members and that every expected member interface appears in the output.

2.

Verify the supported bundle scale for the specified location.

This example output is from a Cisco 8808 8-Slot modular system with an 88-LC2-36EF-M line card.

Example:

Router# show ethernet hardware capability location 0/3/CPU0

System Interface Scale:
    Max L3 Sub-Interfaces                : 30000
    Max L2 Sub-Interfaces                : 131072

  Bundle Scale:
    Max Bundles                          : 1023
    Max Bundle Members per Bundle        : 128

  LC Interface Scale:
    Max L3 Sub-Interfaces                : 16000
    Max L2 Sub-Interfaces                : 57344

  Main Interface Scale:
    Max L3 Sub-Interfaces                : 2000
    Max L2 Sub-Interfaces                : 14336
    Max Sub-Interfaces                   : 14336

This example output is from a Cisco P200 64x800G OSFP 3RU fixed system, 8223-64EF-M.

Example:

Router# show ethernet hardware capability location 0/RP0/CPU0

System Interface Scale:
    Max L3 Sub-Interfaces                : 8000
    Max L2 Sub-Interfaces                : 57344

  Bundle Scale:
    Max Bundles                          : 1023
    Max Bundle Members per Bundle        : 128

  LC Interface Scale:
    Max L3 Sub-Interfaces                : 8000
    Max L2 Sub-Interfaces                : 57344

  Main Interface Scale:
    Max L3 Sub-Interfaces                : 2000
    Max L2 Sub-Interfaces                : 14336
    Max Sub-Interfaces                   : 14336