Cisco IOS XR Troubleshooting Guide for the Cisco XR 12000 Series Router
Troubleshooting Load Balancing on the Cisco XR 12000 Series Router
Downloads: This chapterpdf (PDF - 514.0KB) The complete bookPDF (PDF - 2.34MB) | Feedback

Troubleshooting Load Balancing on the Cisco XR 12000 Series Router

Table Of Contents

Troubleshooting Load Balancing on the Cisco XR 12000 Series Router

Information About Load Balancing with Cisco Express Forwarding

Load Balancing Function

Source Information for Load Balancing

Assignment of Flows to Interfaces

Layer 2 Load Balancing

Terminology

Troubleshooting Unicast IPv4 Load Balancing

Verifying the Routing Table Entries for Parallel Links

Verifying the CEF Database and Measuring Flows

How to Troubleshoot Layer 2 Load Balancing

Verifying the Bundle Status, IGP Route, AIB Database, and CEF Database

Viewing the Expected Paths and Measure the Flows

Configuration Examples for Troubleshooting Load Balancing

Troubleshoting Layer 3 Load Balancing: Example

Verifying the Parallel Links in the Routing Table

Verifying IPv4 Unicast Load Balancing

Troubleshooting Layer 2 Load Balancing: Example

Verifying the Bundle Status

Verifying the IGP Route, AIB Database, and CEF Database

Verifying Bundle Load Balancing

Additional References

Related Documents


Troubleshooting Load Balancing on the Cisco XR 12000 Series Router


This chapter explains how Cisco IOS XR implements Layer 2 (data link layer) and Layer 3 (network layer) load balancing across multiple parallel links when using Cisco Express Forwarding (CEF), in the following sections:

Information About Load Balancing with Cisco Express Forwarding

Troubleshooting Unicast IPv4 Load Balancing

Configuration Examples for Troubleshooting Load Balancing

Additional References

Information About Load Balancing with Cisco Express Forwarding

This section explains load balancing on the Cisco XR 12000 Series Router.

Load Balancing Function

Load balancing is the Cisco IOS XR feature that improves the utilization of parallel links by distributing traffic flows among them. A traffic flow consists of packets that have common Layer 3 characteristics, such as the same source and destination IP addresses. The concept of a flow also extends to Layer 2 bundles and to virtual links such as MPLS traffic-engineering tunnels.

Cisco IOS XR software uses a hash algorithm to identify individual flows and to distribute them across multiple parallel links. Each flow uses a specific link, but there are many flows in a typical production network. Balancing traffic flows is similar to the earlier method of per-destination balancing, which grouped all packets by destination. Grouping packets by flow provides a better degree of granularity, and ultimately balancing, because the hash algorithm considers the source and destination IP addresses, and the router ID, when choosing a path. Cisco IOS XR software does not use the per-packet load balancing option that rotates individual packets around the available links.

Source Information for Load Balancing

Per-flow load balancing in Cisco XR 12000 Series Router uses the universal hash algorithm. This algorithm uses the following information to identify flows.

Layer 3 information from the IP header

Source IP address

Destination IP address

Platform-related information:

Router ID

A hash function computes a unique value from this information, and the hash value is mapped to an available path. The Cisco XR 12000 Series Router supports up to 16 equal-cost paths.

Assignment of Flows to Interfaces

With Cisco Express Forwarding load balancing, each flow is assigned to a single active interface. The flow-to-interface assignment is randomized by a hash function that uses the source and destination IP addresses, along with the router ID, to choose an interface. The show cef ipv4 prefix detail command shows the possible hash results and the interfaces to which they map.

RP/0/5/CPU0:RouterA# show cef ipv4 10.22.23.22 detail
 
   
Mon Nov  3 16:08:39.054 PST DST
10.22.23.22/32, version 0, internal 0x40040001 (0xa8d64558) [1], 0x0 (0xa8d2b01 
 Updated Nov  1 00:07:39.743 
 local adjacency 10.212.16.22
 Prefix Len 32, traffic index 0, precedence routine (0)
  gateway array (0xa8c02d8c) reference count 2, flags 0x400d00, source lsd (2), 
                [3 type 4 flags 0x4101001 (0xa93d77d0) ext 0x0 (0x0)]
  LW-LDI[type=1, refc=1, ptr=0xa8d2b018, sh-ldi=0xa93d77d0]
   via 10.212.16.22, FastEthernet0/2/2/0, 5 dependencies, weight 0, class 0
    next hop 10.212.16.22
    local adjacency
     local label 16070      labels imposed {None}
   via 10.212.20.22, FastEthernet0/2/2/1, 5 dependencies, weight 0, class 0
    next hop 10.212.20.22
    local adjacency
     local label 16070      labels imposed {None}
   via 10.212.24.22, FastEthernet0/2/2/2, 5 dependencies, weight 0, class 0
    next hop 10.212.24.22
    local adjacency
     local label 16070      labels imposed {None}
 
   
    Load distribution: 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 (refcount 3)
 
   
    Hash  OK  Interface                 Address
    0     Y   FastEthernet0/2/2/0       10.212.16.22   
    1     Y   FastEthernet0/2/2/1       10.212.20.22   
    2     Y   FastEthernet0/2/2/2       10.212.24.22   
    3     Y   FastEthernet0/2/2/0       10.212.16.22   
    4     Y   FastEthernet0/2/2/1       10.212.20.22   
    5     Y   FastEthernet0/2/2/2       10.212.24.22   
    6     Y   FastEthernet0/2/2/0       10.212.16.22   
    7     Y   FastEthernet0/2/2/1       10.212.20.22   
    8     Y   FastEthernet0/2/2/2       10.212.24.22   
    9     Y   FastEthernet0/2/2/0       10.212.16.22   
    10    Y   FastEthernet0/2/2/1       10.212.20.22   
    11    Y   FastEthernet0/2/2/2       10.212.24.22   
    12    Y   FastEthernet0/2/2/0       10.212.16.22   
    13    Y   FastEthernet0/2/2/1       10.212.20.22   
    14    Y   FastEthernet0/2/2/2       10.212.24.22 
 
   

There are 16 possible hash values and a hash table maps each value to an interface. The hash table lists the active interfaces, repeated as many times as necessary to fill the 16 table rows. For an even number of interfaces, the hash table is filled completely. For an odd number of interfaces, the interfaces are repeated fully as many times as possible, and the unused hash values are disabled. In this example there are three interfaces so they are repeated 5 times for a total of 15 entries, and the last hash value (15) is disabled. The following table shows how CEF initializes the hash values for two and three active paths.

Table 11-1 Distribution of Hash Values to Interfaces

 
Hash Values
Paths
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

2

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

3

0

1

2

0

1

2

0

1

2

0

1

2

0

1

2

-


All packets in a flow use a single egress interface. Balancing occurs because a typical network has a large number of different flows that are distributed among the parallel interfaces.

Layer 2 Load Balancing

Load balancing at Layer 2 is also done by flow. Cisco IOS XR software provides load balancing for Layer 2 bundles only. For all IP traffic passing over a bundle interface, load balancing is done by the Forwarding Information Base (FIB) on the ingress and egress line cards. When a received packet is switched to a bundle interface, the FIB chooses which member link to use, based on the source and destination IP address of the packets. The Adjacency Information Base (AIB) stores a list of adjacencies that the FIB uses to determine which member links are available for forwarding.

Layer 2 load balancing uses the 3-tuple algorithm, so it uses the source and destination IP addresses in the underlying packet. A modulo operation on the hash result is subsequently performed using the number of entries in the load balancing table. The hash table for Layer two supports eight hash values, instead of the 16 that are supported for Layer 3 load balancing.

There is nothing to configure for Layer 2 load balancing. Active links in the bundle are automatically eligible for load balancing.

Terminology

The following terms apply to load balancing.

Term
Description

Prefix

Describes a destination IP network, such as 192.16.10.0/24. Cisco IOS XR software adds a destination IP prefix to the routing table, using information obtained from exchanging messages using a dynamic routing protocol or by manual configuration of static routes.

Path

Describes a valid route to reach a destination prefix. Cisco IOS XR software assigns a cost to each path. A set of active paths to a destination prefix may have equal or unequal costs. Loads can be balanced across equal-cost paths.

Session

Describes a unidirectional communication flow between two IP nodes. All packets in a session use the same source and destination IP addresses.

Flow

A network flow is a unidirectional sequence of packets that have common header fields, which can include the following:

Source IP address

Destination IP address

IP protocol

Source port (for example UDP or TCP port)

Destination port

Ingress interface

IP type of service

Note The Cisco XR 12000 Series Router uses the first three fields only to identify flows.


Troubleshooting Unicast IPv4 Load Balancing

This section describes how to troubleshoot load balancing for Layer 3 IPv4 unicast flows when Open Shortest Path First (OSPF) is the Interior Gateway Protocol (IGP).

Verifying the Routing Table Entries for Parallel Links

Verifying the CEF Database and Measuring Flows

Verifying the Routing Table Entries for Parallel Links

Cisco Express Forwarding uses the path information in the IP routing table to balance traffic over multiple links. For this reason, confirming correct Cisco Express Forwarding (CEF) load balancing begins with confirming the contents of the IP routing table.


Note Unlike configuration procedures, troubleshooting is not a deterministic process. This section provides a typical procedure for troubleshooting load balancing when OPSF is the IGP. For an example of this procedure, see Troubleshoting Layer 3 Load Balancing: Example.


SUMMARY STEPS

1. show route destination-address

2. configure

3. router ospf process

4. maximum paths number

5. end

6. show route destination-address

7. show ospf process interface brief

8. show running-config router ospf process

9. configure

10. router ospf process area area

11. interface type interface-path-id

12. cost value

13. end

14. show route destination-address

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

show route destination-address

Example:

RP/0/5/CPU0:RouterA# show route 10.22.23.22

Displays the routes to a destination address. Use a destination address on another host that is reachable through the parallel links.

Verify that number of routes in the routing table equals the number of parallel links. If you have fewer routes than expected, continue with this procedure.

Step 2 

configure

Example:

RP/0/5/CPU0:RouterA# configure

Enters global configuration mode.

Step 3 

router ospf process

Example:

RP/0/5/CPU0:RouterA(config)# router ospf 200

Enters the configuration mode for the OSPF process.

Step 4 

maximum paths number

Example:

RP/0/5/CPU0:RouterA(config-ospf)# maximum paths 3

Configures the maximum number of paths over which to load balance. By default, OSPF balances up to 4 equal-cost paths.

Step 5 

end

Example:

RP/0/5/CPU0:RouterA(config-ospf)# exit

Ends the configuration process. Enter yes at the prompt to commit the changes.

Step 6 

show route destination-address

Example:

RP/0/5/CPU0:RouterA# show route 10.22.23.22

Displays the routes to a destination address.

Verify that number of routes in the routing table equals the number of parallel links. If you have fewer routes than expected, continue with this procedure.

Step 7 

show ospf process interface brief

Example:

RP/0/5/CPU0:RouterA# show ospf 200 interface brief

Shows interface information for all routes to the destination address, which displays the cost metric. OSPF balances loads over equal-cost routes only, so verify that the interfaces have equal costs. To load balance over unequal paths, use Enhanced Interior Gateway Routing Protocol or Interior Gateway Routing Protocol (EIGRP/IGRP) as the IGP instead.

Step 8 

show running-config router ospf process

Example:

RP/0/5/CPU0:RouterA# show run router ospf 200

Displays the running configuration for the OSPF process. This is another way to determine if the interfaces have different costs.

Step 9 

configure

Example:

RP/0/5/CPU0:RouterA# configure

Enters global configuration mode.

Step 10 

router ospf process area area

Example:

RP/0/5/CPU0:RouterA(config)# router ospf 200 area 0

Enters OSPF area configuration mode.

Step 11 

interface type interface-path-id

Example:

RP/0/5/CPU0:RouterA(config-ospf-ar)# interface FastEthernet 0/2/2/2

Enters interface configuration mode.

Step 12 

cost value

Example:

RP/0/5/CPU0:RouterA(config-ospf-ar-if)# no cost 10

Sets the cost of this interface to the same value as the cost of the other parallel interfaces. If necessary, repeat the previous step and this step for other interfaces.

Step 13 

end

Example:

RP/0/5/CPU0:RouterA(config-ospf-ar-if)# end

Ends the configuration process. Enter yes at the prompt to commit the changes.

Step 14 

show route destination-address

Example:

RP/0/5/CPU0:RouterA# show route 10.22.23.22

Displays the routes to a destination address.

Verify that number of routes in the routing table equals the number of parallel links.

Verifying the CEF Database and Measuring Flows

Cisco Express Forwarding uses the path information in the Cisco Express Forwarding (CEF) database to balance traffic over multiple links. For this reason, verifying correct CEF load balancing begins with confirming the contents of the CEF database.

SUMMARY STEPS

1. show cef ipv4 [prefix [mask]] [detail]

2. show cef ipv4 exact-route source-address destination-address [detail]

3. show interfaces accounting rates

4. show interfaces [type instance]

5. show cef ipv4 prefix hardware ingress location location

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

show cef ipv4 [prefix [mask]] [detail]

Example:

RP/0/5/CPU0:RouterA# show cef ipv4 10.22.23.22 detail

Displays the CEF forwarding table. Verify that it contains the same interfaces that the routing table has for this destination.

Step 2 

show cef ipv4 exact-route source-address destination-address [detail]

Example:

RP/0/5/CPU0:RouterA# show cef exact-route 192.168.254.1 10.22.23.22

Displays the exact route that a specific flow would take. Use this command for several flows to verify that they are distributed equally over the parallel interfaces.

Step 3 

show interfaces accounting rates

Example:

RP/0/5/CPU0:RouterA# show interfaces accounting rates

Displays the traffic rates of all interfaces. Use this command to verify that the traffic takes the expected egress interface.

Step 4 

show interfaces [type instance]

Example:

RP/0/5/CPU0:RouterA# show interfaces GigabitEthernet0/3/0/3

Displays the traffic rates of individual interfaces. Use this command to display individual interface traffic counters.

Step 5 

show cef ipv4 prefix hardware ingress location location

Example:

RP/0/5/CPU0:RouterA# show cef ipv4 10.22.23.22 hardware ingress location 0/2/CPU0

Displays the adjacency traffic counters, for those cases in which a single interface supports multiple CEF adjacencies. Use this command to verify that the traffic takes the expected CEF adjacency.

How to Troubleshoot Layer 2 Load Balancing

This section describes how to troubleshoot load balancing at Layer 2. This procedure is specific to Layer 2 bundles.

Verifying the Bundle Status, IGP Route, AIB Database, and CEF Database

Viewing the Expected Paths and Measure the Flows

Viewing the Expected Paths and Measure the Flows

Verifying the Bundle Status, IGP Route, AIB Database, and CEF Database

Cisco Express Forwarding uses the path information in the IP routing table to balance traffic over multiple links. For this reason, verifying correct Cisco Express Forwarding load balancing begins with confirming the contents of the IP routing table. When troubleshooting a bundle, verify that the bundle is up and has the expected links, that the IGP route to the desired destination includes the bundle interface, and that the AIB database is complete.


Note Unlike configuration procedures, troubleshooting is not a deterministic process. This section provides a typical procedure for troubleshooting load balancing within an Ethernet bundle. For an example of this procedure, see Troubleshooting Layer 2 Load Balancing: Example.


SUMMARY STEPS

1. show bundle {Bundle-Ether | Bundle-POS} interface-path-id

2. show route destination-address

3. show cef ipv4 destination-address

4. show adjacency ipv4 bundle-ether interface-path-id detail hardware

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

show bundle {Bundle-Ether | Bundle-POS} interface-path-id

Example:

RP/0/5/CPU0:Router# show bundle bundle-ether bundle-ether12

Displays the bundle status.

Verify that the bundle has the expected number of links. If not, troubleshoot the bundle first.

Step 2 

show route destination-address

Example:

RP/0/5/CPU0:Router# show route 10.11.12.11

Displays the routes to a destination address. Use a destination address on another host that is reachable through the bundle.

Verify that the route to the destination address includes the bundle interface. If not, make sure that the bundle interface is included in the IGP process configuration.

Step 3 

show cef ipv4 prefix

Example:

RP/0/5/CPU0:router# show cef ipv4 10.11.12.11

Displays the CEF forwarding table. Verify that it contains the same bundle interface that the routing table has for this destination.

Step 4 

show adjacency ipv4 bundle-ether interface-path-id detail hardware

Example:

RP/0/5/CPU0:RouterA# show adjacency ipv4 bundle-ether 12 detail hardware

Displays the adjacency database. Verify that bundle information is displayed for each linecard and the route processor.

Viewing the Expected Paths and Measure the Flows

Cisco IOS XR software provides a bundle utility that predicts how Layer 2 loads are balanced across member links. This is an interactive toolthat prompts for the information that the load balancing algorithm uses to allocate flows to member links.

SUMMARY STEPS

1. bundle-hash {Bundle-Ether | Bundle-Pos} interface

2. show interfaces [type interface-path-id]

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

bundle-hash {Bundle-Ether | Bundle-Pos} interface

Example:

RP/0/5/CPU0:router# bundle-hash bundle-ether 12

Launches the bundle-hash utility. This is an interactive utility that prompts for the necessary information.

Step 2 

show interfaces [type interface-path-id]

Example:

RP/0/5/CPU0:router# show interfaces Gi0/6/5/5

Displays interface information, which includes the traffic rates. Use this command for each link in the bundle to verify that the simulated traffic takes the expected link. Use clear counters to make it easier to view the traffic allocation.

Configuration Examples for Troubleshooting Load Balancing

This section provides the following configuration examples:

Troubleshoting Layer 3 Load Balancing: Example

Troubleshooting Layer 2 Load Balancing: Example

Troubleshoting Layer 3 Load Balancing: Example

Table 11-2 shows the sample configuration that this section uses to describe the troubleshooting process on the Cisco XR 12000 Series Router. Two routers, Router A and Router B, connect back to back over three serial Gigabit Ethernet interfaces.

Table 11-2 Sample Configuration

Router A
Router B
interface FastEthernet0/2/2/0
 ipv4 address 10.212.16.21 255.255.255.0
!
interface FastEthernet0/2/2/1
 ipv4 address 10.212.20.21 255.255.255.0
!
interface FastEthernet0/2/2/2
 ipv4 address 10.212.24.21 255.255.255.0
 
        
router ospf 200
maximum paths 1
!
area 0
!
  interface FastEthernet0/2/2/0
  interface FastEthernet0/2/2/1
  interface FastEthernet0/2/2/2
  cost 10
interface FastEthernet0/2/2/0
 ipv4 address 10.212.16.22 255.255.255.0
!
interface FastEthernet0/2/2/1
 ipv4 address 10.212.20.22 255.255.255.0
!
interface FastEthernet0/2/2/2
 ipv4 address 10.212.24.22 255.255.255.0
 
        
router ospf 200
area 0
!
  interface FastEthernet0/2/2/0
  interface FastEthernet0/2/2/1
  interface FastEthernet0/2/2/2 

The bold text identifies those configurations that affect load balancing.

Verifying the Parallel Links in the Routing Table

Troubleshooting of load balancing starts with the routing table, as described in the following sections:

Checking the IGP Routing Table

Checking the Maximum Paths

Checking the Maximum Paths

Checking the IGP Routing Table

This section shows how Router A selects one or more paths to reach the loopback interface on Router B, 10.22.23.22/32.

By default, OSPF supports four equal-cost paths to a destination. In this scenario, Router B is configured with maximum-paths equal to one, so it chooses only one path from the possible equal-cost paths. Use the show route command to view the current path to Router A:

RP/0/5/CPU0:RouterA# show route 10.22.23.22
 
   
Fri Oct 31 22:15:10.923 PST DST
 
   
Routing entry for 10.22.23.22/32
  Known via "ospf 200", distance 110, metric 2, type intra area
  Installed Oct 31 22:15:02.580 for 00:00:08
  Routing Descriptor Blocks
    10.212.16.22, from 10.22.23.22, via FastEthernet0/2/2/0
      Route metric is 2
  No advertising protos. 
 
   

This shows that RouterA selected FastEthernet0/2/2/0 as the single path to 10.22.23.22. Use the show cef command to view the corresponding CEF path.

RP/0/5/CPU0:RouterA# show cef ipv4 10.22.23.22
 
   
Fri Oct 31 22:25:59.465 PST DST
10.22.23.22/32, version 0, internal 0x40040001 (0xa8d64558) [1], 0x0 (0xa8d2b018), 0x4400 
(0xa93a83b4)  
 Updated Oct 31 22:15:02.602 
 local adjacency 10.212.16.22
 Prefix Len 32, traffic index 0, precedence routine (0)
   via 10.212.16.22, FastEthernet0/2/2/0, 5 dependencies, weight 0, class 0
    next hop 10.212.16.22
    local adjacency
     local label 16070      labels imposed {None}
 
   

This verifies that the CEF database also has FastEthernet0/2/2/0 as the egress interface for traffic forwarded to 10.22.23.22. No load balancing can occur in this configuration, because there are three parallel links but only one is allowed.

Checking the Maximum Paths

Use the maximum-paths subcommand in the OSPF configuration mode to allow up to three paths in the routing table:

RP/0/5/CPU0:RouterA# configure 
Fri Oct 31 22:35:21.972 PST DST
RP/0/5/CPU0:RouterA(config)# router ospf 200
RP/0/5/CPU0:RouterA(config-ospf)# maximum paths 3
RP/0/5/CPU0:RouterA(config-ospf)# 
 
   

Use the show route command to confirm that the routing table contains the desired number of paths:

RP/0/5/CPU0:RouterA# show route 10.22.23.22   
 
   
Fri Oct 31 23:56:05.238 PST DST
 
   
Routing entry for 10.22.23.22/32
  Known via "ospf 200", distance 110, metric 2, type intra area
  Installed Oct 31 22:35:39.816 for 01:20:25
  Routing Descriptor Blocks
    10.212.16.22, from 10.22.23.22, via FastEthernet0/2/2/0
      Route metric is 2
    10.212.20.22, from 10.22.23.22, via FastEthernet0/2/2/1
      Route metric is 2
  No advertising protos. 
 
   

OSPF allows equal-cost paths only, so one of the three paths must have a higher cost.

Checking the Route Metric

Although OSPF is configured to support three equal-cost paths, only two active paths are displayed in the routing table. Use the show ospf interface command to determine the reason:

RP/0/5/CPU0:RouterA# show ospf 200 interface brief
 
   
Fri Oct 31 23:57:57.584 PST DST
 
   
* Indicates MADJ interface
 
   
Interfaces for OSPF 200
 
   
Interface    PID   Area            IP Address/Mask    Cost  State Nbrs F/C
Fa0/2/2/0    200   0               10.212.16.21/24    1     DR    1/1
Fa0/2/2/1    200   0               10.212.20.21/24    1     DR    1/1
Fa0/2/2/2    200   0               10.212.24.21/24    10    DR    1/1
 
   

Fa0/2/2/2 has a higher cost than Fa0/2/2/0 and Fa0/2/2/1, so it is excluded from the routes. Use the show run command to confirm that Fa0/2/2/2 is configured with the cost 10 command:

RP/0/5/CPU0:RouterA# show run router ospf 200
 
   
Sat Nov  1 00:00:19.499 PST DST
router ospf 200
 router-id 10.21.22.21
 maximum paths 3
 area 0
  interface Loopback1
  !
  interface FastEthernet0/2/2/0
  !
  interface FastEthernet0/2/2/1
  !
  interface FastEthernet0/2/2/2
   cost 10
 
   

Use the no cost interface subcommand in router ospf area configuration mode to restore the default cost to Fa0/2/2/2:

RP/0/5/CPU0:RouterA# configure 
Sat Nov  1 00:07:18.382 PST DST
RP/0/5/CPU0:RouterA(config)# router ospf 200 area 0
RP/0/5/CPU0:RouterA(config-ospf-ar)# interface Fa0/2/2/2
RP/0/5/CPU0:RouterA(config-ospf-ar-if)# no cost 10
RP/0/5/CPU0:RouterA(config-ospf-ar-if)# end
 
   

The show route command now displays three paths to the 194.168.20.0 network:

RP/0/5/CPU0:RouterA# show route 10.22.23.22   
 
   
Sat Nov  1 00:08:40.524 PST DST
 
   
Routing entry for 10.22.23.22/32
  Known via "ospf 200", distance 110, metric 2, type intra area
  Installed Nov  1 00:07:39.714 for 00:01:00
  Routing Descriptor Blocks
    10.212.16.22, from 10.22.23.22, via FastEthernet0/2/2/0
      Route metric is 2
    10.212.20.22, from 10.22.23.22, via FastEthernet0/2/2/1
      Route metric is 2
    10.212.24.22, from 10.22.23.22, via FastEthernet0/2/2/2
      Route metric is 2
  No advertising protos. 
 
   

The next step is to look at how Cisco Express Forwarding uses the information in the routing table to forward packets.

Verifying IPv4 Unicast Load Balancing

CEF load balancing for Layer 3 flows (3-tuple algorithm) is enabled by default, but Layer 4 flows are used in this example. Using Layer 4 information has the advantage of distributing flows more evenly over the parallel links, because it utilizes up to seven packet and platform parameters when calculating the egress interface. The same principles apply to Layer 3 flows, except that the load balancing algorithm checks only the source address, destination address, and router ID.

The troubleshooting process for Layer 3 and Layer 4 load balancing is described in the following sections:

Checking the CEF Database

Checking the Load Balancing

Checking the CEF Database

Verify that the CEF database has the expected interfaces to the test destination, 10.22.23.22:

RP/0/5/CPU0:RouterA# show cef ipv4 10.22.23.22 detail
 
   
Sat Nov  1 04:14:21.731 PST DST
10.22.23.22/32, version 0, internal 0x40040001 (0xa8d64558) [1], 0x0 (0xa8d2b018), 0x4400 
(0xa970e050)  
 Updated Nov  1 00:07:39.743 
 local adjacency 10.212.16.22
 Prefix Len 32, traffic index 0, precedence routine (0)
  gateway array (0xa8c02d8c) reference count 2, flags 0x400d00, source lsd (2), 
                [3 type 4 flags 0x4101001 (0xa93d77d0) ext 0x0 (0x0)]
  LW-LDI[type=1, refc=1, ptr=0xa8d2b018, sh-ldi=0xa93d77d0]
   via 10.212.16.22, FastEthernet0/2/2/0, 5 dependencies, weight 0, class 0
    next hop 10.212.16.22
    local adjacency
     local label 16070      labels imposed {None}
   via 10.212.20.22, FastEthernet0/2/2/1, 5 dependencies, weight 0, class 0
    next hop 10.212.20.22
    local adjacency
     local label 16070      labels imposed {None}
   via 10.212.24.22, FastEthernet0/2/2/2, 5 dependencies, weight 0, class 0
    next hop 10.212.24.22
    local adjacency
     local label 16070      labels imposed {None}
 
   
 
   
    Load distribution: 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 (refcount 3)
 
   
    Hash  OK  Interface                 Address
    0     Y   FastEthernet0/2/2/0       10.212.16.22   
    1     Y   FastEthernet0/2/2/1       10.212.20.22   
    2     Y   FastEthernet0/2/2/2       10.212.24.22   
    3     Y   FastEthernet0/2/2/0       10.212.16.22   
    4     Y   FastEthernet0/2/2/1       10.212.20.22   
    5     Y   FastEthernet0/2/2/2       10.212.24.22   
    6     Y   FastEthernet0/2/2/0       10.212.16.22   
    7     Y   FastEthernet0/2/2/1       10.212.20.22   
    8     Y   FastEthernet0/2/2/2       10.212.24.22   
    9     Y   FastEthernet0/2/2/0       10.212.16.22   
    10    Y   FastEthernet0/2/2/1       10.212.20.22   
    11    Y   FastEthernet0/2/2/2       10.212.24.22   
    12    Y   FastEthernet0/2/2/0       10.212.16.22   
    13    Y   FastEthernet0/2/2/1       10.212.20.22   
    14    Y   FastEthernet0/2/2/2       10.212.24.22 
 
   

This shows that the CEF database does contain the three parallel links to 10.22.23.22 and shows the hash table that maps the hash values to interfaces. Therefore, CEF can correctly balance loads across these three interfaces.

Checking the Load Balancing

Now you can verify that the flows are actually distributed among the parallel interfaces. The show cef exact-route command displays the expected result of the hash calculation.

In this example, a traffic generator is sending TCP packets from 192.168.254.1 to 10.22.23.22 through Router A. The following command displays the calculated egress interface for this flow:

RP/0/5/CPU0:RouterA#  show cef exact-route 192.168.254.1 10.22.23.22 
 
   
Sat Nov  1 04:12:32.792 PST DST
10.22.23.22/32, version 0, internal 0x40040001 (0xa8d64558) [1], 0x0 (0xa8d2b018), 0x4400 
(0xa970e050)  
 Updated Nov  1 00:07:39.743 
 local adjacency 10.212.24.22
 Prefix Len 32, traffic index 0, precedence routine (0)
   via FastEthernet0/2/2/2
 
   

CEF load balancing sends this particular flow out interface FastEthernet0/2/2/2.

You can verify that a flow with the specified characteristics is actually using FastEthernet0/2/2/2 using the show interfaces accounting rates command:

RP/0/5/CPU0:RouterA# show interfaces accounting rates
 
   
Sat Nov  1 04:00:32.648 PST DST
FirewallManagement1
FastEthernet0/2/2/0
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  IPV4_UNICAST                0                0            0                0
  ARP                         0                0            0                0
 
   
FastEthernet0/2/2/1
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  ARP                         0                0            0                0
 
   
FastEthernet0/2/2/2
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  IPV4_UNICAST                0                0         4000                0
  ARP                         0                0            0                0
 
   
FastEthernet0/2/2/7
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  IPV4_UNICAST             4000                0            0                0
  ARP                         0                0            0                0
 
   

This shows that the ingress flow on GigabitEthernet0/6/5/4 and that the egress data rate on interface FastEthernet0/2/2/2 is 4000 bps. Because there is only one flow, this verifies that the flow is using the predicted egress interface. An actual network has more traffic; so to the extent possible, reduce existing traffic to troubleshoot the problem at hand.

Other flows usually use a different egress interface, because the hash algorithm produces a different hash value. For example, a second flow to the same destination address but from a different source address is forwarded out FastEthernet0/2/2/0.

RP/0/5/CPU0:RouterA#  show cef exact-route 192.168.253.1 10.22.23.22     
 
   
Sat Nov  1 05:34:39.170 PST DST
10.22.23.22/32, version 0, internal 0x40040001 (0xa8d64558) [1], 0x0 (0xa8d2b018), 0x4400 
(0xa970e050)  
 Updated Nov  1 00:07:39.743 
 local adjacency 10.212.16.22
 Prefix Len 32, traffic index 0, precedence routine (0)
   via FastEthernet0/2/2/0
 
   

After adding the second flow and waiting for the traffic counters to stabilize, the display now shows that the ingress flow on FastEthernet0/2/2/7 has doubled and that traffic is distributed equally between two egress interfaces, FastEthernet0/2/2/0 and FastEthernet0/2/2/2, as expected for the two flows.

RP/0/5/CPU0:RouterA# show interfaces accounting rates
 
   
Tue Nov  4 15:01:48.302 PST DST
FirewallManagement1
FastEthernet0/2/2/0
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  IPV4_UNICAST                0                0         4000                0
  ARP                         0                0            0                0
 
   
FastEthernet0/2/2/1
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  IPV4_UNICAST                0                0            0                0
  ARP                         0                0            0                0
 
   
FastEthernet0/2/2/2
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  IPV4_UNICAST                0                0         4000                0
  ARP                         0                0            0                0
 
   
FastEthernet0/2/2/3
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  ARP                         0                0            0                0
 
   
FastEthernet0/2/2/4
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  ARP                         0                0            0                0
 
   
No accounting statistics available for FastEthernet0/2/2/5
 
   
No accounting statistics available for FastEthernet0/2/2/6
 
   
FastEthernet0/2/2/7
                                Ingress                        Egress          
  Protocol             Bits/sec         Pkts/sec     Bits/sec         Pkts/sec
  IPV4_UNICAST             9000               10            0                0
  ARP                         0                0            0                0
 
   

The network topology used in this example has a single CEF adjacency for each interface. Therefore, you can view the total interface traffic to verify correct load balancing. In more complex networks, a single interface can have multiple CEF adjacencies. So to view the traffic counters for each adjacency, you must view the hardware counters instead of the interface counters. The following example shows how to display the hardware adjacency counters for the two flows in this example:

RP/0/5/CPU0:RuterA# show cef ipv4 10.22.23.22 hardware ingress location 0/2/CPU0  
 
   
Wed Nov  5 17:57:17.263 PST DST
10.22.23.22/32, version 0, internal 0x40040001 (0x51d50fec) [1], 0x0 (0x51d1b4c0), 0x4400 
(0x528f1050)  
 Updated Nov  5 17:47:30.291 
 local adjacency 10.212.16.22
 Prefix Len 32, traffic index 0, precedence routine (0)
   via 10.212.16.22, FastEthernet0/2/2/0, 5 dependencies, weight 0, class 0
    next hop 10.212.16.22
    local adjacency
     local label 16070      labels imposed {None}
   via 10.212.20.22, FastEthernet0/2/2/1, 5 dependencies, weight 0, class 0
    next hop 10.212.20.22
    local adjacency
     local label 16070      labels imposed {None}
   via 10.212.24.22, FastEthernet0/2/2/2, 5 dependencies, weight 0, class 0
    next hop 10.212.24.22
    local adjacency
     local label 16070      labels imposed {None}
 
   
 
   
 Hardware Leaf:      TCAM offset 0x00000b04
 Hardware Leaf:      Value 
[ 0x4000000a 16171600 00000000 00000000 00000000 ] 
 
   
 Hardware Leaf:      Mask 
[ 0x01ffff00 00000000 00000000 00000000 00000000 ] 
 
   
 Hardware Leaf:      PLU Leaf Value 
[ 0x00000000 1fff0000 80000000 028288c6 ] 
 
   
 FCR  2 TLU Address 0x000a2319 TI 0 AS 0 
 
   
VPN Label 1 0 
 
   
 ************* IGP LoadInfo *****************
 Loadinfo HW Max Index  14
 Loadinfo SW Max Index  14
 PBTS Loadinfo Attached: No
 LI Path [ 0]  HFA Info: 0x10202240   FCR: 4
 LI Path [ 1]  HFA Info: 0x1020223c   FCR: 4
 LI Path [ 2]  HFA Info: 0x10202238   FCR: 4
 LI Path [ 3]  HFA Info: 0x10202240   FCR: 4
 LI Path [ 4]  HFA Info: 0x1020223c   FCR: 4
 LI Path [ 5]  HFA Info: 0x10202238   FCR: 4
 LI Path [ 6]  HFA Info: 0x10202240   FCR: 4
 LI Path [ 7]  HFA Info: 0x1020223c   FCR: 4
 LI Path [ 8]  HFA Info: 0x10202238   FCR: 4
 LI Path [ 9]  HFA Info: 0x10202240   FCR: 4
 LI Path [10]  HFA Info: 0x1020223c   FCR: 4
 LI Path [11]  HFA Info: 0x10202238   FCR: 4
 LI Path [12]  HFA Info: 0x10202240   FCR: 4
 LI Path [13]  HFA Info: 0x1020223c   FCR: 4
 LI Path [14]  HFA Info: 0x10202238   FCR: 4
 ********************************************
 
   
------------------------------------------------
 HW Rx Adjacency  0 Detail:
------------------------------------------------
    Rx Adj HW Address   0x02022400  (ADJ)  
    packets 5509  bytes 297486 
    HFA Bits 0x80 gp 45 mtu 9248 (Fabric MTU) TAG length 0 
    OI 0x8040e (Tx uidb 1 PPindex 1038) 
    OutputQ 0 Output-port 0x0 local-outputq 0x8800 
 
   
[ 0x80182d40 00002420 0008040e 00008800 ]
[ 0x00000000 00000000 00000000 00000000 ]
[ 0x00000000 00000000 00000000 00000000 ] 
 
   
------------------------------------------------
 HW Rx Adjacency  1 Detail:
------------------------------------------------
    Rx Adj HW Address   0x020223c0  (ADJ)  
    packets 0  bytes 0 
    HFA Bits 0x80 gp 45 mtu 9248 (Fabric MTU) TAG length 0 
    OI 0x10040d (Tx uidb 2 PPindex 1037) 
    OutputQ 0 Output-port 0x0 local-outputq 0x8800 
 
   
[ 0x80182d40 00002420 0010040d 00008800 ]
[ 0x00000000 00000000 00000000 00000000 ]
[ 0x00000000 00000000 00000000 00000000 ] 
 
   
------------------------------------------------
 HW Rx Adjacency  2 Detail:
------------------------------------------------
    Rx Adj HW Address   0x02022380  (ADJ)  
    packets 5509  bytes 297486 
    HFA Bits 0x80 gp 45 mtu 9248 (Fabric MTU) TAG length 0 
    OI 0x18040c (Tx uidb 3 PPindex 1036) 
    OutputQ 0 Output-port 0x0 local-outputq 0x8800 
 
   
[ 0x80182d40 00002420 0018040c 00008800 ]
[ 0x00000000 00000000 00000000 00000000 ]
[ 0x00000000 00000000 00000000 00000000 ] 
 
   

The last three sections show the counters for adjacencies 0, 1, and 2, which correspond to the three interfaces, Fa0/2/2/0, Fa0/2/2/1, and Fa0/2/2/2. The two flows are using adjacency 0 and adjacency 1 equally. Each of these interfaces has a single CEF adjacency, but in a more complex network, this command would show the traffic for each adjacency, not just the total interface traffic.

The results in this example were obtained with a traffic generator, but you can obtain similar results using the extended ping command. Just make sure that the ping packets pass through the router to reach the destination IP address.

Troubleshooting Layer 2 Load Balancing: Example

Table 11-3 shows the sample configuration that this section uses to describe the troubleshooting process for Layer 2 load balancing. Two routers, Router A and Router B, connect back to back over two Gigabit Ethernet interfaces that are bundled into a single interface.

Table 11-3 Sample Configuration for Layer 2 Load Balancing

Router A
Router B
interface Bundle-Ether12
 description Connected to Router B
 ipv4 address 10.221.12.22 255.255.255.0
!
interface GigabitEthernet0/3/0/3
  bundle id 12 mode active
!
interface GigabitEthernet0/3/0/4
 bundle id 12 mode active
!
router ospf 200
 router-id 10.22.23.22
 area 0
  interface Bundle-Ether12
interface Bundle-Ether12
 description Connected to Router A
 ipv4 address 10.221.12.11 255.255.255.0
!
interface GigabitEthernet0/2/3/5
 bundle id 12 mode active
!
interface GigabitEthernet0/2/3/7
 bundle id 12 mode active
!
router ospf 200
 router-id 10.11.12.11
 area 0
  interface Bundle-Ether12

Verifying the Bundle Status

The configuration in Table 11-3 shows that a bundle is configured. Enter the show bundle command to verify the configuration:

RP/0/5/CPU0:RouterA# show bundle bundle-ether 12
 
   
Mon Nov 17 22:28:59.098 PST DST
State: 0 - Port is Detached. 1 - Port is Waiting.
       2 - Port is Attached. 3 - Port is Collecting.
       4 - Port is Distributing.
 
   
Bundle-Ether12
  Bandwidth (Kbps)                          Min active          Max active
  Effective   Available    MAC address      Links  B/W (Kbps)   Links
  ----------  ----------   --------------   -----  ----------   -----
     2000000     2000000   0019.aaa3.b3fb       1     1000000       8
 
   
  Port           State  Port ID          B/W (Kbps)  MAC address
  ------------   -----  --------------   ----------  ---------------
  Gi0/3/0/3      4      0x8000, 0x0001      1000000  0019.aaa3.b3fb*
  Gi0/3/0/4      4      0x8000, 0x0002      1000000  0019.aaa3.b3fc 
 
   

This verifies that the bundle is active and is using the two expected interfaces (ports). If the bundle on your system is not active, troubleshoot that problem first. It might not have the minimum number of active links or minimum bandwidth to come up.

Verifying the IGP Route, AIB Database, and CEF Database

Now verify that Router A uses Bundle-Ether12 to reach the loopback interface of Router B, 10.11.12.11. Use the show route command to view the current path:

RP/0/5/CPU0:RouterA# show route 10.11.12.11
 
   
Mon Nov 17 22:42:50.031 PST DST
 
   
Routing entry for 10.11.12.11/32
  Known via "ospf 200", distance 110, metric 2, type intra area
  Installed Nov 17 22:42:04.857 for 00:00:45
  Routing Descriptor Blocks
    10.221.12.11, from 10.11.12.11, via Bundle-Ether12
      Route metric is 2
  No advertising protos. 
 
   

This shows that Router B selected Bundle-Ether12 as the single path to 10.11.12.11. Use the show cef command to view the corresponding CEF path:

RP/0/5/CPU0:RouterA# show ip cef 10.11.12.11
 
   
Mon Nov 17 22:44:36.238 PST DST
10.11.12.11/32, version 0, internal 0x40040001 (0xa8d934c8) [1], 0x0 (0xa8d2bfb4), 0x4400 
(0xa93a8d0c)  
 Updated Nov 17 22:42:04.864 
 local adjacency 10.221.12.11
 Prefix Len 32, traffic index 0, precedence routine (0)
   via 10.221.12.11, Bundle-Ether12, 5 dependencies, weight 0, class 0
    next hop 10.221.12.11
    local adjacency
     local label 16102      labels imposed {None}
 
   

This verifies that CEF also uses Bundle-Ether12 as the egress interface for traffic forwarded to 10.11.12.11. No Layer 3 or Layer 4 load balancing can occur in this configuration, because there is only one link. Layer 2 load balancing can occur, however, because the single link is a bundle of two separate interfaces.

Now verify that theAdjacency Information Base (AIB) has the necessary information for the bundle:

RP/0/5/CPU0:RouterA# show adjacency ipv4 bundle-ether 12 detail hardware
 
   
Mon Nov 17 23:41:10.775 PST DST
-------------------------------------------------------------------------------
0/5/CPU0
-------------------------------------------------------------------------------
Interface                   Address                 Version  Refcount  Protocol
BE12                        10.221.12.11                 96         2  ipv4   
                           38303133323500
                           flags 0 0 50001000
                           Adj: 0xa836cfa8              Platform adj: 0xa836ce14 
                           MBOX-Hdr: LB Type             
 
   
                           BLC: FALSE   Number of members: 2 Owner-Id 0xfffffffd
                           Number of remote BLC: 1 Table-Id:0xe0000000 
                           Member Id: 0 Local: FALSE  Weight: 1 Slot: 3 
                           Member Id: 1 Local: FALSE  Weight: 1 Slot: 3 
                           Slot: 3  Label:0x03440409 OI:0x00300409 (6 / 1033)
 
   
                           (externally managed adj)
                           Client id : 0x96e
                           Client dll : libbundlemgr_platform_c12000_cm
                           Client completion function : bmp_adj_cmpl_adj
-------------------------------------------------------------------------------
0/4/CPU0
-------------------------------------------------------------------------------
Interface                   Address                 Version  Refcount  Protocol
BE12                        10.221.12.11                 23         2  ipv4   
                           38303133323500
                           flags 0 0 50000000
                           Adj: 0x50e88308              Platform adj: 0x50e88174 
                           MBOX-Hdr: LB Type             
 
   
                           BLC: FALSE   Number of members: 2 Owner-Id 0xfffffffd
                           Number of remote BLC: 1 Table-Id:0xe0000000 
                           Member Id: 0 Local: FALSE  Weight: 1 Slot: 3 
                           Member Id: 1 Local: FALSE  Weight: 1 Slot: 3 
                           Slot: 3  Label:0x03440409 OI:0x00300409 (6 / 1033)
 
   
                           (externally managed adj)
                           Client id : 0x96e
                           Client dll : libbundlemgr_platform_c12000_cm
                           Client completion function : bmp_adj_cmpl_adj
-------------------------------------------------------------------------------
0/3/CPU0
-------------------------------------------------------------------------------
Interface                   Address                 Version  Refcount  Protocol
BE12                        (src mac only)               53         3  ipv4   
                           0000000000000019aaa3b3fb0800
                           flags 0 0 50000001
                           Adj: 0x50e87318              Platform adj: 0x50e87244 
                           MBOX-Hdr: LB Type             
 
   
                           BLC: FALSE   Number of members: 0 Owner-Id 0x1
 
   
                           (externally managed adj)
                           Client id : 0x96e
                           Client dll : libbundlemgr_platform_c12000_cm
                           Client completion function : bmp_adj_cmpl_adj
BE12                        10.221.12.11                 54         2  ipv4   
                           001b0c63654a0019aaa3b3fb0800
                           flags 0 0 50001000
                           Adj: 0x50e87110              Platform adj: 0x50e86f7c 
                           MBOX-Hdr: LB Type            Local HW-TX-ADJ: 0x00080080  
Ppindex: 1033 
 
   
                           BLC: TRUE    Number of members: 2 Owner-Id 0x1
                           Member Id: 0 Local: TRUE   Weight: 1 Slot: 3 
                           Member Id: 1 Local: TRUE   Weight: 1 Slot: 3 
 
   
                           (externally managed adj)
                           Client id : 0x96e
                           Client dll : libbundlemgr_platform_c12000_cm
                           Client completion function : bmp_adj_cmpl_adj
...
 
   

This verifies that each linecard has the necessary adjacency information for bundle 12.

Verifying Bundle Load Balancing

Cisco IOS XR software has a bundle utility that predicts how load balancing will distribute flows among the bundle links. This is an interactive utility that prompts for the information that the load balancing algorithm uses to assign flows to links. For more information about the command usage, see Cisco IOS XR Interface and Hardware Component Command Reference.

This example shows how to use the bundle hash utility to predict the link that a specific Layer 3 flow would take, along with the links that other subnets would use:

RP/0/5/CPU0:RouterA# bundle-hash bundle-ether 12
 
   
Tue Nov 18 01:12:13.106 PST DST
Single SA/DA pair or range: S/R [S]: S
Enter source IP V4 address: 10.22.23.22
Enter destination IP V4 address: 10.11.12.11
Compute destination address set for all members? [y/n]: y
Enter subnet prefix for destination address set: 10.0.0.0
Enter bundle IP V4 address [10.22.23.22]: 10.221.12.22
 
   
Link hashed [hash:2] to is GigabitEthernet0/3/0/3 member id 0 ifh 0x4000600
 
   
Destination address set for subnet 10.0.0.0:
  10.0.0.2 [hash:4] hashes to link GigabitEthernet0/3/0/3
  10.0.0.1 [hash:7] hashes to link GigabitEthernet0/3/0/4
 
   

This shows that an ingress flow from source address 10.22.23.22 to destination address 10.11.12.11 would use egress interface GigabitEthernet0/3/0/3. You can verify that this is the case by generating this traffic flow, using extended ping for example, and by viewing the interface traffic for each of the bundle links:

RP/0/5/CPU0:RouterA# show interfaces Gi0/3/0/3
...
     1008 packets output, 118844 bytes, 0 total output drops
     Output 0 broadcast packets, 8 multicast packets
     0 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
 
   
RP/0/5/CPU0:RouterA# show interfaces Gi0/3/0/4
...
     2 packets output, 256 bytes, 0 total output drops
     Output 0 broadcast packets, 2 multicast packets
     0 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
 
   

This verifies that the flow is using GigabitEthernet0/3/0/3, as expected. The other link in the bundle has incidental traffic.

Repeat this procedure for other flows. For example, the bundle-hash utility provides the following information for a flow with a source address of 10.22.24.22 and a destination address of 10.11.12.11:

RP/0/5/CPU0:RouterA# bundle-hash bundle-ether 12
Tue Nov 18 01:18:56.879 PST DST
Single SA/DA pair or range: S/R [S]: S
Enter source IP V4 address: 10.22.24.22
Enter destination IP V4 address: 10.11.12.11
Compute destination address set for all members? [y/n]: y
Enter subnet prefix for destination address set: 10.0.0.0
Enter bundle IP V4 address [10.22.24.22]: 10.221.12.22
 
   
Link hashed [hash:5] to is GigabitEthernet0/3/0/4 member id 1 ifh 0x4000700
 
   
Destination address set for subnet 10.0.0.0:
  10.0.0.1 [hash:0] hashes to link GigabitEthernet0/3/0/3
  10.0.0.2 [hash:3] hashes to link GigabitEthernet0/3/0/4
 
   

This verifies that a flow from source address 10.22.24.22 to destination address 10.11.12.11:23 would use egress interface GigabitEthernet0/3/0/4. You can verify that this occurs by viewing the interface traffic for each of the bundle links, as shown in the following excerpts from the show interfaces command.

RP/0/5/CPU0:RouterA# show interfaces Gi0/3/0/3  
...
     6 packets output, 618 bytes, 0 total output drops
     Output 0 broadcast packets, 6 multicast packets
     0 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
 
   
RP/0/5/CPU0:RouterA# show interfaces Gi0/3/0/4  
...
     1002 packets output, 118256 bytes, 0 total output drops
     Output 0 broadcast packets, 2 multicast packets
     0 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
 
   

This verifies that the new flow is using GigabitEthernet0/3/0/4, as expected.

Additional References

The following sections provide references related to load balancing.

Related Documents

Related Topic
Document Title

Implementing Cisco Express Forwarding on Cisco IOS XR Software

Cisco IOS XR IP Addresses and Services Configuration Guide for the Cisco XR 12000 Series Router

Cisco Express Forwarding Commands on Cisco IOS XR Software

Cisco IOS XR IP Addresses and Services Command Reference for the Cisco XR 12000 Series Router

Configuring Link Bundling on Cisco IOS XR Software

Cisco IOS XR Interface and Hardware Component Configuration Guide for the Cisco XR 12000 Series Router

Link Bundling Commands on Cisco IOS XR Software

Cisco IOS XR Interface and Hardware Component Command Reference for the Cisco XR 12000 Series Router