This chapter describes how to configure the equal-cost multipathing (ECMP) protocol for host routes on the Cisco NX-OS switch.
This chapter includes the following sections:
Information About ECMP for Host Routes
When you enable ECMP support for host routes, all unicast host routes are programmed into the longest-prefix match algorithm
(LPM) table. ECMP for host routes is provided in the switch hardware. You configure this feature in the CLI using the hardware profile unicast enable-host-ecmp command.
Note
Host entries are stored in the LPM routing table instead of the host table when ECMP is configured for IPv4 (/32) routes and
IPv6 (/128) routes.
Guidelines for ECMP for Host Routes
ECMP for host routes has the following configuration guidelines and limitations:
After enabling or disabling ECMP for host routes by using the [no] hardware profile unicast enable-host-ecmp command, ensure
that you do the following:
– Save the current configuration on the switch by using the copy running-config startup-config command.
– Reload the switch by using the reload command so that the configuration can be applied.
Licensing Requirements for ECMP for Host Routes
The following table shows the licensing requirements for this feature:
Product
License Requirement
Cisco NX-OS
ECMP for host routes requires no license. Any feature not included in a license package is bundled with the Cisco NX-OS system
images and is provided at no extra charge to you. For a complete explanation of the Cisco NX-OS licensing scheme, see the
Cisco NX-OS Licensing Guide.
Prerequisites for ECMP for Host Routes
ECMP for host routes has the following prerequisites:
Before you use this command, we recommend that you disable Unicast Reverse Path Forwarding (URPF) globally on the switch using
the system urpf disable command, and then save the configuration and reload the switch. Disabling URPF globally extends the LPM table size.
switch(config)# reload
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
Reloads the Cisco Nexus 3000 Series switch software.
Configuring Weighted ECMP over BGP
ECMP is a mechanism that allows multiple routes to the same destination with different next-hops and it load-balances the
routed traffic over those multiple next-hops.The basic ECMP works for most of the customers’ requirements. The load entropy
is the best way to maximize the link usage efficiency.
Often, the application distribution in the network can be unbalanced. The new clusters roll in at different over-subscription
rates than the old clusters. The new clusters have powerful servers than the old clusters and they are capable of handling
more load per CPU. As the network is not perfect, some control over routing behavior is needed. You can configure Weighted
ECMP over BGP for balancing the load traffic and for administering control over the routing behavior.
The following use-cases are considered for this deployment:
Unequal distribution of resources
SDN or non-Homogeneous Fabric
Configuring Weighted ECMP over BGP
ECMP is a mechanism that allows multiple routes to the same destination with different next-hops and it load-balances the
routed traffic over those multiple next-hops.The basic ECMP works for most of the customers’ requirements. The load entropy
is the best way to maximize the link usage efficiency.
Often, the application distribution in the network can be unbalanced. The new clusters roll in at different over-subscription
rates than the old clusters. The new clusters have powerful servers than the old clusters and they are capable of handling
more load per CPU. As the network is not perfect, some control over routing behavior is needed. You can configure Weighted
ECMP over BGP for balancing the load traffic and for administering control over the routing behavior.
The following use-cases are considered for this deployment:
Unequal distribution of resources
SDN or non-Homogeneous Fabric
SDN/Non-Homogeneous Fabric
For SDN/non-homogeneous fabric, consider the following example:
You have two instances of VIP1 (an any service).
The server NIC IP address is advertised as the next hop (NH) for the VIP.
The VIP1 is deployed evenly across the data center.
Cluster 1 is oversubscribed 3:1 and Cluster 2 is oversubscribed 12:1.
As a result, the traditional ECMP delivers the traffic sub-optimally.
Solutions for the Use-Cases
The solution for the unequal distribution of the resources and sub-optimal traffic distribution use-cases is to configure
Weighted ECMP over BGP. You can inject the VIP routes (from the host or the controller) and signal a weight for each instance.
You can then aggregate the weights across the infrastructure and deliver the traffic in the direct proportion to the application
deployment distribution.
Guidelines and Limitations for Configuring Weighted ECMP over BGP
See the following guidelines for configuring Weighted ECMP over BGP:
Cisco Nexus 3100 Series devices support weighted ECMP only in non-resilient mode.
BGP uses the Link Bandwidth EXTCOMM defined in the draft-ietf-idr-link-bandwidth-06.txt to implement the weighted ECMP feature.
The weighted ECMP feature is supported for IPv4 address-family only.
You can accept Link Bandwidth EXTCOMM from both iBGP and eBGP peers.
Do not send the Link Bandwidth EXTCOMM in the BGP updates. The BGP controller/peer sends the link bandwidth to all the routers.
For weights programming, the link bandwidth EXTCOMM has the link bandwidth encoded in bytes/second, as a four byte floating
point integer, that is converted to kbits/second unsigned integer before downloading to RIB.
The hardware ECMP width is fixed as 64 in size.
Displaying Link Bandwidth EXTCOMM fields
See the following example for the displaying the link bandwidth EXTCOMM fields:
Link Bandwidth EXTCOMM fields:
Link Bandwidth attribute – “40 04 00 64 47 80 00 00”
Where “40 04” specifies a Link Bandwidth EXTCOMM
Where “00 64” specifies a AS #.
Where “47 80 00 00” specifies the Link Bandwidth value as floating point integer.
The Link Bandwidth floating point value bits are encoded as follows:
#define IEEE_NUMBER_WIDTH 32 /* bits in number */
#define IEEE_EXP_WIDTH 8 /* bits in exponent */
#define IEEE_MANTISSA_WIDTH (IEEE_NUMBER_WIDTH - 1 - IEEE_EXP_WIDTH)
#define IEEE_SIGN_MASK 0x80000000
#define IEEE_EXPONENT_MASK 0x7F800000
#define IEEE_MANTISSA_MASK 0x007FFFFF
Link Bandwidth value programmed to RIB is calculated as follows:
uint32_t ieee_bw_32 = ntohl(GETLONG(&extcomm->value[4])); = 0x47800000
int64_t dmzlink_bw_64 = ptr_ieee_to_int64(&ieee_bw_32); = 65536
uint32_t value = 520 = (uint32_t)((dmzlink_bw_64 / 1000) * 8) = 520
Configuring Dynamic ECMP Group Resizing
Configuring the dynamic ECMP group resizing feature allows you to configure more number of ECMP groups on Cisco Nexus 3000
and 3100 Series switches. You can configure upto 1022 ECMP Groups. The ECMP groups sizes are not fixed in the hardware.
Note
In Cisco Nexus 3000 Series switches, the dynamic ECMP group resizing feature is enabled by default. In Cisco Nexus 3100 Series
switches, the dynamic ECMP group resizing feature is available in non-resilient mode.
The following new CLI command is introduced to configure more number of ECMP groups in Cisco Nexus 3100 Series switches:
(config)# no hardware profile ecmp resilient
Warning: The command will take effect after next reload.
(config)#
Verifying Dynamic ECMP Group Resizing
Use the following command to verify the configuration of the ECMP groups:
# show hardware profile status
slot 1
=======
Total LPM Entries = 7679.
Total Host Entries = 16384.
Reserved LPM Entries = 1024.
Max Host4/Host6 Limit Entries (shared)= 8192/4096*
Max Mcast Limit Entries = 4096.
Max Ucast IPv6 LPM Limit Entries = 2048.
Max Ucast IPv6 LPM_65_to_127 Limit Entries = 128.
Used LPM Entries (Total) = 4.
Used IPv4 LPM Entries = 1.
Used IPv6 LPM Entries = 2.
Used IPv6 LPM_65_to_127 Entries = 1.
Used Host Entries in LPM (Total) = 0.
Used Host4 Entries in LPM = 0.
Used Host6 Entries in LPM = 0.
Used Mcast Entries = 0.
Used Mcast OIFL Entries = 0.
Used Host Entries in Host (Total) = 0.
Used Host4 Entries in Host = 0.
Used Host6 Entries in Host = 0.
Max ECMP Table Entries = 1022.
Used ECMP Table Entries = 0.
Max ECMP Next Hop Table Entries = 16384.
Used ECMP Next Hop Table Entries = 0.
MFIB prefer-source-tree = Disabled/0/0.
*Unicast Host Table is in shared mode b/n v4 & v6...
#
Verifying the ECMP for Host Routes Configuration
To display the ECMP for host routes configuration information, perform one of the following tasks:
Command
Purpose
show hardware profile status
Displays the unicast and multicast routing entries in hardware tables.
show running-config
Displays the running system configuration.
Configuration Examples for ECMP for Host Routes
This example shows how to disable URPF and configure ECMP for host routes:
Feedback