The unicast RIB (IPv4 RIB and IPv6 RIB) and FIB are part of the Cisco NX-OS forwarding architecture, as shown in Figure 16-1.
Figure 16-1 Cisco NX-OS Forwarding Architecture
The unicast RIB exists on the active supervisor. It maintains the routing table with directly connected routes, static routes, and routes learned from dynamic unicast routing protocols. The unicast RIB also collects adjacency information from sources such as the Address Resolution Protocol (ARP). The unicast RIB determines the best next hop for a given route and populates the unicast forwarding information bases (FIBs) on the modules by using the services of the unicast FIB distribution module (FDM).
Each dynamic routing protocol must update the unicast RIB for any route that has timed out. The unicast RIB then deletes that route and recalculates the best next hop for that route (if an alternate path is available).
In rare instances, an inconsistency can occur between the unicast RIB and the FIB on each module. In Cisco NX-OS Release 4.0(3) and later releases, Cisco NX-OS supports the Layer 3 consistency checker. This feature detects inconsistencies between the unicast IPv4 RIB on the supervisor module and the FIB on each interface module. Inconsistencies include the following:
Wrong next-hop address
Incorrect Layer 2 rewrite string in the ARP or neighbor discovery (ND) cache
The Layer 3 consistency checker compares the FIB entries to the latest adjacency information from the Adjacency Manager (AM) and logs any inconsistencies. The consistency checker then compares the unicast RIB prefixes to the module FIB and logs any inconsistencies. See the “Triggering the Layer 3 Consistency Checker” section.
Dynamic TCAM allocation reallocates unused TCAM blocks on M1 Series non-XL modules to an adjacent region when all existing blocks in that region are full. Dynamic TCAM allocation allows more flexibility in the number of routes that the FIB can allocate for a route type.
Cisco NX-OS divides the FIB to support multiple address families. The FIB TCAM for M1 Series non-XL modules has 128,000 physical entries.
The FIB TCAM entries are system wide resources that are shared across virtual device contexts (VDC) configured on the module.
Table 16-2 describes the supported maximum FIB scale entries on the Nexus 7000 system configuration per route-type.
Table 16-2 Maximum Supported TCAM Entries and FIB Scale Limits
1.Utilization may vary based on the sequence of routes being added and on the mix of unicast and multicast routes.
NoteTable 16-2 captures the scale limits in a VDC. In a Cisco Nexus 7000 system, the total memory on the supervisor module restricts the actual route scale limits across all VDCs in the system.
Note Do not exceed the maximum route limits for non-XL modules in a VDC that contains both XL modules and non-XL modules.
Note The actual FIB TCAM can scale to a higher scale number from a hardware perspective. Table 16-2 captures the currently supported FIB sizes.
Note The maximum routes are individual route-type maximum values and these values are not cumulative across each route-type.
You must install the Scalable Services License (see the
Cisco NX-OS Licensing Guide
) and configure the higher shared memory sizes (see the
Cisco Nexus 7000 Series NX-OS Virtual Device Context Configuration Guide
) for the routing table to enable the higher FIB scale on the XL modules. See the
Cisco Nexus 7000 Series Hardware Installation and Reference Guide
for more information on the XL modules.
When you install the Scalable Services license, you may see the following system message:
"2011 Mar 30 12:38:13 switch %PLTFM_CONFIG-4-XL_LICENSE_MIX_NOTIFY: Mixed use of non-XL with XL modules in the same VDC may limit common resources to non-XL capacity."
This message occurs if you install the Scalable Services license in a system with non-XL modules or when non-XL modules come on line after you install this license.
Note The full IPv4 Internet route tables currently have more than 300,000 routes and require the XL modules.
The unicast RIB and FIB support virtual routing and forwarding (VRF) instances. VRF exists within VDCs. By default, Cisco NX-OS places you in the default VDC and default VRF unless you specifically configure another VDC and VRF. For more information, see the
Cisco Nexus 7000 Series NX-OS Virtual Device Context Configuration Guide
and Chapter15, “Configuring Layer 3 Virtualization”
Licensing Requirements for the Unicast RIB and FIB
The following table shows the licensing requirements for this feature:
The unicast RIB and FIB require no license. Any feature not included in a license package is bundled with the Cisco NX-OS system images and is provided at no extra charge to you. For a complete explanation of the Cisco NX-OS licensing scheme, see the
Cisco NX-OS Licensing Guide
Guidelines and Limitations
Unicast RIB and FIB have the following configuration guidelines and limitations:
You must install the Scalable Services license and configure the higher shared memory sizes to enable the higher FIB sizes for XL modules.
Table 16-3 lists the default settings for unicast RIB and FIB parameters.
Note If you are familiar with the Cisco IOS CLI, be aware that the Cisco NX-OS commands for this feature might differ from the Cisco IOS commands that you would use.
Displaying Module FIB Information
You can display the FIB information on a module.
To display the FIB information on a module, use the following commands in any mode:
switch# show forwarding ipv6 adjacency module 2
Displays the adjacency information for IPv4 or IPv6.
switch# show forwarding ipv6 route module 2
Displays the route table for IPv4 or IPv6.
Configuring Load Sharing in the Unicast FIB
Dynamic routing protocols such as Open Shortest Path First (OSPF) support load balancing with equal-cost multipath (ECMP). The routing protocol determines its best routes based on the metrics configured for the protocol and installs up to the protocol-configured maximum paths in the unicast RIB. The unicast RIB compares the administrative distances of all routing protocol paths in the RIB and selects a best path set from all of the path sets installed by the routing protocols. The unicast RIB installs this best path set into the FIB for use by the forwarding plane.
The forwarding plane uses a load-sharing algorithm to select one of the installed paths in the FIB to use for a given data packet.
You can globally configure the following load-sharing settings:
Load-share mode—Selects the best path based on the destination address and port or the source and the destination address and port.
Universal ID—Sets the random seed for the hash algorithm. You do not need to configure the Universal ID. Cisco NX-OS chooses the Universal ID if you do not configure it.
Note Load sharing uses the same path for all packets in a given flow. A flow is defined by the load-sharing method that you configure. For example, if you configure source-destination load sharing, then all packets with the same source IP address and destination IP address pair follow the same path.
To configure the unicast FIB load-sharing algorithm, use the following command in global configuration mode:
ip load-sharing address
destination port destination
switch(config)# ip load-sharing address source-destination
Configures the unicast FIB load-sharing algorithm for data traffic. The
range is from 1 to 4294967295.
To display the unicast FIB load-sharing algorithm, use the following command in any mode:
show ip load-sharing
switch(config)# show ip load-sharing address source-destination
Displays the unicast FIB load-sharing algorithm for data traffic.
To display the route that the unicast RIB and FIB use for a particular source address and destination address, use the following command in any mode:
Displays the route that the unicast RIB FIB use for a source and destination address pair. The source address and destination address format is x.x.x.x. The source port and destination port range is from 1 to 65535. The VRF name can be any case-sensitive, alphanumeric string up to 64 characters.
This example shows the route selected for a source/destination pair:
switch# show routing hash 10.0.0.5 126.96.36.199
Load-share parameters used for software forwarding:
load-share mode: address source-destination port source-destination
Universal-id seed: 0xe05e2e85
Hash for VRF "default"
Hashing to path *188.8.131.52 (hash: 0x0e), for route:
Configuring Per-Packet Load Sharing
You can use per-packet load sharing to evenly distribute data traffic in an IP network over multiple equal-cost connections. Per-packet load sharing allows the router to send successive data packets over paths on a packet-by-packet basis rather than on a per-flow basis.
Note Using per-packet load sharing can result in out-of-order packets. Packets for a given pair of source-destination hosts might take different paths and arrive at the destination out of order. Make sure you understand the implications of out-of-order packets to your network and applications. Per-packet load sharing is not appropriate for all networks. Per-flow load sharing ensures packets always arrive in the order that they were sent.
Per-packet load sharing uses the round-robin method to determine which path each packet takes to the destination. With per-packet load sharing enabled on interfaces, the router sends one packet for destination1 over the first path, the second packet for (the same) destination1 over the second path, and so on. Per-packet load sharing ensures balancing over multiple links.
Use per-packet load sharing to ensure that a path for a single source-destination pair does not get overloaded. If most of the traffic passing through parallel links is for a single pair, per-destination load sharing will overload a single link while other links will have very little traffic. Enabling per-packet load sharing allows you to use alternate paths to the same busy destination.
Note Per-packet load sharing on an interface overrides the global load-sharing configuration.
You configure per-packet load sharing on the input interface. This configuration determines the output interface that Cisco NX-OS chooses for the packet.
For example, if you have ECMP paths on two output interfaces, Cisco NX-OS uses the following load-sharing methods for input packets on Ethernet 1/1:
Per-packet load sharing if you configure per-packet load sharing on Ethernet 1/1.
Per-flow load sharing.
The configurations for the other interfaces have no effect on the load-sharing method used for Ethernet 1/1 in this example.
To configure per-packet load sharing, use the following command in interface configuration mode:
ip load-sharing per-packet
switch(config-if)# ip load-sharing per-packet
Configures per-packet load sharing on an interface.
Checking Routes in the Unicast FIB
You can configure a policy in the Embedded Event Manager (EEM) to check for inconsistent, missing, or failed routes in the unicast Forwarding Information Base (FIB).
BEFORE YOU BEGIN
Ensure that you have network-admin or vdc-admin user privileges to configure EEM.
Ensure that you are in the correct VDC (or use the
] | warning-only]
switch(config-vrf-af-ipv4)# maximum routes 250 90
Configures the maximum number of routes allowed in the routing table. The range is from 1 to 4294967295.
You can optionally specify the following:
—Percentage of maximum routes that triggers a warning message. The range is from 1 to 100.
—Logs a warning message when the maximum number of routes is exceeded.
—Reinstalls routes that previously exceeded the maximum route limit and were rejected and specifies the threshold value at which to reinstall them. The threshold range is from 1 to 100.