Understanding and Resolving TCAM Resource Exhaustion on Cisco Nexus 3064Q

Introduction

This document describes how syslog messages indicating TCAM resource exhaustion were observed on a Cisco Nexus 3064PQ switch running NX-OS version 9.3(9).

Background Information

The exhaustion occurs in the LPM_128 TCAM region, responsible for storing IPv6 routes with prefix lengths greater than /64.

Observed Syslog Messages

%IPFIB-2-FIB_TCAM_RESOURCE_EXHAUSTION_LPM_128_IPV6: FIB TCAM exhausted for IPV6 routes in LPM-128 table
%IPFIB-2-FIB_TCAM_RESOURCE_EXCEP_CLRD: FIB TCAM resource usage exception cleared
%IPFIB-2-FIB_TCAM_RESOURCE_EXHAUSTION_LPM_128_IPV6: FIB TCAM exhausted for IPV6 routes in LPM-128 table

Additional Consistency Check Messages

%UFDM-3-FIB_IPv6_ROUTE_CONSISTENCY_CHECKER_FAIL: FIB IPv6 route consistency checker FAILED on slot 1

These logs confirm recurring TCAM allocation failures for IPv6 routes within the LPM_128 table.

Forwarding Table Architecture - Cisco Nexus 3000 Series

System Output - IPv6 Route Summary

Command

show system internal forwarding ipv6 route summary

Output Summary

• Max LPM table entries: 15359
• Total (<= /64) LPM routes: 266
• Total (> /64) LPM routes: 157

Based on this output, 157 routes are utilizing the LPM_128 table, which exceeds the platform capacity.

LPM_128 Table Utilization

The LPM_128 TCAM region stores IPv6 prefixes from /65 to /127.

On Nexus 3064PQ, the LPM_128 capacity is:

• With uRPF enabled: 64 entries
• With uRPF disabled: 128 entries

Since uRPF is disabled in the current configuration, the switch can accommodate up to 128 IPv6 LPM_128 entries.

The current utilization of 157 entries exceeds this threshold, resulting in TCAM exhaustion.

Verification - uRPF Configuration

Command

show running-config | i urpf

Output

system urpf disable

urpf disable

This confirms uRPF is disabled, allowing extended LPM_128 capacity.

Resource Utilization Summary

• Used IPv6 LPM_128 Entries: 157
• Max IPv6 LPM_128 Capacity: 128

Once the number of entries exceeds the maximum available TCAM capacity, the system triggers this syslog alert:

%IPFIB-2-FIB_TCAM_RESOURCE_EXHAUSTION_LPM_128_IPV6: FIB TCAM exhausted for IPV6 routes in LPM-128 table

Root Cause Analysis

This is a confirmed case of TCAM resource exhaustion on the Cisco Nexus 3064PQ platform, caused by:

• Excessive IPv6 routes with prefix lengths greater than /64, exceeding the available LPM_128 TCAM region capacity.
• Hardware limitations inherent to the Nexus 3064PQ architecture.

Platform-Specific Capacity Details

Note: These limits are specific to Nexus 3064PQ and are lower than most other Nexus 3000 series switches, where the LPM_128 table typically supports 128 (uRPF enabled) and 256 (uRPF disabled) entries.

Recommended Remediation

1. Route Summarization
• Implement IPv6 route summarization for prefixes between /65 and /127 in order to reduce entries in the LPM_128 TCAM region.
2. Review Routing Policies
   
• Optimize route advertisements from upstream peers to limit long-prefix IPv6 routes.
• Verify that unnecessary static or redistributed routes are not contributing to table exhaustion.
3. Hardware Upgrade
   
• If summarization is not feasible, consider migrating to a higher-capacity Nexus platform (for example, Nexus 9000 series).

Summary

Conclusion

The Nexus 3064PQ switch experienced TCAM exhaustion due to IPv6 routes exceeding the LPM_128 TCAM capacity.

Route summarization and optimization are required to prevent further occurrences.