Understand Object Manager on Catalyst 9000 Switches

Introduction

This document describes the Object Manager component and its functionality on Catalyst 9000 Series Switches. 

Prerequisites

There are no specific prerequisites for this document.

Requirements

There are no specific requirements for this document.

Components Used

This information in this document is based on these hardware versions:

• Catalyst 9200
• Catalyst 9300
• Catalyst 9400
• Catalyst 9500
• Catalyst 9600
• Cisco IOS XE 16 and later

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Object Manager is a software process that runs within Cisco IOS XE. It is responsible for sending state creation messages from the active route complex to all forwarding complexes on the system. These state creation messages contain an object to be programmed, such as a MAC address, an ARP entry, or a network prefix.

There are two types of object managers to consider:

• Object Manager (OM) - runs between FMAN RP and FMAN FP. This channel is mostly an ACK'less model for performance reasons, though some exceptions exist for synchronization purposes.
• Asynchronous Object Manager (AOM) - AOM has 2 async interfaces. One is northbound to FMAN RP and the other is southbound to FED. It receives messages from FMAN RP and sends them down to FED for creation in hardware. This model does contain an ACK system in order for hardware to signal to the software that the state creation was successful.

Terminology

Troubleshoot

Symptoms of Congestion

State creation from IOSd to FMAN-RP is not a common point of congestion. However, during periods of instability or high rates of churn in the network, communication between FMAN-RP and FMAN-FP can become impacted. If FMAN-FP is unable to successfully program an object in hardware we expect to see a syslog message from FMAN-FP.

%FMFP-3-OBJ_DWNLD_TO_DP_FAILED: F0/0: fman_fp_image: PREFIX 10.10.10.0/26 (Table id 4) download to DP failed

%FMFP - Feature Manager FP

OBJ_DWNLOD_TO_DP_FAILED - Object Download to Dataplane Failed

PREFIX 10.10.10.0/26 - The prefix not successfully downloaded into FED

(Table id 4) - The VRF ID represented as a number

This FMAN-FP syslog indicated that FED was not programmed successfully. If for some reason FED is having scale issues there is a message produced:

%FED_L3_ERRMSG-3-RSRC_ERR: F0/0: fed: Failed to allocate hardware resource for ecr entry due to hardware resource exhaustion - rc:3

If this log is seen, verify if the device is within scale limits and confirm there is no unicast/multicast or other protocol churn. Refer to Understand IPv4 Hardware Resources on Catalyst 9000 for further information.

Note: If these logs from FMFP or FED are observed expect to see forwarding problems to the object type in question (such as lack of forwarding to a failed prefix).

Validate Statistics

Confirm current object manager counters to see if at a high level there is any problem with object communication

Switch#show platform software object-manager switch active f0 statistics
Forwarding Manager Asynchronous Object Manager Statistics

Object update: Pending-issue: 0, Pending-acknowledgement: 0   <--- No current pending objects 
Batch begin:   Pending-issue: 0, Pending-acknowledgement: 0
Batch end:     Pending-issue: 0, Pending-acknowledgement: 0
Command:       Pending-acknowledgement: 0
Total-objects: 1924
Stale-objects: 0
Resolve-objects: 0
Childless-delete-objects: 0
Backplane-objects: 0
Error-objects: 0        <--- No current error objects
Number of bundles: 156
Paused-types: 0

Note: If your device is operating in standalone mode and is not part of a stackwise stack or stackwise-virtual setup then omit the "switch active" part of the command

Use this command to see types of objects and which ones are waiting. This can help identify what feature may be unstable, excessively updating, or having an issue.

Switch#show platform software object-manager switch active f0 object-type-count 
Type  Name                        Paused Actions           Count       Waiting  
------------------------------------------------------------------------------
0     unknown                                              0           0        
1     acl                                                  6           0        
7     acl-group-cg                                         0           0        
8     acl-group-bind-lock                                  0           0        
9     unknown                                              0           0        
10    acl-reflect-group                                    0           0        
11    sgacl global config                                  0           0              
39    punt-interface                                       1           0        
40    adj                                                  7           0        
41    adj-recv                                             1           0        
42    adj-punt                                             1           0        
52    illegal                                              1           0        
53    mcast                                                1           0        
54    dpidb                                                0           0        
55    eos-choice                                           0           0        
56    frr                                                  0           0        
57    frr primary path                                     0           0        
58    label                                                0           0        
59    label-entry                                          0           0        
60    loadbalance                                          0           0        
61    loadbalance-map                                      0           0        
<...snip...>

If error objects or pending objects are non-zero, view the objects to confirm what they are

Switch#show platform software object-manager switch active f0 pending-issue-update       
Update identifier: 80177, Object identifier: 14934           <--- object ID
  Description: adj 0x96, Flags Incomplete                    <--- object type (adjacency in this example)
  Action: Delete, Pending seconds: 33                        <--- action that failed
  Number of retries: 0, Number of batch begin retries: 0, Number of nacked download retries: 0

Related Information

• Technical Support & Documentation - Cisco Systems
• Cisco Catalyst 9200 Series Switches Data Sheet
• Cisco Catalyst 9300 Series Switches Data Sheet
• Cisco Catalyst 9400 Series Switches Data Sheets
• Cisco Catalyst 9500 Series Switches Data Sheets
• Cisco Catalyst 9600 Series Switches Data Sheet
• Cisco Catalyst 9500 Architecture White Paper
• Understand IPv4 Hardware Resources on Catalyst 9000
• Understand Hardware Resources on Catalyst 9000 Switches