Cisco IOS XR Troubleshooting Guide for the Cisco XR 12000 Series Router
Troubleshooting Forwarding
Downloads: This chapterpdf (PDF - 607.0KB) The complete bookPDF (PDF - 2.34MB) | Feedback

Troubleshooting Forwarding

Table Of Contents

Troubleshooting Forwarding

Troubleshooting IPv4 CEF Information

Troubleshooting Adjacency Information

Examples

Troubleshooting Transient Traffic Drop in Forwarding

Troubleshooting Control Plane Information

Examples

Troubleshooting the Interface Manager

Interface Manager Control Process

Troubleshooting the Trace Logs for the Interface Manager

Troubleshooting the Client for the Interface Manager

Troubleshooting the Rules for the Interface Manager

Troubleshooting the Control Chain and Interface Information

Troubleshooting the Registrations for the Interface Manager

Troubleshooting the Interface Manager Distributor

Interface Manager Distributor Overview

Troubleshooting the Trace Logs for the Interface Manager Distributor

Troubleshooting the Clients for the Interface Manager Distributor

Troubleshooting the Interface Information for the Interface Manager Distributor

Troubleshooting the Global Registrations for the Interface Manager Distributor

Troubleshooting the Rules for the Interface Manager Distributor


Troubleshooting Forwarding


Cisco Express Forwarding (CEF) is the mechanism that enables packet forwarding. CEF information is examined when data forwarding is not occurring as expected. Troubleshooting CEF involves comparing the Routing Information Base (RIB) information to the software Forwarding Information Base (FIB), verifying that the hardware is programmed correctly, verifying that the adjacencies are built correctly, verifying the control plane is built correctly, and gathering any necessary trace information.

The only prerequisite for CEF is a valid route in the RIB.

This chapter describes techniques that you can use to troubleshoot router forwarding. It includes the following sections:

Troubleshooting IPv4 CEF Information

Troubleshooting Adjacency Information

Troubleshooting Transient Traffic Drop in Forwarding

Troubleshooting Control Plane Information

Troubleshooting the Interface Manager

Troubleshooting the Interface Manager Distributor

Troubleshooting IPv4 CEF Information

CEF is an advanced, Layer 3 IP switching technology that optimizes network performance. It also improves the scalability for networks with large and dynamic traffic patterns, such as the Internet and networks characterized by intensive Web-based applications.

Information conventionally stored in a route cache is stored in several data structures for CEF switching. The data structures provide optimized lookup for efficient packet forwarding. The two main components of CEF operation are forwarding information base (FIB) and adjacency tables:

CEF uses a FIB to make IP destination prefix-based switching decisions. FIB maintains a mirror image of the forwarding information contained in the IP routing table. When routing or topology changes occur in the network, the IP routing table is updated, and those changes are reflected in the FIB. The FIB maintains next hop address information based on the information in the IP routing table. There is a one-to-one correlation between FIB entries and routing table entries, therefore FIB contains all known routes and eliminates the need for route cache maintenance that is associated with switching paths such as fast switching and optimum switching.

Nodes in the network are said to be adjacent if they can reach each other with a single hop across a link layer. In addition to the FIB, CEF uses adjacency tables to prepend Layer 2 addressing information. The adjacency table maintains Layer 2 next-hop addresses for all FIB entries.

Figure 3-1 provides an overview of the components involved in contributing information to the CEF process and displays the interaction between the software and hardware elements of the CEF process.

Figure 3-1 CEF Process

To troubleshoot IPv4 CEF information on Cisco IOS XR software, perform the following procedure.

SUMMARY STEPS

1. show route ipv4 prefix mask

2. show cef ipv4 prefix mask detail

3. show cef ipv4 prefix mask detail location node-id (on ingress line card)

4. show cef ipv4 prefix mask detail location node-id (on egress line card)

5. show cef ipv4 prefix mask hardware ingress detail location node-id

6. show cef ipv4 prefix mask hardware egress detail location node-id

7. show cef ipv4 interface type instance location node-id

8. show cef ipv4 summary location node-id

9. show cef ipv4 trace location node-id

10. show cef platform trace ipv4 all location node-id

11. Contact Cisco Technical Support if the problem is not resolved

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

show route ipv4 prefix mask

Example:

RP/0/0/CPU0:router# show route 192.168.2.0 255.255.255.0

Displays the current routes in the Routing Information Base (RIB).

Check the prefix and mask, as well as the next hop and outgoing interface, to ensure that they are what is expected.

Note the timer value that shows how long the route has been in the routing table. If the timer value is low the route may be flapping.

A lower timer value is present when a route is installed in the RIB for a short period of time. A low timer value may indicate flapping. For example, if a BGP route was being installed and removed from the RIB table every sixty seconds, then the route is flapping.

Look for routes that have not been installed in the routing table for very long. The route will either be stable or flapping. If the route is flapping, contact contact Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section in the Preface.

Check that route is learned via the routing protocol you are expecting it to be known via, and that the metric is what you expect.

Step 2 

show cef ipv4 prefix mask detail

Example:

RP/0/0/CPU0:router# show route ipv4 192.168.2.0 255.255.255.0 detail

Displays the IPv4 Cisco Express Forwarding (CEF) table detailed entry information.

Compare the prefix, mask, next hop ip, and outgoing interface information with the information in the RIB. The information in the RIB is displayed using the show route ipv4 prefix mask command.

Check that the adjacency is valid or the expected type of adjacency. For example, if it is a remote adjacency, then the adjacency information exists on another node.

Check that the expected hash (load balance) and egress interfaces are listed.

Step 3 

show cef ipv4 prefix mask detail location node-id

Example:

RP/0/0/CPU0:router# show cef ipv4 192.168.2.0 255.255.255.0 detail location 0/14/cpu0

Displays the IPv4 CEF table for the designated ingress node.

Compare the prefix, mask, next hop ip, and outgoing interface information with the information in the RIB. The information in the RIB is displayed using the show route ipv4 prefix mask command.

Check that the adjacency is valid or the expected type of adjacency. For example, if it is a remote adjacency, then the adjacency information exists on another node.

Check that the expected hash (load balance) and egress interfaces are listed.

Step 4 

show cef ipv4 prefix mask detail location node-id

Example:

RP/0/0/CPU0:router# show cef ipv4 192.168.2.0 255.255.255.0 detail location 0/13/cpu0

Displays the IPv4 CEF table for the designated egress node.

Compare the prefix, mask, next hop ip, and outgoing interface information with the information in the RIB. The information in the RIB is displayed using the show route ipv4 prefix mask command.

Check that the adjacency is valid or the expected type of adjacency. For example, if it is a remote adjacency, then the adjacency information exists on another node.

Check that the expected hash (load balance) and egress interfaces are listed.

Step 5 

show cef ipv4 prefix mask hardware ingress detail location node-id

Example:

RP/0/0/CPU0:router# show cef ipv4 192.168.2.0 255.255.255.0 hardware ingress detail location 0/14/cpu0

Displays the IPv4 CEF table for the designated ingress node.

Check that the prefix and mask are valid.

Check the nexthop IP address is as expected

Check that the entry type is set to forward.

Check that the hardware and software representations in hex format match. For example:

SW: 0x0c000000 00000020 00000000 00000000
HW: 0x0c000000 00000020 00000000 00000000

Step 6 

show cef ipv4 prefix mask hardware egress detail location node-id

Example:

RP/0/0/CPU0:router# show cef ipv4 192.168.2.0 255.255.255.0 hardware detail egress location 0/13/cpu0

Displays the IPv4 CEF table for the designated egress node.

Check that the prefix and mask are valid.

Check the nexthop IP address is as expected

Check that the entry type is set to forward.

Check that the hardware and software representations in hex format match. For example:

SW: 0x0c000000 00000020 00000000 00000000
HW: 0x0c000000 00000020 00000000 00000000
 
        

Step 7 

show cef ipv4 interface type instance location node-id

Example:
RP/0/0/CPU0:router# show cef ipv4 interface 
tengige 1/3/0/7 location 1/3/cpu0

Displays IPv4 CEF-related information for an interface.

Verify the interface handle `interface is marked' is as expected. The command output also shows how many references there are to the interface in CEF table and the IPv4 MTU.

Use this command for the ingress and egress interfaces.

Step 8 

show cef ipv4 summary location node-id

Example:
RP/0/0/CPU0:router# show cef ipv4 summary 
location 0/3/cpu0

Displays a summary of the IPv4 CEF table. Check the VPN routing and forwarding (VRF) names associated with the node, the route update drops, and that there are the expected number of incomplete adjacencies.

Note the number of routes CEF has entries for, the number of load sharing elements, and the number of references to this node.

Use this command for the ingress and egress line cards and route processor (RP).

Step 9 

show cef ipv4 trace location node-id

Example:
RP/0/0/CPU0:router# show cef ipv4 trace 
location 0/3/cpu0

Displays IPv4 CEF trace table information.

Check if there is any flap on the prefix.

Use this command for the RP, and ingress and egress interfaces for the local line card.

Step 10 

show cef platform trace ipv4 all location node-id

Example:
RP/0/0/CPU0:router# show cef platform trace 
ipv4 all location 0/3/cpu0

Displays CEF IPv4 hardware status and configuration trace table information.

Verify that the TLU pointer and RPF pointer are as expected.

Use this command for the ingress and egress interfaces for the local line card.

Step 11 

Contact Cisco Technical Support.

If the problem is not resolved, contact Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section in the Preface.

Troubleshooting Adjacency Information

To troubleshoot adjacency information on Cisco IOS XR software, perform the following procedure.

SUMMARY STEPS

1. show arp location node-id

2. show arp traffic location node-id

3. show adjacency interface-type interface-instance remote detail location node-id

4. show adjacency interface-type interface-instance remote detail hardware location node-id

5. show adjacency ipv4 nexthop ipv4-address detail location node-id

6. show adjacency interface-type interface-instance detail location node-id

7. show adjacency ipv4 nexthop ipv4-address detail hardware location node-id

8. show adjacency interface-type interface-instance detail hardware location node-id

9. show adjacency trace location node-id

10. show adjacency trace client aib-client location node-id

11. show adjacency hardware trace location node-id

12. Contact Cisco Technical Support if the problem is not resolved

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

show arp location node-id

Example:
RP/0/0/CPU0:router# show arp location 0/12/cpu0

Displays the Address Resolution Protocol (ARP) for an egress line card with a broadcast interface.

Ensure that you can find the IP address and that correct MAC address of the neighbor is learned.

Step 2 

show arp traffic location node-id

Example:
RP/0/0/CPU0:router# show arp traffic location 
0/12/cpu0

Displays ARP traffic statistics for an egress line card with a broadcast interface.

Check for any errors or IP packet drops.

Step 3 

show adjacency interface-type interface-instance remote detail location node-id

Example:
RP/0/0/CPU0:router# show adjacency pos 0/13/0/2 
remote detail location 0/14/cpu0

Displays detailed CEF adjacency table information for a remote ingress line card.

Ensure that the output shows IPv4 adjacency information and that an adjacency exists.

Step 4 

show adjacency interface-type interface-instance remote detail hardware location node-id

Example:
RP/0/0/CPU0:router# show adjacency pos 0/13/0/2 
remote detail hardware location 0/14/cpu0

Displays adjacency information for a remote ingress line card.

Check that the prefix and mask are valid.

Check that the table look-up (TLU) pointers match the TLU pointers in the show cef ipv4 prefix mask hardware ingress detail location node-id command. For example:

Step 5 

show adjacency ipv4 nexthop ipv4-address detail location node-id

Example:
RP/0/0/CPU0:router# show adjacency ipv4 nexthop 
192.168.2.0 detail location 0/12/cpu0

Displays adjacencies on an egress line card with a broadcast interface that are destined to the specified IPv4 next hop.

When an egress interface is broadcast, use the show adjacency ipv4 nexthop command to display the adjacency information.

Compare the mac layer rewrite information that shows the destination L2 address in the first part followed by the source L2 address, and the Ethernet value with the output from the show arp location node-id command.

Step 6 

show adjacency interface-type interface-instance detail location node-id

Example:
RP/0/0/CPU0:router# show adjacency pos 0/13/0/2 
detail location 0/13/cpu0

Displays CEF adjacency table information for an egress line card with a point to point interface.

There should be two IPv4 entries in the command output. Ensure both entries exist.

The SRC MAC only entry is used for multicast switching

The point to point entry is used for unicast switching.

On broadcast interfaces you will have a SRC MAC only and one for each nexthop IP address. Please note the MTU is for the IPv4 minus the layer 2 header. Use the show im chains command to display MTU details.

Step 7 

show adjacency ipv4 nexthop ipv4-address detail hardware location node-id

Example:
RP/0/0/CPU0:router# show adjacency ipv4 nexthop 
192.168.2.0 detail hardware location 0/12/cpu0

Displays the hardware programming associated with the adjacency. Verify that the packets are being switched in the hardware.

Step 8 

show adjacency interface-type interface-instance detail hardware location node-id

Example:
RP/0/0/CPU0:router# show adjacency pos 0/13/0/2 
detail hardware location 0/13/cpu0

Displays the hardware programming information for a point-to-point interface such as the Packet-over-SONET/SDH (POS) interface. The rewrite information is slightly different because there is no MAC rewrite string as there is in Ethernet.

Verify that the rewrite is appropriate for the encapsulation on the interface. Compare the CEF hardware output and verify that the pointer matches the egress adjacency.

Step 9 

show adjacency trace location node-id

Example:
RP/0/0/CPU0:router# show adjacency trace 
location 0/13/cpu0

Displays CEF adjacency trace table information.

Use this command for the egress interfaces for the local line card.

Step 10 

show adjacency trace client aib-client location node-id

Example:
RP/0/0/CPU0:router# show adjacency trace client 
ipv4_fib_mgr location 0/13/cpu0

Displays CEF adjacency trace table information for a specified adjacency information base (AIB) client.

Use this command for the egress interfaces for the local line card.

Step 11 

show adjacency hardware trace location node-id

Example:
RP/0/0/CPU0:router# show adjacency hardware 
trace location 0/13/cpu0

Displays CEF adjacency hardware trace table information.

Use this command for the egress interfaces for the local line card.

Step 12 

Contact Cisco Technical Support.

If the problem is not resolved, contact Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section in the Preface.

Examples

The following example shows that the TLU pointers match. The TLU pointers are indicated in bold.

RP/0/0/CPU0:router# show cef 202.112.36.224/30 hardware ingress location 0/0/CPU0 
 
   
202.112.36.224/30, version 1536, internal 0x42040001[1] 
.
.
.
TLU2 0x0101041d
       TLU2 ENTRY        0
        SW: 0x00000000 00000000 00000000 00a22800
        HW: 0x00000000 00000000 00000000 00a22800
       label1:            0    label2:            0
       num of labels:     0    next ptr: 0x0000a228
 
   
RP/0/0/CPU0:router# show adjacency pos 0/4/0/15 remote detail hardware location 0/0/CPU0
.
.
.
ingress adjacency
  TLU3            : 0x200a228
  [HW: 0x00400000 0x00000000 0x00000000 0x00082800]
    num. entries  : 1
    num. labels   : 0
    label 1       : 0
    label 2       : 0
    next ptr      : 0x828
  TLU4            : 0x3000828
    Entry[0]
    [HW: 0x00000000 0x11410000 0x01480600 
 
   

The following example shows that the address information matches. The addresses are indicated in bold.

RP/0/0/CPU0:router# show arp location 0/1/cpu0
 
   
Address         Age        Hardware Addr   State      Type  Interface
212.27.50.157   02:08:34   0016.c761.f509  Dynamic    ARPA  TenGigE0/1/0/2
                                            
RP/0/0/CPU0:router# show adjacency ipv4 nexthop 212.27.50.157 detail loccation 0/1/cpu0
 
   
Interface                   Address                 Version  Refcount  Protocol
TenGigE0/1/0/2              212.27.50.157                41         2     ipv4
                           0016c761f5090015fa9959890800
                           mtu: 1500, flags 0 0 0
                           2894 packets, 156876 bytes
                           0xffffffff
 
   

Troubleshooting Transient Traffic Drop in Forwarding

To troubleshoot a momentary drop in IPV4 packet forwarding on Cisco XR12000 Series router, you must perform the following procedure.

SUMMARY STEPS

1. show interfaces interface name

2. show controller <ingressq/egressq> qmstats 0 location <line card>

3. show controller pse [ingress|egress] statistics location node-id

4. show route <prefix> det

5. show cef <prefix> det location location-id

6. show cef <prefix> hardware ingress detail location location-id

7. Contact Cisco Technical Support if the problem is not resolved

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

show interfaces interface name

Example:
RP/0/0/CPU0:router# show interface POS0/5/0/0

Displays the packet count in the ingress and egress interfaces before and after the transmission of packets.

If the packets are dropped either in ingress or egress interface, then stop here and contact the Cisco Technical Support team for troubleshooting the plim interfaces.

Step 2 

show controller <ingressq|egressq> qmstats 0 location location-id

Example:
RP/0/0/CPU0:router# show controller ingressq 
qmstats 0 location 0/12/cpu0

Checks the statistics in buffer management ASIC (BMA) at both ingress and egress line cards before and after transmission.

If you see any drop count incrementing other than soft drop, then contact the Cisco Technical Support team for resolution. If there is no increment in the drop count or only the soft drop is incrementing, continue with the further steps.

Step 3 

show controller pse <ingress|egress> statistics location location-id

Example:
RP/0/9/CPU0:router#show controller pse ingress 
statistics loc 0/5/cpu0 | inc drop
RP/0/9/CPU0:EE10-1#show controller pse ingress 
statistics loc 0/5/cpu0 | inc mtu
 
        

Displays the packets that are dropped at the packet switching engine (PSE), the drop count, and maximum transmission unit (mtu) count.

Ensure that the show command is executed multiple times in order to check if the count is actually incrementing. To check if the route has been removed and re-installed, you must identify the traffic stream or prefix which experienced the drop first.

Step 4 

show route <prefix> det

Example:
RP/0/0/CPU0:router# show route 20.20.1.2

Checks the routing table and the timestamp provides you information about when the route was installed.

Step 5 

show cef <prefix> det

show cef <prefix> det location location-id

Example:
RP/0/0/CPU0:router# show cef ipv4 nexthop 
192.168.2.0 detail location 0/12/cpu0

Checks if the route is installed in FIB or CEF is platform-independent. If the output of this show command indicates that the CEF is pointing to drop adjacency, then you can stop here and contact the Cisco Technical Support team.

Step 6 

show cef <prefix> hardware ingress detail location location-id

Checks if the Cisco Express Forwarding information is embedded in the hardware. If the MTU in the output is 5 or 6, then it indicates that the route has been programmed as drop in hardware.

Troubleshooting Control Plane Information

To troubleshoot control plane information on Cisco IOS XR software, perform the following procedure.

SUMMARY STEPS

1. show netio idb interface-type interface-instance location node-id

2. show uidb index

3. show uidb data interface-type interface-instance location node-id

4. show im chains interface-type interface-instance location node-id

5. show imds interface brief

6. show tbm hardware {ipv4 | ipv6} unicast dual detail location node-id

7. Contact Cisco Technical Support if the problem is not resolved

DETAILED STEPS

 
Command or Action
Purpose

Step 1 

show netio idb interface-type interface-instance location node-id

Example:
RP/0/0/CPU0:router# show netio idb 
tengige0/0/0/0 location 0/0/cpu0

Displays control plane information for the software switching path. The output provides useful statistics for determining software forwarding issues.

Verify the encap and decap paths

Ensure that all of the appropriate steps in the chain are shown for all of the features that may be enabled on the interface.

Note Fixup is a direct pointer to a routine in the output path after a CEF rewrite. this is an optimized path if a CEF rewrite exists and can be used.

Verify that the ifhandle and global uidb value is correct.

Use this command for the ingress and egress interfaces for the local line card.

Step 2 

show uidb index

Example:

RP/0/0/CPU0:router# show uidb index

Displays the micro-interface descriptor block (IDB) index assigned by the software.

Check that the interface and the universal interface descriptor block (UIDB) value are what is expected.

Compare the IDB index to the uidb index value in the show adjacency ipv4 interface-type interface-instance detail hardware location node-id command output.

Step 3 

show uidb data interface-type interface-instance location node-id

Example:
RP/0/0/CPU0:router# show uidb data tengige 
1/3/0/0 location 0/3/cpu0

Displays, from a software perspective, features that are enabled on a selected interface.

Check the UIDB value.

Check what flags are enabled for the UIDB.

Check the ifhandle in the UIDB to make sure it is correct.

Compare the output to the configuration of the interface and expected features.

Use this command for the ingress and egress interfaces for the local line card.

Step 4 

show im chains interface-type interface-instance location node-id

Example:
RP/0/0/CPU0:router# show im chains pos 0/14/0/0 
location 0/14/cpu0

or

RP/0/0/CPU0:router# show im chains pos 0/13/0/2 
location 0/13/cpu0

Displays the output of the control plane encapsulations for data plane forwarding.

Check the different layers for the interface, the status (up or down) of each layer, and the maximum transmission unit (MTU) at each layer.

Verify the ifhandle value the ingress line card will use to forward packets that are destined out of the interface.

Compare the output to the expected encapsulations on the interface, the correct MTU values, and the correct ifhandle value from the show cef ipv4 interface command output.

Use this command for the ingress interface on the ingress line card and the egress interface on egress line card.

Step 5 

show imds interface brief

Example:
RP/0/0/CPU0:router# show imds interface brief

Displays interface manager distribution server (IMDS) interface information.

Note This is just a partial output not full output.

Check the state, MTU, encapsulation being used, and the ifhandle for each interface.

Step 6 

show tbm hardware {ipv4 | ipv6} unicast dual detail location node-id

Example:
RP/0/0/CPU0:router# show tbm hardware ipv4 
unicast dual detail location 0/13/cpu0

or

RP/0/0/CPU0:router# show tbm hardware ipv4 
unicast dual detail location 0/14/cpu0

Displays tree bitmap hardware-related ingress and egress information.

Check if there have been any failures in the different stages of the lookup.

Use this command for the ingress and egress interfaces for the local line card.

Step 7 

Contact Cisco Technical Support.

If the problem is not resolved, contact Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section in the Preface.

Examples

The following example displays the control plane information for the software switching path. Check for any errors or drops.

RP/0/0/CPU0:router# show netio idb tenGigE 0/1/1/0 location 0/1/cpu0 
 
   
TenGigE0/1/1/0 (handle: 0x01180020, nodeid:0x11) netio idb:
---------------------------------
name:                    TenGigE0_1_1_0
interface handle:        0x01180020
interface global index:  2
physical media type:     30
dchain ptr:              <0x482ae8e0>
echain ptr:              <0x482d791c>
fchain ptr:              <0x482d79b8>
driver cookie:           <0x4824ad58>
driver func:             <0x4824ad44>
number of subinterfaces: 4096
subblock array size:     3
DSNCNF:                  0x00000000
interface stats info: 
   IN  unknown proto pkts:  0
   IN  unknown proto bytes: 0
   IN  multicast pkts:      0
   OUT multicast pkts:      0
   IN  broadcast pkts:      0
   OUT broadcast pkts:      0
   IN  drop pkts:           0
   OUT drop pkts:           0
   IN  errors pkts:         0
   OUT errors pkts:         0
 
   
Chains
--------------------
Base decap chain: 
     ether                <30>  <0xfd7aef88, 0x48302824>  <       0,        0>
 
   
Protocol chains:
---------------
<Protocol number> (name)  Stats
  Type  Chain_node       <caps num>  <function, context> <drop pkts, drop bytes>
<7> (arp)   Stats IN: 0 pkts, 0 bytes; OUT: 0 pkts, 0 bytes
    Encap: 
     l2_adj_rewrite       <86>  <0xfcec7a88, 0x4834efec>  <       0,        0>
     queue_fifo           <56>  <0xfcedda68, 0x482dbee4>  <       0,        0>
     txm_nopull           <60>  <0xfcea2a5c, 0x482dc11c>  <       0,        0>
    Decap: 
     queue_fifo           <56>  <0xfcedda4c, 0x482dbee4>  <       0,        0>
     arp                  <24>  <0xfd1082cc, 0x00000000>  <       0,        0>
    Fixup: 
     l2_adj_rewrite       <86>  <0xfcec745c, 0x00000000>  <       0,        0>
     queue_fifo           <56>  <0xfcedda68, 0x482dbee4>  <       0,        0>
     txm_nopull           <60>  <0xfcea2a5c, 0x482dc11c>  <       0,        0>
<12> (ipv4)   Stats IN: 0 pkts, 0 bytes; OUT: 0 pkts, 0 bytes
    Encap: 
     ipv4                 <26>  <0xfd10f41c, 0x482d7724>  <       0,        0>
     ether                <30>  <0xfd7aeb44, 0x48302824>  <       0,        0>
     l2_adj_rewrite       <86>  <0xfcec7a88, 0x4834f104>  <       0,        0>
     queue_fifo           <56>  <0xfcedda68, 0x482dbee4>  <       0,        0>
     txm_nopull           <60>  <0xfcea2a5c, 0x482dc11c>  <       0,        0>
    Decap: 
     queue_fifo           <56>  <0xfcedda4c, 0x482dbee4>  <       0,        0>
     ipv4                 <26>  <0xfd10f474, 0x00000000>  <       0,        0>
    Fixup: 
     l2_adj_rewrite       <86>  <0xfcec745c, 0x00000000>  <       0,        0>
     queue_fifo           <56>  <0xfcedda68, 0x482dbee4>  <       0,        0>
     txm_nopull           <60>  <0xfcea2a5c, 0x482dc11c>  <       0,        0>
<22> (ether_sock)   Stats IN: 0 pkts, 0 bytes; OUT: 0 pkts, 0 bytes
    Encap: 
     ether_sock           <98>  <0xfd7b1630, 0x48302824>  <       0,        0>
     l2_adj_rewrite       <86>  <0xfcec7a88, 0x48304c1c>  <       0,        0>
     queue_fifo           <56>  <0xfcedda68, 0x482dbee4>  <       0,        0>
     txm_nopull           <60>  <0xfcea2a5c, 0x482dc11c>  <       0,        0>
    Decap: 
     queue_fifo           <56>  <0xfcedda4c, 0x482dbee4>  <       0,        0>
     ether_sock           <98>  <0xfd7b1874, 0x48302824>  <       0,        0>
    Fixup: 
     l2_adj_rewrite       <86>  <0xfcec745c, 0x00000000>  <       0,        0>
     queue_fifo           <56>  <0xfcedda68, 0x482dbee4>  <       0,        0>
     txm_nopull           <60>  <0xfcea2a5c, 0x482dc11c>  <       0,        0>
 
   
 
   
Protocol SAFI counts:
--------------------
 
   
       Protocol        SAFI       Pkts In    Bytes In    Pkts Out   Bytes Out
---------------  ----------    ----------  ----------  ----------  ----------
           ipv4     Unicast             0           0           0           0
           ipv4   Multicast             0           0           0           0
           ipv4   Broadcast             0           0           0           0
           ipv6     Unicast             0           0           0           0
           ipv6   Multicast             0           0           0           0
 
   

The following example shows that the micro-idb index value is 12.

RP/0/0/CPU0:router# show uidb index tengige1/3/0/6.30 location 1/3/cpu0 
 
   
------------------------------------------------------------------------------
  Location Interface-name      Interface-Type   Ingress-index  Egress-index
---------------------------------------------------------------------------
  1/3/CPU0 TenGigE1_3_0_6.30   Sub-interface       20               12
 
   

Comparing the IDB index value of 12 in the show uidb index command to the uidb index value in the following command output shows that the values are the same.

RP/0/0/CPU0:router# show adjacency ipv4 tengige1/3/0/6.30 detail hardware location 
1/3/cpu0 
 
   
Interface                   Address                 Version  Refcount  Protocol
TenGigE1/3/0/6.30           (src mac only)               90         1     ipv4
                           000000000000001243602d8b8100001e0800
                           mtu: 1500, flags 1 0 1
                           453 packets, 42582 bytes
                           453 hw-only-packets, 42582 hw-only-bytes
 ether egress adjacency
  TLU1            : 0x4407
  [HW: 0x00401862 0xc4170800 0x8100001e 0x01060700]
    num. entries  : 1
    uidb index    : 12
    counter msb   : 0x2
    counter lsb   : 0xc417
    vlan e or len : 0x800
    ether len     : 0x8100 (33024)
    vlan info     : 30
    next ptr      : 0x10607
 
   

The following example displays, from a software perspective, features that are enabled on a selected interface. Compare the output to the configuration of the interface and expected features. Verify that the configured features are correctly enabled.

RP/0/0/CPU0:router# show uidb data location 0/6/cpu0 
 
   
--------------------------------------------------------------------------
  Location = 0/6/CPU0
  Index = 0
  Pse direction = INGRESS
 
   
  Global general 16 bytes:
  ------------------------
  ROUTER_ID: 45.104.151.108 
  MINIMUM MASK DESTINATION: 0 / 0 
  MINIMUM MASK SOURCE: 0 / 0
  BYTES OF SNIFF PACKET: 0
  SUPPRESS PUNT ACL: 0
  MPLS PROPAGATE TTL FLAG: 1
  PARITY: 0
  FABRIC QOS ENABLE FLAG: 0
--------------------------------------------------------------------------
  Location = 0/6/CPU0
  Index = 0
  Pse direction = EGRESS
 
   
  Global general 16 bytes:
  ------------------------
  ROUTER_ID: 45.104.151.108 
  MINIMUM MASK DESTINATION: 0 / 0 
  MINIMUM MASK SOURCE: 0 / 0
  BYTES OF SNIFF PACKET: 0
  SUPPRESS PUNT ACL: 0
  MPLS PROPAGATE TTL FLAG: 1
  PARITY: 0
  IPV4 PREFIX ACCNTG: 0
--------------------------------------------------------------------------
  Location = 0/6/CPU0
  Ifname/Ifhandle = GigabitEthernet0_6_5_0
  Index = 1
  Pse direction = INGRESS
 
   
  General 16 bytes:
  -----------------
  IFHANDLE: 0x168002
  STATUS: 0
  IPV4 ENABLE: 0
  IPV6 ENABLE: 0
  MPLS ENABLE: 0
  STATS POINTER: 0x2c400
  SPRAYER QUEUE: 32
  IPV4 MULTICAST: 0
  IPV6 MULTICAST: 0
  USE TABLE ID IPV4: 0
  USE TABLE ID IPV6: 0
  USE TABLE ID MPLS: 0
  TABLE ID: 0
  QOS ENABLE: 0
  QOS ID: 0
  NETFLOW SAMPLING PERIOD: 0
  L2 PKT DROP: 0
  L2 QOS ENABLE: 0
  SRC FWDING: 0
  *BUNDLE IFHANDLE: 0
  *TUNNEL IFHANDLE: 0
  *L2 ENCAP: 3
 
   
  * Not programmed in hardware
.
.
.

Troubleshooting the Interface Manager

These sections describe how to troubleshoot the interface manager (IM):

Interface Manager Control Process

Troubleshooting the Trace Logs for the Interface Manager

Troubleshooting the Client for the Interface Manager

Troubleshooting the Rules for the Interface Manager

Troubleshooting the Control Chain and Interface Information

Troubleshooting the Registrations for the Interface Manager

Interface Manager Control Process

The interface manager (IM) is the main control process in the Packet Forwarding Infrastructure (PFI) and runs one copy on each line card (LC), route processor (RP), and distributed route processor (DRP). The IM process creates interfaces, adds protocols and capsulations, initiates state and MTU walks, and registers for notification changes.

The following sample output from the show im server activity command with the summary keyword shows an overall summary of information about the instance of IM on a particular node:

RP/0/0/CPU0:router# show im server activity summary
 
   
INTERFACE MANAGER ACTIVITY SUMMARY (v1.0)
---------------------------------------------
 
   
Notifications
-----------------------------------------------------------------------
Owner                                                        :    2298
Third Party                                                  :    2058
Total notifications sent                                     :    4356
-----------------------------------------------------------------------
 
   
Control tree nodes
                     intf:      caps: [     ctrl:]     proto:        vc:
-------------------------------------------------------------------------
Created                506       1037 [      1543]        511          3
Deleted                  0          0 [         0]          6          0
Existing               506       1037 [      1543]        505          3
-------------------------------------------------------------------------
Memory (bytes):     194304     157624 [    351928]      67682      49212
-------------------------------------------------------------------------
        Total memory (bytes):     468822
Avg. memory per intf (bytes):        926
          Avg. caps per intf:          2
 Increases of intf proto map:          1
      Increases in proto max:          2
-------------------------------------------------------------------------
 
   
Client requests
Request Type                       Requests:  Operations:     Max Ops:
-----------------------------------------------------------------------
SPECIFY_CONTROL                           23           23            1
NOTIFY_GET                               136          136            1
INTERFACE_REPLICATE                        0            0            0
INTERFACE_CREATE                           4            4            1
INTF_CREATE_WITH_BCAPS                     0            0            0
INTERFACE_DELETE                           0            0            0
DELETE_CONFIRM                             0            0            0
CAPS_BASE_DEFINE                           0            0            0
RESOURCE_ALLOC_DONE                        0            0            0
CAPS_ADD                                   6            6            1
CAPS_REMOVE                                0            0            0
ADOPT_CONFIRM                              0            0            0
PROTO_LOOKUP                               5            5            1
PROTO_REVERSE_LOOKUP                       0            0            0
CAPS_LOOKUP                               10           10            1
CAPS_REVERSE_LOOKUP                        1            1            1
INTF_TYPE_LOOKUP                           3            3            1
INTF_TYPE_REVERSE_LOOKUP                   0            0            0
INTERFACE_FIND                            29           29            1
INTERFACE_NAME                           223          223            1
INTERFACE_GET_BASE_CAPS                    7            7            1
INTERFACE_READY                            4            4            1
INTERFACE_GET_TYPE                         3            3            1
CONTROL_PARENT_INTF_LOOKUP                 0            0            0
INTERFACE_DESC_LOOKUP                      0            0            0
CAPS_DISPLAY_NAME_LOOKUP                   0            0            0
STATE_QUERY                             3263         3263            1
STATE_REGISTER                             1            1            1
STATE_UPDATE                               4            4            1
CREATE_REGISTER                            9            9            1
CREATE_REGISTER_NAME                       0            0            0
CHILDREN_CREATE_REGISTER                   0            0            0
OWNER_SUPPORT_SET                          1            1            1
OWNER_SUPPORT_GET                          0            0            0
OWNER_CHANGE                               0            0            0
CONFIG_NEW_ENCAP                           0            0            0
IF_CONFIGURABLE                            0            0            0
IF_IS_HW                                   0            0            0
MTU_UPDATE                                 4            4            1
MTU_IMPOSE                                 0            0            0
MTU_QUERY                                  1            1            1
MTU_REGISTER                               1            1            1
DATA_CAPS_LOAD                             0            0            0
DATA_CAPS_UNLOAD                           0            0            0
IF_IS_CONTROL_ONLY                         0            0            0
GET_MAIN_INT_PARSER_DATA                   0            0            0
GET_SUB_INT_PARSER_DATA                    0            0            0
REMOVE_CALLBACK                            0            0            0
REPLACE_BASE_CAPS                          0            0            0
CAPS_DPC_DISTRIBUTE                        5            5            1
PROTO_CHAINS_DISABLE                       0            0            0
PROTO_CHAINS_ENABLE                        0            0            0
                                ------------ ------------ ------------
IF_CREATE                                  3          502          500
IF_READY                                   0            0            0
IF_FIND                                  578         1578          500
IF_DELETE                                  0            0            0
DEL_CONFIRM                                0            0            0
IF_UPDATE_PRIMARY                          1            1            1
IF_FLAGS                                   0            0            0
CAPS_ADD                                  26          526          500
CAPS_REMOVE                                0            0            0
CAPS_BASE_DEFINE                           3            3            1
ID_LOOKUP                                  9            9            1
STATE_UPDATE                              14           14            1
MTU_UPDATE                                17          202           93
REG_WALK                                   9           11            2
REG_CREATE                                27           29            3
IF_NAME                                   13           14            2
STATE_QUERY                                0            0            0
MTU_QUERY                                  0            0            0
PIC_UPDATE                                 0            0            0
RSRC_ALLOC                                 3            3            1
REPL_INTF_OPER                             0            0            0
REPL_INTF_OPER_ACK                         0            0            0
INTF_CONFIG_PARAM                          0            0            0
IF_REPLICATE                               0            0            0
ADOPT_CONFIRM                              0            0            0
CFG_NEW_BC                                 0            0            0
REPLACE_BC                                 0            0            0
IF_TYPE_GET                                3            3            1
IF_DESC_LOOKUP                             0            0            0
IF_DESC_LOOKUP_BYNAME                      0            0            0
CAPS_DN_LOOKUP                             0            0            0
CTRLP_IF_LOOKUP                            0            0            0
ID_REV_LOOKUP                              0            0            0
REG_CREATE_NAME                            0            0            0
IF_GET_BC                                  1            1            1
DPC                                       19          521          500
DP_RESTART_REG                             2            3            2
IF_CREATE_CHILDREN                         0            0            0
IF_CREATE_IND                              0            0            0
CAPS_ADD_SP                                0            0            0
CAPS_ADD_TGT                               0            0            0
CAPS_GET_NODESET                           0            0            0
IIR_UL_INFO                                0            0            0
CFG_RESTORE                                0            0            0
Total non-bulk requests:                3743         3743            1
Total     bulk requests:                 728         3420          500
-----------------------------------------------------------------------
Request data received: 230058 bytes
-----------------------------------------------------------------------
 
   
IM to IMProxy download commands
                                      #cmds          #bytes   #mallocs
-----------------------------------------------------------------------
Total                                  5891          135328      11782
Per cmd                                   -              22          2
-----------------------------------------------------------------------
 
   
Downloads
    #Super    #GS_SHM      Total [     Empty     Errors   Timeouts]
-----------------------------------------------------------------------
         3         55         58 [         1          0          0]
-----------------------------------------------------------------------
Total data downloaded:     169652 bytes
Avg. size of download:       2925 bytes
-----------------------------------------------------------------------
 
   
Download Memory (in bytes)
Type                      Reserved      In Use  Max In Use    Max Dnld
-----------------------------------------------------------------------
Super                            -           0      106032      106032
GS_SHM                      821688           0       12196       12196
Atomic Ring                 448016      400000           -           -
-----------------------------------------------------------------------
 
   
Rules
-----------------------------------------------------------------------
      Max Protocol:                                             :   85
  Max Protocol Chg:                                             :    2
-----------------------------------------------------------------------
 
   
Interface Handle Allocation Memory: (element is 16 bytes)
Allocation Type:            Initial Sz: Chunk Sz: #Chunks: Max Chunks:
-----------------------------------------------------------------------
Physical interface handles            0        32       16          16
Virtual interface handles             0        32        0           0
Other physical handles                0       512        4           4
Other virtual handles                 0       512        0           0
-----------------------------------------------------------------------
Initial Memory:          0 bytes
Current Memory:      40960 bytes
    Max Memory:      40960 bytes
-----------------------------------------------------------------------
 
   
Interface Handle Freelist Memory (element is 16 bytes)
Freelist Type                             Size    Max Size      Memory
-----------------------------------------------------------------------
Physical interface handles                 518         544        8288
Virtual interface handles                    0           0           0
Other physical handles                    2551        2560       40816
Other virtual handles                        0           0           0
-----------------------------------------------------------------------
Current usage                             3069                   49104
Max usage                                             3104       49664
-----------------------------------------------------------------------
 
   
PFI-IFH Broadcast Queue Memory (element is 20 bytes)
Allocation Type:            Initial Sz: Chunk Sz: #Chunks: Max Chunks:
-----------------------------------------------------------------------
Create Queue                       1024      1024        0           0
Delete Queue                          0         0        0           0
-----------------------------------------------------------------------
Current Total Memory:      20480 bytes
          Max Memory:      20480 bytes
-----------------------------------------------------------------------
 
   
1500 items committed in 23 sec (62)items/sec
----------------------------------------------------------------------
Updating
Timer Queue (element is 24 bytes)
Queue                                       Size   Max Size     Memory
-----------------------------------------------------------------------
State                                         11         11        264
MTU                                          706        706      16944
Delete                                         0          0          0
-----------------------------------------------------------------------
Current Total                                717        717      17208
Max Memory                                                       17208
Freelist                                       0          8          0
-----------------------------------------------------------------------
 
   
Bulk stores
Store                                Entries  Max Entries       Memory
-----------------------------------------------------------------------
Bulk                                       0            0            0
Bulk Delete                                0            0            0
 
   
Updated Commit database in 1 sec
ulk Delete                                0            0            0
DP Restart                                 2            2           56
-----------------------------------------------------------------------
Total                                      2            2           56
-----------------------------------------------------------------------
 
   
Client callback information
          #Active             #Dead             Total    Memory (bytes)
-----------------------------------------------------------------------
               23                 0                23             19568
-----------------------------------------------------------------------
Requested Client Connections                               :        23
Requested Disconnections                                   :         0
Unrequested Disconnections                                 :         0
IM server (re)start count                                  :         1
-----------------------------------------------------------------------
 
   
Third Party Registrations
    Create/Delete             State               MTU    Memory (bytes)
-----------------------------------------------------------------------
               36                 6                 7              1072
-----------------------------------------------------------------------
 
   
IM->IMD communications information
Type         Chunk Sz    #Chunks       Used        Max        Mem    Max Mem
----------------------------------------------------------------------------
Big              1024         30          0          0          0          0
Small             512        300         44       1358        880     106800
No-chunk            0          0          0          0          0          0
Total                        330         44       1358        880     106800
----------------------------------------------------------------------------
Pre-allocated memory for small chunks               :     155057 bytes
Pre-allocated memory for big chunks                 :      31095 bytes
----------------------------------------------------------------------------
Transmit queue size: 142 elements
[range MIN:100 -> MAX:1000
 based on nfn size: 108 bytes, optimal msg size: 15376 bytes]
----------------------------------------------------------------------------
Comms type      Count    #Elem  Average  Success   #Retry     Fail   #Retry
Tx start         1875   945489      504       72        0        0        0
Tx done            73     4054       55
----------------------------------------------------------------------------
Total bytes sent:          186332
       attempted:          186332
----------------------------------------------------------------------------
 
   
Checkpoint information
----------------------------------------------------------------------------
Timer Expiries                                       :             234
Checkpoints                                          :              19
----------------------------------------------------------------------------
Node status          Chkptd Avg nodes/chkpt    Bytes chkptd Avg bytes/chkpt
Clean                   300              15           35360            1861
Dirty                  2616             137          353984           18630
Total                  2916             153          389344           20491
----------------------------------------------------------------------------
 
   
Memory summary
----------------------------------------------------------------------------
Total memory reserved                               :    1251438 bytes
Total memory used                                   :     997670 bytes
----------------------------------------------------------------------------
 
   

Troubleshooting the Trace Logs for the Interface Manager

To debug the problem that occurred on the router and the IM, use the show im trace command with both the location and internal keywords to trace logs that are recovered from the router for the applicable location.

The debug im errors command enables the server for error debugging. When the problem is narrowed down, debugging is enabled to provide more information.

The debug im callbacks command is used to display when notifications are arriving at the clients from IM or Interface Manager Distributor (IMD). In addition, the command checks to see the number and types of messages that are in the set of notifications.

Troubleshooting the Client for the Interface Manager

A client must be connected to use any of the IM services. If the client is an owner of the nodes in the IM or is registered for the notifications with the IM, the client must specify a callback handle. The show im client command is used to find which clients are connected and have registered callback handles.


Note The context is a unique parameter that the IM uses to identify a client and the registration. The IM is allowed to know when a client has reconnected to the IM after a process restart. Because the IM tracks the client's status, the show im client command is also used to display which clients have previously connected to the IM. If there is no connection, the status is dead.


The following sample output from the show im client command shows which clients are connected:

RP/0/0/CPU0:router# show im client info
 
   
INTERFACE MANAGER ACTIVITY CLIENT-INFO
--------------------------------------
 
   
Process                PID     JID   Context                 Status  Handle
-------------------- ------- ------- ----------------------- ------- ----------
pfi_ifh_server         53321     246 pfi_ifh                 alive   0x30000001
sysldr                 53297     287 slr_im_evc              alive   0x30000002
driver_infra_partner   24603      54 di-node0_0_CPU0-2       alive   0x30000003
ether_caps_partner     61515     154 ether                   alive   0x30000004
qos_ma                 61517     254 qos_ma                  alive   0x30000005
driver_infra_partner   24603      54 caps_netio_0            alive   0x30000006
aib                    61520     100 aib                     alive   0x30000007
ipv4_fib_mgr           61525     188 ipv4_fib_cfg_im_connect alive   0x30000008
arp                    61522     102 arp                     alive   0x30000009
arp                    61522     102 arp_ipv4_caps_reg       alive   0x3000000a
ipv4_io                65623     189 ipv4                    alive   0x3000000b
ntpd                   65632     192 ntpv4                   alive   0x3000000c
cdp                    65633     125 cdp                     alive   0x3000000d
cdp                    65633     125 cdp_caps                alive   0x3000000e
mpls_lfd               65631     227 mpls_lfd                alive   0x3000000f
tcp                    65640     294 tcp                     alive   0x30000010
clns                   65643     130 clns_transport          alive   0x30000011
parser_server          65657     244 parser                  alive   0x30000012
bundlemgr_distrib      65653     124 BM-DISTRIB_LOCAL        alive   0x30000013
nd_partner             65671     235 Null0                   alive   0x30000014
ipv6_io                65670     202 ipv6-fint               alive   0x30000015
null_caps_partner      69770     239 null_caps_partner       alive   0x30000016
ipv6_nd                65672     204 ipv6-nd                 alive   0x30000017
ipv6_grp               69785     201 ipv6-grp                alive   0x30000018
mpls_lsd               69788     229 mpls_lsd                alive   0x30000019
fint_partner           53302     157 FINT                    alive   0x30000000
 
   

Troubleshooting the Rules for the Interface Manager

A process that calls the IM to create an interface or add a capsulation is defined as an owner. The owners own sets of triplets from an interface, protocol, or capsulation. The triplets are secure domain router (SDR) unique identifiers to control the nodes.

The interface handle is displayed in hexadecimal and contains platform-specific mapping for the node ID, virtual bit, and interface instance number for the node ID.

The protocol and capsulation numbers are small numbers and are displayed in hexadecimal or decimal. They represent a unique identifier for a specific protocol such as IPv4, IPv6, or MPLS.

The show im rules command is used to set interfaces, protocols, and capsulations that are installed on the router.

The following sample output from the show im rules command with the interface-types keyword shows the types of interfaces:

RP/0/0/CPU0:router# show im rules interface-types
 
   
IM rules interface-type data
============================
tag:                 IFT_LOOPBACK
id:                  16
source:              /pkg/rules/loopback.intf
description:         Loopback interface(s)
allowed-base-caps:   loopback
default-base-caps:   loopback
dynamic-ifname:      Loopback
owner-string:        loopback
default-mtu:         1514
subint-range:        0 - 65535
invisibility:        0x2
ignore:              mtu delete 
 
   

The following sample output from the show im rules command with the capsulations keyword shows the rule of each type:

RP/0/0/CPU0:router# show im rules capsulations
 
   
IM rules caps data
==================
tag:                 ipv6_preroute
id:                  128
source:              /pkg/rules/ipv6.caps
description:         IPv6 preroute capsulation nodes
dll:                 libipv6_netio.dll
ordering:            0x6fffffff
switching:           decaps encaps 
ignore:              state mtu 
 
   

The following sample output from the show im rules command with the protocols keyword shows the rules for the protocols:

RP/0/0/CPU0:router# show im rules protocols
 
   
IM rules proto data
===================
tag:                 fint_n2n
id:                  6
source:              /pkg/rules/fint_n2n.caps
description:         Forwarder netio 2 netio packet forwarding protocol
dll:                 libfint_n2n.dll
chain:               use-base-caps 
 
   

If the rules for a specific capsulation, protocol, or interface are missing from the IM on a given node, you cannot create a control node for the type. If the rules are on the router, a process restart of ifmgr can resynchronize the rules.

Troubleshooting the Control Chain and Interface Information

When the control nodes are created, you can use both the show im chains command and the show im children command to see the given location and state for the IM nodes. Both commands display the ASCII names, which map to the interface, protocol, and capsulation triplets.


Tip Use the following tips to:

Only configurable interfaces display the names that are in the list. If the location, all, and ifhandle keywords are used, all relevant interfaces are shown (for example, SONET).

Take care when using the location keyword. If the wrong location is specified, the queried IM does not contain the chains that you are looking for. For example, the POS 0/2/0/0 location does not display anything.

The show im chains command is also used as an interface handle (ifhandle) to find a name.


The following sample output from the show im chains command shows all the interface PICs and the current interface flags:

RP/0/0/CPU0:router# show im chains location all
 
   
Showing all interface control chains:
--------------------------------------------
Interface MgmtEth0/0/CPU0/0, ifh 0x01000100 (up)
  [pic:0x0, intf_flags:0x5]
  Protocol        Caps (state, mtu)
   <base>          txm_nopull (up, 1514)
                   queue_fifo (up, 1514)
                   ether (up, 1514)
   arp             arp (up, 1500)
   ipv4            ipv4 (up, 1500)
   ether_sock      ether_sock (up, 1500)
 
   
Interface Null0, ifh 0x01000080 (up)
  [pic:0x0, intf_flags:0x1c]
  Protocol        Caps (state, mtu)
   <base>          null (up, 1500)
 
   
...
 
   
Interface POS0/2/0/0, ifh 0x03000400 (administratively down)
  [pic:0x0, intf_flags:0x15]
  Parent interface: SonetPath0_2_0_0, ifh 0x03000300
  Protocol        Caps (state, mtu)
   <base>          txm_nopull (administratively down, 4474)
                   queue_fifo (administratively down, 4474)
                   hdlc (administratively down, 4474)
   chdlc           chdlc (administratively down, 4470)
                   slarp (administratively down, 4470)
 
   
...
 
   
Controller SONET0/3/0/0, ifh 0x04000200 (administratively down)
  [pic:0x0, intf_flags:0x37]
 
   

The following sample output from the show im children command with the ifhandle keyword shows when the subinterfaces or other control parent and child summaries are required:

RP/0/0/CPU0:router# show im children ifhandle 0x03000200
 
   
SONET0/2/0/0, ifh 0x03000200 (administratively down)
    SonetPath0/2/0/0, ifh 0x03000300 (administratively down)
      POS0/2/0/0, ifh 0x03000400 (administratively down)
 
   

When the interfaces are created, use the show interfaces command to examine the state. The following sample output is from the show interfaces command for POS 0/2/0/0:

RP/0/0/CPU0:router# show interfaces POS 0/2/0/0
 
   
POS0/2/0/0 is up, line protocol is up
  Hardware is Packet over SONET
  Description: router4 POS 2\0
  Internet address is 3.4.5.6/24
  MTU 4474 bytes, BW 155520 Kbit
     reliability 25/255, txload Unknown, rxload Unknown
  Encapsulation HDLC, crc 16, controller loopback not set, keepalive set (10 sec)
  Last clearing of "show interface" counters never
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 total input drops
     0 drops for unrecognized upper-level protocol
     Received 0 broadcast packets, 0 multicast packets
              0 runts, 0 giants, 0 throttles, 0 parity
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     3 packets output, 62 bytes, 0 total output drops
     Output 0 broadcast packets, 0 multicast packets
     0 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
 
   

The following sample output from the show interfaces command shows the router summary:

RP/0/0/CPU0:router# show interfaces summary
 
   
Interface Type          Total    UP       Down     Admin Down
--------------          -----    --       ----     ----------
ALL TYPES               54       32       1        21      
--------------         
IFT_ETHERBUNDLE         1        1        0        0       
IFT_VLAN_SUBIF          3        3        0        0       
IFT_POSBUNDLE           1        1        0        0       
IFT_LOOPBACK            1        1        0        0       
IFT_NULL                1        1        0        0       
IFT_SBC                 2        2        0        0       
IFT_POS                 20       11       1        8       
IFT_SERIAL_T3E3         4        0        0        4       
IFT_GETHERNET           16       8        0        8       
IFT_ETHERNET            4        4        0        0 
 
   

The following sample output from the show interfaces command shows a per interface summary:

RP/0/0/CPU0:router# show interfaces brief
 
   
Intf        Intf         LineP              Encap  MTU       BW
               Name        State        State               Type (byte)   (Kbps)
--------------------------------------------------------------------------------
                Nu0           up           up               Null  1500   Unknown
       Mg0/0/CPU0/0           up           up               ARPA  1514    100000
          PO0/2/0/0           up         down               HDLC  4474    155520
          PO0/2/0/1   admin-down   admin-down               HDLC  4474    155520
          PO0/2/0/2   admin-down   admin-down               HDLC  4474    155520
          PO0/2/0/3   admin-down   admin-down               HDLC  4474    155520
          PO0/7/0/0   admin-down   admin-down               HDLC  4474    622080
 
   

Troubleshooting the Registrations for the Interface Manager

When the interfaces and control nodes are created, various processes want to obtain information that includes the notifications of the state, MTU, and existence.

You can use the show im registrations command to see which clients are registered with the IM. With the exception of the all option, the other options filter the table to show only the rows of a certain type. The location keyword is used to direct the command to a specific IM. The job ID (JID) is used with the show process command to find more information about the client process, or with the show im server activity command with the client-info keyword to find out more about the client connection to the IM.

The following sample output from the show im registrations command shows all the IM registrations:

RP/0/0/CPU0:router# show im registrations all
 
   
INTERFACE MANAGER REGISTRATIONS
Reg: C - Create, CH - Child create, D - Delete, M - MTU, O - Owner, S - State
 
   
Node  JID      Context     Interface Name or Type  Protocol   Capsulation   Reg
                           CH:(parent) type
---- ----- --------------- ----------------------- -------- --------------- ---
   0   189  ipv4_fib_cfg_f           IFT_FINT_INTF        0               0   C 
   0   244         pfi_ifh                     any        0             any   C 
   0   102             arp           IFT_GETHERNET  unknown         unknown   C 
   0   100             aib                     any     ipv6            ipv6   C 
   0   100             aib                     any        0               0   C 
   0   102             arp            IFT_ETHERNET  unknown         unknown   C 
   0   100             aib            FINT0_0_CPU0     ipv6            ipv6   M 
   0   102  arp_ipv4_caps_       MgmtEth0_0_CPU0_0     ipv4            ipv4   S 
  48   148  ipv4_fib_cfg_f           IFT_FINT_INTF        0               0   C 
  48   102             arp                 unknown  unknown         unknown   C 
  48   173         pfi_ifh                     any        0             any   C 
  48   102             arp           IFT_GETHERNET  unknown         unknown   C 
  48   101             aib                     any     ipv6            ipv6   C 
  48   101             aib                     any        0               0   C 
  48   102             arp            IFT_ETHERNET  unknown         unknown   C 
  48    50  SONET-0_3_CPU0  (SonetPath0_3_0_3) any      any               0   CH
  48    50    POS-      30              POS0_3_0_3        0               0   S 
  48    50  SONET-0_3_CPU0              POS0_3_0_3        0               0   S 
  48   101             aib            FINT0_3_CPU0     ipv6            ipv6   M 
 112    50  di-node0_7_CPU            Node0_7_CPU0        0               0   O 
 112   129           ether  GigabitEthernet0_7_0_2  ether_s      ether_sock   O 
 112   129           ether  GigabitEthernet0_7_0_2        0           ether   O 
 112   129           ether  GigabitEthernet0_7_0_1  ether_s      ether_sock   O 
 112   129           ether  GigabitEthernet0_7_0_1        0           ether   O
 
   

The registered client receives the changes for the IM notifications. If a client appears to be missing notifications, see the possible reasons in Table 3-1.

Table 3-1 List of Reasons for Missed Interface Manager Notifications 

Reason
Solution

Client is not connected to IM.

Check the client connection by using the show im client command with the info keyword.

IM is not running.

Check the interface manager by using the show process command with the ifmgr keyword.

Client is not getting pulses from IM.

Check to see if the client is blocked by using the show process command. Use the debug im callbacks command to debug the callbacks from the IM.

Client is processing the notifications slowly and has notifications queued.

Use the show im client command with the notify keyword to see the activity of the IM.


The following sample output from the show im client command with the notify keyword shows the IM activity notifications:

RP/0/0/CPU0:router# show im client notify
 
   
INTERFACE MANAGER ACTIVITY CLIENT-NOTIFY
----------------------------------------
 
   
Context                          #Notifications Max queued
-------------------------------- -------------- ----------
pfi_ifh                                       7          2
slr_im_evc                                    1          1
di-node0_0_CPU0-2                             4          2
 
   

Troubleshooting the Interface Manager Distributor

These sections provide information on how to troubleshoot the Interface Manager Distributor (IMD):

Troubleshooting the Interface Manager Distributor

Troubleshooting the Trace Logs for the Interface Manager Distributor

Troubleshooting the Clients for the Interface Manager Distributor

Troubleshooting the Interface Information for the Interface Manager Distributor

Troubleshooting the Global Registrations for the Interface Manager Distributor

Troubleshooting the Rules for the Interface Manager Distributor

Interface Manager Distributor Overview

The IMD is an aggregation service. The following elements are supported:

Runs one copy on each RP and DRP.

Discovers information about each control node in each interface manager service on each card through the GSP.

Contains the interface name, handle, state, MTU, and capsulation state.

Registers with the IMD for notifications of create, delete, state, and MTU changes for RP and DRP clients (for example, routing protocols) to find information on any interface in the secure domain router (SDR).

IMD cannot be used to update any information.

Every IMD instance is a copy of other instances, because the same updates are received from every IM. Although the IMD runs on the standby cards, the IMD is not aware of the standby issues; instead, the IMD provides a full copy of all information as if it is on an active card and clients can connect, make registrations, and make queries.

Troubleshooting the Trace Logs for the Interface Manager Distributor

If the problem occurred on the router and affected the IMD, previous information is the most useful. You can use the show imds trace command with the location and file keywords to ensure that the trace logs are recovered from the router for the applicable location.

If there are problems with the connections to the IMD server, use the debug imd client command. For problems with the IMD contents (for example, not synchronized with the IM or with the configuration), use the debug imd imdc command and debug imd collector command. For registrations and notification, use the debug imd filter command and the debug imd notifier command. For show command issues, use the debug imd edm command. For IMD update processing, use the debug imd interface command and the debug imd msg command.

No specific IMD client-side debugging is provided. Because the IMD clients use the IM client library, the same IM client debugging is used.

Troubleshooting the Clients for the Interface Manager Distributor

To use any of the IMD services, a client must be connected. IMD clients who want to register notifications must specify a callback handle. Part of the function of the application programming interface (API) is to pass a unique client context string.

You can use the show imds client command to find out which clients are connected and registered to callback handles, as shown in the following sample output:


Note The Filter Ref column refers to the number of registration filters that the client is referencing. If multiple clients make the same set of registrations, the filters are shared between clients. The Interface Ref column refers to the number of interfaces in which the client currently has registrations.


RP/0/0/CPU0:router# show imds client location 0/0/CPU0
 
   
IMD CLIENTS
Node: 0x0
Internal States
---------------
State: 0x02    LC_req:  0    LC_res: 12    ID: 0x00
Sync:  4       Gather:  0    Resync:  0    Check:  0
 
   
Clients
        Name                ID     Filter Ref Interface Ref
----------------------- ---------- ---------- -------------
ipv4_arm                0x00000064 1          9             
ipv6_arm                0x00000065 1          4             
arp                     0x30000009 4          0             
parser                  0x30000012 2          0             
di-node0_0_CPU0-2       0x30000003 0          0             
mpls_lsd                0x30000019 0          0             
ipv6-grp                0x30000018 1          0             
slr_im_evc              0x30000002 1          0             
arp_ipv4_caps_reg       0x3000000a 0          1             
ISIS_207_BaseCaps       0x00000068 2          0             
ISIS_207_CLNS           0x00000069 2          4             
ISIS_207_v4/v6          0x0000006a 2          2
 
   

Troubleshooting the Interface Information for the Interface Manager Distributor

The show imds interface command is used to display the contents of the IMD database. The following sample output from the show imds interface command shows the interface information for the IMD database :

RP/0/0/CPU0:router# show imds interface all location 0/0/CPU0
 
   
IMDS INTERFACE DATA (Node 0x0)
 
   
FINT0_0_CPU0 (0x01000000)
-----------------------
flags: 0x00000007    type: 27 (IFT_FINT_INTF)    encap: 91 (fint_base)
state: 3 (up)     mtu: 8000    protocol count: 9
control parent: 0x00000000     data parent: 0x00000000
       protocol         capsulation           state        mtu
    --------------- -------------------- --------------- --------
    0 (Unknown)    
                    91 (fint_base)       3 (up)          6000    
    18 (lpts)      
                    81 (lpts)            3 (up)          6000    
    6 (fint_n2n)   
                    92 (fint_n2n)        3 (up)          6000    
    10 (clns)      
                    15 (clns)            3 (up)          6000    
    12 (ipv4)      
                    26 (ipv4)            2 (down)        0       
    30 (ipv4_prero 
                    115 (ipv4_preroute)  3 (up)          6000    
    13 (mpls)      
                    25 (mpls)            3 (up)          6000    
    32 (ipv6_prero 
                    128 (ipv6_preroute)  3 (up)          6000    
    19 (ipv6)      
                    90 (ipv6_preswitch)  3 (up)          6000    
                    82 (ipv6)            3 (up)          6000    
 
   
Null0 (0x01000080)
-----------------------
flags: 0x000000ab    type: 17 (IFT_NULL)    encap: 17 (null)
state: 3 (up)     mtu: 1500    protocol count: 1
control parent: 0x00000000     data parent: 0x00000000
       protocol         capsulation           state        mtu
    --------------- -------------------- --------------- --------
    0 (Unknown)    
                    17 (null)            3 (up)          1500    
 
   
MgmtEth0_0_CPU0_0 (0x01000100)
-----------------------
flags: 0x0000002f    type: 8 (IFT_ETHERNET)    encap: 30 (ether)
state: 3 (up)     mtu: 1514    protocol count: 4
control parent: 0x00000000     data parent: 0x00000000
       protocol         capsulation           state        mtu
    --------------- -------------------- --------------- --------
    0 (Unknown)    
                    60 (txm_nopull)      3 (up)          1514    
                    56 (queue_fifo)      3 (up)          1514    
                    30 (ether)           3 (up)          1514    
    22 (ether_sock 
                    98 (ether_sock)      3 (up)          1500    
    12 (ipv4)      
                    26 (ipv4)            3 (up)          1500    
    7 (arp)        
                    24 (arp)             3 (up)          1500    
 
   
SONET0_3_0_0 (0x04000200)
-----------------------
flags: 0x0000006d    type: 22 (IFT_SONET)    encap: 0 (Unknown)
state: 3 (up)     mtu: 10000    protocol count: 0
control parent: 0x00000000     data parent: 0x00000000
 
   
SonetPath0_3_0_0 (0x04000300)
-----------------------
flags: 0x00000005    type: 33 (IFT_SONET_PATH)    encap: 0 (Unknown)
state: 3 (up)     mtu: 10000    protocol count: 0
control parent: 0x04000200     data parent: 0x00000000
 
   
POS0_3_0_0 (0x04000400)
-----------------------
flags: 0x0000002f    type: 19 (IFT_POS)    encap: 14 (hdlc)
state: 3 (up)     mtu: 4474    protocol count: 4
control parent: 0x04000300     data parent: 0x00000000
       protocol         capsulation           state        mtu
    --------------- -------------------- --------------- --------
    0 (Unknown)    
                    60 (txm_nopull)      3 (up)          4474    
                    56 (queue_fifo)      3 (up)          4474    
                    14 (hdlc)            2 (down)        4474    
    9 (chdlc)      
                    13 (chdlc)           2 (down)        4470    
                    12 (slarp)           2 (down)        4470    
    10 (clns)      
                    15 (clns)            2 (down)        4470    
    12 (ipv4)      
                    26 (ipv4)            2 (down)        4470 
 
   

Troubleshooting the Global Registrations for the Interface Manager Distributor

The show imds registrations command is used to display the information for the IMD client registrations.

IMD supports the concept of an encapsulation change registration so that clients are informed of the changes on an interface.

The following sample output from the show imds registrations command with the all keyword shows all the IMD registrations:

RP/0/0/CPU0:router# show imds registrations all location 0/0/CPU0
 
   
IMD REGISTRATIONS
Reg: C - Create, E - Encap, M - MTU, S - State
 
   
Node   JID  Context         Interface Name or Type  Protocol Capsulation     Reg
------ ---- --------------- ----------------------- -------- --------------- ---
0x0    187  ipv4_arm        any                     ipv4     ipv4            C  
0x0    198  ipv6_arm        any                     ipv6     ipv6            C  
0x0    244  parser          any                     0        0               C  
0x0    201  ipv6-grp        any                     ipv6     ipv6            C  
0x0    287  slr_im_evc      IFT_OTHER               0        0               C  
0x0    102  arp             IFT_ETHERNET            mpls     any             C  
0x0    102  arp             IFT_GETHERNET           mpls     any             C  
0x0    102  arp             IFT_ETHERBUNDLE         mpls     any             C  
0x0    102  arp             IFT_VLAN_SUBIF          mpls     any             C  
0x0    207  ISIS_207_BaseC  POS0_3_0_0              0        any             C  
0x0    207  ISIS_207_CLNS   0                       clns     clns            C  
0x0    207  ISIS_207_v4/v6  0                       ipv4     ipv4            C  
0x0    207  ISIS_207_CLNS   0                       clns     clns            C  
0x0    207  ISIS_207_v4/v6  0                       ipv4     ipv4            C  
0x0    244  parser                                                           E  
0x0    187  ipv4_arm        FINT0_0_CPU0            ipv4     ipv4            S  
0x0    198  ipv6_arm        FINT0_0_CPU0            ipv6     ipv6            S  
0x0    187  ipv4_arm        MgmtEth0_0_CPU0_0       ipv4     ipv4            S  
0x0    102  arp_ipv4_caps_  MgmtEth0_0_CPU0_0       ipv4     ipv4            S  
0x0    187  ipv4_arm        FINT0_3_CPU0            ipv4     ipv4            S  
0x0    198  ipv6_arm        FINT0_3_CPU0            ipv6     ipv6            S  
0x0    207  ISIS_207_CLNS   POS0_3_0_0              clns     clns            M  
0x0    207  ISIS_207_CLNS   POS0_3_0_0              clns     clns            S  
0x0    187  ipv4_arm        POS0_3_0_0              ipv4     ipv4            S  
0x0    207  ISIS_207_v4/v6  POS0_3_0_0              ipv4     ipv4            S 
 
   

The following sample output from the show imds registrations command shows how the context keyword is useful for a particular client callback:

RP/0/0/CPU0:router# show imds registrations context ipv4_connected all
 
   
IMD REGISTRATIONS
 
   
Context: ipv4_connected
Node   JID   Interface Name or Type  Protocol Capsulation     Reg    
------ ----- ----------------------- -------- --------------- -------
0x0    0     any                     ipv4     ipv4            Create 
0x0    0     any                     0        0               Create 
0x0    0     MgmtEth0/0/CPU0/0       ipv4     ipv4            State  
0x0    0     GigabitEthernet0/7/0/0  ipv4     ipv4            State  
0x0    0     GigabitEthernet0/7/0/1  ipv4     ipv4            State  
0x0    0     GigabitEthernet0/7/0/2  ipv4     ipv4            State 
 
   

Troubleshooting the Rules for the Interface Manager Distributor

The show imds rules command is used to display information for the IMD rules, which is a summary of basic information cached in the IMD. The show commands for the IM are used to display detailed data information on the rules.