Verify ASR9000 VQI Assignments in CEF

Introduction

This document describes how to verify Virtual Queue Indexes (VQIs) and assign them properly in Cisco Express Forwarding (CEF) in an Aggregation Services Router 9000 (ASR9K).

Background Information

In order for packets to forward from one interface to another in an ASR9K, packets must traverse the fabric. There is no local switching in an ASR9K. How does a packet get from one interface to another though? This is accomplished through the use of VQIs which are assigned to each interface. This way the fabric knows which linecard (LC) and Network Processor (NP) to route the packet.

Sometimes though, as in the case of CSCvc83681 , a wrong VQI might be assigned and traffic can be blackholed inside the router.

Verify VQI Assignments

Refer to this section in order to verify VQI assignments. 

First, identify the ingress and egress interfaces for the flow, source and destination Internet Protocol (IP) address, with show cef <prefix> detail command.

This helps identify which LCs need to be looked at for the VQI assignments.

Here is the source address:

RP/0/RSP0/CPU0:ASR9006-H#show cef 123.29.62.12 detail
Tue May  1 10:54:50.356 EDT
123.29.62.12/32, version 325561, internal 0x1000001 0x0 (ptr 0x76a07a40) [1], 0x0 (0x73ffbf50), 0xa28 (0x75e3133c)
Updated May  1 10:26:51.592
remote adjacency to TenGigE0/1/0/5
Prefix Len 32, traffic index 0, precedence n/a, priority 1
  gateway array (0x74bff484) reference count 3, flags 0x68, source lsd (5), 1 backups
                [2 type 5 flags 0x8401 (0x7216f3d0) ext 0x0 (0x0)]
  LW-LDI[type=5, refc=3, ptr=0x73ffbf50, sh-ldi=0x7216f3d0]
  gateway array update type-time 1 May  1 10:26:51.592
LDI Update time May  1 10:26:51.592
LW-LDI-TS May  1 10:26:51.592
   via 10.94.1.182/32, TenGigE0/1/0/5, 6 dependencies, weight 0, class 0 [flags 0x0]
    path-idx 0 NHID 0x0 [0x7181cfc4 0x0]
    next hop 10.94.1.182/32
    remote adjacency
     local label 24088      labels imposed {86}
   via 10.94.1.150/32, TenGigE0/1/0/7, 6 dependencies, weight 0, class 0 [flags 0x0]
    path-idx 1 NHID 0x0 [0x7181d018 0x0]
    next hop 10.94.1.150/32
    remote adjacency
     local label 24088      labels imposed {86}


    Load distribution: 0 1 (refcount 2)

    Hash  OK  Interface                 Address
    0     Y   TenGigE0/1/0/5            remote
    1     Y   TenGigE0/1/0/7            remote

Here is the destination address:

RP/0/RSP0/CPU0:ASR9006-H#show cef 123.29.62.1 detail
Tue May  1 10:53:14.531 EDT
123.29.62.1/32, version 334286, internal 0x1000001 0x0 (ptr 0x74bf1a04) [1], 0x0 (0x73ffbeb0), 0xa20 (0x75e310d4)
Updated May  1 10:53:12.459
remote adjacency to TenGigE0/0/0/2
Prefix Len 32, traffic index 0, precedence n/a, priority 1
  gateway array (0x74c025ec) reference count 27, flags 0x68, source lsd (5), 1 backups
                [19 type 4 flags 0x8401 (0x7216f390) ext 0x0 (0x0)]
  LW-LDI[type=1, refc=1, ptr=0x73ffbeb0, sh-ldi=0x7216f390]
  gateway array update type-time 1 Apr 30 17:03:05.246
LDI Update time Apr 30 17:03:05.246
LW-LDI-TS Apr 30 17:03:05.247
   via 10.94.0.10/32, TenGigE0/0/0/2, 4 dependencies, weight 0, class 0 [flags 0x0]
    path-idx 0 NHID 0x0 [0x7181ce20 0x7181d06c]
    next hop 10.94.0.10/32
    remote adjacency
     local label 24012      labels imposed {ImplNull}
   via 10.94.2.9/32, TenGigE0/0/0/3, 4 dependencies, weight 0, class 0 [flags 0x0]
    path-idx 1 NHID 0x0 [0x7181ce74 0x7181d0c0]
    next hop 10.94.2.9/32
    remote adjacency
     local label 24012      labels imposed {ImplNull}


    Load distribution: 0 1 (refcount 19)

    Hash  OK  Interface                 Address
    0     Y   TenGigE0/0/0/2            remote
    1     Y   TenGigE0/0/0/3            remote

From these outputs, you see that LC 1 is the ingress LC and LC 0 is the egress LC, both have two ports in order to load-balance the traffic.

Next, you need to identify how many NPs are on the ingress and egress LC with the show controller np ports all loc <LC> command.

The ingress LC has 8 NPs:

RP/0/RSP0/CPU0:ASR9006-H#show controller np ports all loc 0/1/CPU0
Tue May  1 10:56:57.996 EDT

                Node: 0/1/CPU0:
----------------------------------------------------------------

NP Bridge Fia                       Ports
-- ------ --- ---------------------------------------------------
0  --     0   TenGigE0/1/0/0 - TenGigE0/1/0/2
1  --     0   TenGigE0/1/0/3 - TenGigE0/1/0/5
2  --     1   TenGigE0/1/0/6 - TenGigE0/1/0/8
3  --     1   TenGigE0/1/0/9 - TenGigE0/1/0/11
4  --     2   TenGigE0/1/0/12 - TenGigE0/1/0/14
5  --     2   TenGigE0/1/0/15 - TenGigE0/1/0/17
6  --     3   TenGigE0/1/0/18 - TenGigE0/1/0/20
7  --     3   TenGigE0/1/0/21 - TenGigE0/1/0/23

The egress LC has 2 NPs:

RP/0/RSP0/CPU0:ASR9006-H#show controller np ports all loc 0/0/cPU0
Tue May  1 10:55:27.661 EDT

                Node: 0/0/CPU0:
----------------------------------------------------------------

NP Bridge Fia                       Ports
-- ------ --- ---------------------------------------------------
0  --     0   TenGigE0/0/0/0 - TenGigE0/0/0/3
1  --     1   TenGigE0/0/1/0 - TenGigE0/0/1/3

Next, check the ingress LC with the show cef <destination prefix> hardware ingress detail loc <ingress lc> | I vqi command and the egress LC with the show cef <dst prefix> hardware egress detail loc <egress lc> I vqi command.

This information gives us information about how each NP is programmed to reach the egress interfaces. In this case, because there are eight NPs on the ingress LC and two Equal Cost Multi-Path (ECMP) links on the egress LC, there are 16 entries. The first eight entries are for the first ECMP link and the next eight entries are for the second ECMP link. Each set of eight should match, and it means that each NP is programmed to do the same. Each set should be different, although there are two separate interfaces. If they are the same, then you might be hitting a VQI CEF misprogramming problem.

RP/0/RSP0/CPU0:ASR9006-H#show cef 123.29.62.1 hardware ingress loc 0/1/CPU0 | i vqi
Tue May  1 10:56:27.064 EDT
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59

Check the egress LC to ensure that it is programmed properly. In this case, there are two NPs and two ECMP links so there are two sets of two VQIs that need to be programmed.

RP/0/RSP0/CPU0:ASR9006-H#show cef 123.29.62.1 hardware egress loc 0/0/CPU0 | i vqi
Tue May  1 10:57:29.221 EDT
         out_lbl_invalid: 0               match: 0          vqi/lag-id: 0x0
         out_lbl_invalid: 0               match: 0          vqi/lag-id: 0x0
    sfp/vqi         : 0x58
    sfp/vqi         : 0x58
         out_lbl_invalid: 0               match: 0          vqi/lag-id: 0x0
         out_lbl_invalid: 0               match: 0          vqi/lag-id: 0x0
    sfp/vqi         : 0x59
    sfp/vqi         : 0x59

The last thing to check is the VQI assignment on the interfaces.

Here, you can check the switch_fabric_port variable and convert from decimal to hex. 88 being 58 and 89 being 59, these values match with the VQI assignments from these commands which mean that CEF is programmed properly for VQI transport in the ASR9K.

RP/0/RSP0/CPU0:ASR9006-H#show controller pm interface ten 0/0/0/2
Tue May  1 10:58:52.024 EDT

Ifname(1): TenGigE0_0_0_2, ifh: 0x4000140 :
iftype             0x1e
egress_uidb_index  0x7, 0x7
ingress_uidb_index 0x7, 0x7
port_num           0x2
subslot_num        0x0
ifsubinst          0x0
ifsubinst port     0x2
phy_port_num       0x2
channel_id         0x0
channel_map        0x0
lag_id             0x0
virtual_port_id    0x0
switch_fabric_port 88
in_tm_qid_fid0     0x20002
in_tm_qid_fid1     0xffffffff
in_qos_drop_base   0x690001
out_tm_qid_fid0    0x20022
out_tm_qid_fid1    0xffffffff
np_port            0x6

out_qos_drop_base  0x6900a1
bandwidth          10000000 kbps
ing_stats_ptrs     0x53016a, 0x0
egr_stats_ptrs     0x53017b, 0x0
l2_transport       0x0
ac_count           0x0
parent_ifh         0x0
parent_bundle_ifh  0x0
L2 protocols bmap  0x1000000
Cluster interface  0

RP/0/RSP0/CPU0:ASR9006-H#show controller pm interface ten 0/0/0/3
Tue May  1 10:59:08.886 EDT

Ifname(1): TenGigE0_0_0_3, ifh: 0x4000180 :
iftype             0x1e
egress_uidb_index  0x8, 0x8
ingress_uidb_index 0x8, 0x8
port_num           0x3
subslot_num        0x0
ifsubinst          0x0
ifsubinst port     0x3
phy_port_num       0x3
channel_id         0x0
channel_map        0x0
lag_id             0x0
virtual_port_id    0x0
switch_fabric_port 89
in_tm_qid_fid0     0x30002
in_tm_qid_fid1     0xffffffff
in_qos_drop_base   0x6e0001
out_tm_qid_fid0    0x30022
out_tm_qid_fid1    0xffffffff
np_port            0x7

out_qos_drop_base  0x6e00a1
bandwidth          10000000 kbps
ing_stats_ptrs     0x530183, 0x0
egr_stats_ptrs     0x530194, 0x0
l2_transport       0x0
ac_count           0x0
parent_ifh         0x0
parent_bundle_ifh  0x0
L2 protocols bmap  0x1000000
Cluster interface  0