Contents

Introduction

Prerequisites

Requirements

Components Used

Background Information

Network Diagram

Configure

Troubleshoot

Consistency Checker

Useful commands to gather and check for any Q-in-Q issue

Related Information

Introduction

This document describes how to configure and troubleshoot Selective Q-in-Q in Nexus 9300 (EX-FX-GX) model.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

- Q-in-Q encapsulation

- Dot1q switching

- Nexus 9000

Components Used

The information in this document is based on these software and hardware versions:

- Hardware: N9K-C9336C-FX2

- Version: 10.2(3)

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

A Q-in-Q VLAN tunnel enables a service provider to segregate the traffic of different users in their infrastructure, while still giving the you a full range of VLANs for their internal use by adding a second 802.1Q tag to an already tagged frame.

Selective Q-in-Q with multiple provider VLANs is a tunneling feature that allows user-specific range of VLANs on a port to be associated with one specific provider VLAN.

Q-in-Q allows you to have multiple user VLAN-to-provider VLAN mappings on a port.

Packets that come in with a VLAN tag that matches any of the configured user VLANs on the port are tunneled across the fabric using the properties of the service provider VLAN. The encapsulated packet carries the user VLAN tag as part of the Layer 2 header of the inner packet.

Network Diagram

Configure

The goal of the configuration of Selective Q-in-Q is to map the VLANs from the traffic received in N9KFX2 and N9KFX2-1 on interfaces eth1/1.

The Nexus encapsulates the traffic and sends it with the DOT1Q header VLAN 20 to the provider, simulated by Nexus 93180FX.

Host-A

interface Ethernet1/1

switchport

switchport mode trunk

no shutdown

interface Vlan10

no shutdown

ip address 192.168.1.1/24

Host-B

interface Ethernet1/1

switchport

switchport mode trunk

no shutdown

interface Vlan10

no shutdown

ip address 192.168.1.2/24

N9KFX2

interface Ethernet1/1

switchport

switchport mode trunk

switchport vlan mapping all dot1q-tunnel 20

switchport trunk allowed vlan 20

spanning-tree port type edge trunk

spanning-tree bpdufilter enable

no shutdown

interface Ethernet1/35

switchport

switchport mode trunk

no shutdown

N93180FX

system dot1q-tunnel transit

interface Ethernet1/17

switchport

switchport mode trunk

no shutdown

interface Ethernet1/18

switchport

switchport mode trunk

no shutdown

N9KFX2-1

interface Ethernet1/1

switchport

switchport mode trunk

switchport vlan mapping all dot1q-tunnel 20

switchport trunk allowed vlan 20

spanning-tree port type edge trunk

spanning-tree bpdufilter enable

no shutdown

interface Ethernet1/35

switchport

switchport mode trunk

no shutdown

Troubleshoot

The Q-in-Q programming moves from the control plane to the hardware through the components:

:

Consistency Checker

Recent versions of the Nexus switch have consistency checker to use in case misprogramming.

N9KFX2# show consistency-checker selective-qinq

Box level hardware programming checks      : PASS

Fetching ingressVlanXlate entries from slice: 0 HW

Fetching ingressVlanXlate entries from slice: 1 HW

Fetching ingressVlanXlate TCAM entries

Performing port specific checks for intf Eth1/1

Port specific selective QinQ checks for interface  Eth1/1 : PASS

Selective QinQ Overall status      : PASS

Control plane

Confirm if the VLAN is active on the ports.

N9KFX2#sh vlan

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

1    default                          active    Eth1/1

20   VLAN0020                         active    Eth1/1, Eth1/35

Confirm that the VLAN mapping is clearly seen.

N9KFX2# sh vlan internal info mapping

Per Port Vlan Translation Details

================

Entry No                :  0

----------------------

ifindex Eth1/1(0x1a000000)

vlan mapping enabled: TRUE

vlan translation mapping information (count=1):

Original Vlan      Translated Vlan

-------------      ---------------

all                  20

Entry No                :  1

----------------------

ifindex Eth1/35(0x1a004400)

vlan mapping enabled: FALSE

vlan translation mapping information (count=0):

Original Vlan      Translated Vlan

-------------      ---------------

ELTM

Confirm if there is any errors increasing or non zero.

N9KFX2# show system internal eltm error count

interface config error: 0

dce error: 0

port struct error: 0

port channel error: 0

tunnel error: 0

tlv count error: 0

vpn count error: 0

SI add to DB error: 0

Run the command attach module X. The X is the module where the physical interface is allocated.

N9KFX2# attach module 1

module-1# show system internal eltm info vlan 20

IFTMC Detailed info for VLAN = 20

cr_flags = VLAN , bd = 21(0x15)  stg-id = 0 type = 1 stp:0

shut_state = 2 , oper_state = 1

vni = 0, ipmc_idx = 0x0 ipmc_remap_idx = 0x0 ipmc_remap_idx2 = 0x0 dvif: 0

ipmc_mrouter_idx = 0x0 ipmc_mrouter_remap_idx = 0x0 ipmc_mrouter_remap_idx2 = 0x0

pbp_idx=0

eid[0] = 0, alt_eid[0] = 0eid[1] = 0, alt_eid[1] = 0eid[2] = 0, alt_eid[2] = 0eid[3] = 0, alt_eid[3] = 0eid[4] = 0, alt_eid[4] = 0eid[5] = 0, alt_eid[5] = 0eid[6] = 0, alt_eid[6] = 0eid[7] = 0, alt_eid[7] = 0

Active ports:                                             <<<<< Confirm Active ports are listed for the vlan

Ethernet1/1(0x1a000000)  Ethernet1/35(0x1a004400)

Active Local ports:

Ethernet1/1(0x1a000000)  Ethernet1/35(0x1a004400)

Ports using this VLAN as Provider:                          <<<<< Displays the interface that is doing the tunneling

Ethernet1/1(0x1a000000)

SDB Active ports:

Ethernet1/1

pbp_idx:0

VLAN Features:

prim_vlan      : 0, prim_intf_vlan  : 0, vlan mode      : 0(CE)

pt_cam_en      : 1, ipv4_igmp_snoop  : 1, ipv4_pim_snoop  : 0

ipv6_mld_snoop : 0, ipv6_pim_snoop  : 0, l2l3_lkup_cfg  : 1

v4_table_id    : 1        (0x00000001), v4_vpn_id        : 1 (0x1)

v6_table_id    : 2147483649 (0x80000001), v6_vpn_id      : 1 (0x1)

age_tmr_sel    : 1, sm_en  : 0 fcf  : 0 omf : 1, dhcp_snoop_en  : 0

proxy_learn    : 0, iic_en : 0, use_oxid : 0, age_grp_sel        : 0

disable_learn  : 0, otv_en : 0, global  : FALSE

bc_uc_ftag    : 4095  , mcast_ftag_num : 1, arp_sup: 0x0 ,icmpv6 nd local proxy: 0x0

isdot1qvlan    : 0  , num_dot1q_ports: 0  num_sel_dot1q_qinvni_ports: 0

num_multi_tag_ports: 0,  eid: 0,  xconnect: 0,  mcast_ftag[1]  : {4095}

is_transit    : 0    mdns_en: 0,

TAH:VLAN_CONTROL:

------------------

vrf: 0 forwarding_vlan: 0 forwarding mode: 0

Flags: 0x0

up: 0 vni: 0 is_sr_evpn: 0

You can check for interface programming. The output displays information about the port programming.

N9KFX2# attach module 1

module-1# show system internal eltm info interface e1/1

IFTMC Detailed info for Interface Ethernet1/1(0x1a000000)

CR_flags = INTF LIF  (0x3),  LTL = 6144 (0x1800), (S 0x0, P 0x0)

IF_INDEX = Ethernet1/1(0x1a000000), LIF = 4100 (0x1004), SDB LIF = 4100 (0x1004)

State = UP

Layer = L2, Mode = TRUNK, native_vlan = 1 tag_native_vlan = 0   <<< Confirm mode and Layer

AdminMode = 0x100000 is_monitor = 0 src_if_pruning 1

Operational VLAN's (2):                                         <<< Confirm if the VLAN for tunnel and native are correct

1,20

voice vlan = 4096

MVR Configured VLAN's (0):

local_port = 1, ldb_sharing = 1, ilm_sharing = 0x1

ldb_port_flags = 0x0, dsm = 0, dnl = 0

pc_ifindx = 0x0, fpc = 0x0

Interface Features:

ipv4_en        : 0, ipv4_mcast_en : 0, df_mask : 0, mpls_en : 0

ipv6_en        : 0, ipv6_mcast_en : 0  pim_dr_en :0 ipsg_en  : 0

FC port mode:0 vsan: 0 trunk mode: 0 speed: 0

FCF Enabled : 0 FCF MAC:0000:0000:0000

PTP fid = 0

mpls_strip_en    :0

external port :0

Operational VSAN's (0):

v4_table_id    : 1          (0x00000001), v4_vpn_id    : 1 (0x1)

v6_table_id    : 2147483649 (0x80000001), v6_vpn_id    : 1 (0x1)

per_pkt_ls_en  : 0,  bd              : N/A (0x0)

icmp_redirect  : 1, ipv6_redirect    : 1

v4_same_if_chk : 0, v6_same_if_check : 0

mtu_index      : 1    (0x0001), new_mtu_index  : 1    (0x0001)

mtu            : 1500 (0x05dc), port_trust    : 0

v4_rpf_mode    : 0  , v6_rpf_mode            : 0

v4_rpf_mode (sdb) : 0  , v6_rpf_mode (sdb) : 0

intf_type      : 0x0, intf_flags  : 0x20

sub_flags      : 0x0, port_type  : 0x0, is_aida  : 0x0, vpc_dvp_no  : 0x0

fcoe_mac_ip_chk: 0 buffer_boost: 0 openflow_en: 0 span_buffer_cfg: 0 span_mode: 0 outer_bd = 512

interface mac: 00:00:00:00:00:00 old_mac: 00:00:00:00:00:00 mac_index:0

Src-Idx:1

mc_index : 4100, egress_vsl_drop: 0, pv_en : 0, is_selective_qinvni_port 1 ether_type 0x8100 vlan_bmp_State_change : False, policy flood_l2_local 1.  <<<Confirm the ether type is 8100 and is_selective_qinvni_port is 1

is_multi_tag_port 0, nat_realm_id 0

global_id 0

enm_if_type  : 0

pbp_idx  : 0

pinned_ifindex : 0x0

normal_vlan_bmp (0):

arp_suppress inst[0] 0 arp_suppress inst[1] 0 arp_suppress inst[2] 0 arp_suppress inst[3] 0 arp_suppress inst[4] 0 arp_suppress inst[5] 0 arp_suppress inst[6] 0 arp_suppress inst[7] 0 icmpv6 nd local proxy en inst[0] 0 icmpv6 nd local proxy en inst[1] 0 icmpv6 nd local proxy en inst[2] 0 icmpv6 nd local proxy en inst[3] 0 icmpv6 nd local proxy en inst[4] 0 icmpv6 nd local proxy en inst[5] 0 icmpv6 nd local proxy en inst[6] 0 icmpv6 nd local proxy en inst[7] 0 mdns_en inst[0] 0 mdns_en inst[1] 0 mdns_en ins

t[2] 0 mdns_en inst[3] 0 mdns_en inst[4] 0 mdns_en inst[5] 0 mdns_en inst[6] 0 mdns_en inst[7] 0 arp_suppress inst[0] 0 arp_suppress inst[1] 0 arp_suppress inst[2] 0 arp_suppress inst[3] 0 arp_suppress inst[4] 0 arp_suppress inst[5] 0 arp_suppress inst[6] 0 arp_suppress inst[7] 0 icmpv6 nd local proxy en inst[0] 0 icmpv6 nd local proxy en inst[1] 0 icmpv6 nd local proxy en inst[2] 0 icmpv6 nd local proxy en inst[3] 0 icmpv6 nd local proxy en inst[4] 0 icmpv6 nd local proxy en inst[5] 0 icmpv6 nd local prox

y en inst[6] 0 icmpv6 nd local proxy en inst[7] 0 mdns_en inst[0] 0 mdns_en inst[1] 0 mdns_en inst[2] 0 mdns_en inst[3] 0 mdns_en inst[4] 0 mdns_en inst[5] 0 mdns_en inst[6] 0 mdns_en inst[7]

service provider Vlans 0

vlan_xlt_tlb_en_ingress : 1  num_vlan_xlt_ingr : 1

Vlan Translation Table (INGRESS)         <<< This table displays xlt the vlan used to send to the provider

--------------------------------

in_vlan  xlt_vlan

1      20

SDB INFO:

--------------------------------

MVR VLAN's (0):

STATE = UP, is_mcec = 0, is_mct = 0

TAH:L2 INFO: 4100

----------

Flags : 0x41

NIV Index: 1 Source Chip: 1 Source Port: 72

slice: 1 slice_port: 0

is_monitor: 0 lif: 4100 phy_intf_id: 1 vnic_if: 0 learn_en: 1 native_vlan 1

trunk_port: 1 src_if_check: 1

FC info: intf_type = 0x0 port_mode = 0x0 port_vsan = 0 port_speed = 0 fcf_mac = 0:0:0:0:0:0

IFTMC

For IFTMC you can run the 2 commands. Run the attach module 1 command before running the next commands.

N9KFX2# attach module 1

module-1# show system internal iftmc info interface e1/1

module-1# show system internal iftmc info vlan 20

Both Commands

- show system internal iftmc info interface ethernet1/1

- show system internal iftmc info vlan 20

Displays similar information as ELTM outputs, make sure to verify you have same values between ELTM and IFTMC modules.

Hardware

To confirm the outputs are correctly programmed on the hardware you can run the command show hardware internal tah interface ethernet 1/1. In the output of the command, you can go to the BdStateTable part.

N9KFX2-1(config-if)# show hardware internal tah interface ethernet 1/1

BdStateTable:

ENTRY: 1

info_leaf_flood_dst_ptr : 1                 <<<Displays native vlan

info_leaf_igmp_mld_dst_ptr : 4098

info_leaf_fid : 1                           <<<Displays native vlan

info_leaf_vrf : 1

info_leaf_igmp_snp_en : 1

info_leaf_l3_bind_check_en : 1

info_leaf_v4_omf : 1

info_leaf_unknown_uc_flood : 1

info_leaf_unknown_mc_flood : 1

info_leaf_ecn_mark_en : 1

info_leaf_l2_bind_check_en : 1

info_leaf_bd_profile_idx : 1

info_leaf_analytics_en : 1

info_leaf_flow_collect_en : 1

info_spine_info0_gipo_idx : 1              <<<Displays native vlan

info_spine_info0_scope : 64

info_spine_info0_spare : 0x00000000:0x00100044

info_spine_info1_spare : 0x00001000:0x00000000

info_spine_info2_gipo_idx : 512

info_spine_info2_scope : 12416

info_spine_info2_spare : 0x00005000:0x10000000

info_spine_info3_gipo_idx : 384

info_spine_info3_spare : 0x00000000:0x00000000

=========================================

Useful commands to gather and check for any Q-in-Q issue

- show vlan internal info mapping

- show consistency-checker selective-qinq

- show consistency-checker selective-qinq interface <ethX/Y, port-channel X>

- show hardware internal tah interface ethernet x/y

- show hardware internal tah interface all

- show hardware internal tah sdk event-history error

- show system internal iftmc info interface <>

On Nexus platform you can collect show tech-support more specific than the general, which provides a series of commands:

- show tech details

- show tech vlan

- show tech usd-all

- show tech eltm

- attach module 1 ; show tech iftmc

Note: Look at the guidelines and limitations sections on the configuration guide listed on Related Information for the version your Nexus is running to confirm that your configuration is not hitting any hardware or software limitations.

Related Information

- Release 9.3.X

- Release 10.1.X

- Release 10.2.X

- Release 10.3.X

- Release 10.4.X

- Cisco Technical Support & Downloads