Nexus 9000 TCAM值设置为0，丢弃ARP、UDLD、LACP数据包
简介
本文档说明当端口因UDLD错误而关闭时，如何对Nexus 9000 TCAM进行故障排除
它涵盖当前和常见的概念、故障排除方法和错误消息。
本文档的目的是帮助用户了解如何在端口因UDLD错误而关闭时排除TCAM故障
预请求
了解Cisco NXOS命令
拓扑
通过简单的拓扑可以看到问题
(N9k-1)Eth2/1-2 -(N9k-2)Eth2/1-2
1.1.1.1 /24 1.1.1.2/24
故障排除
以下协议无法在控制平面上工作：
ARP解析失败
Nexus 9000上的端口因模块1和2的UDLD错误而报告关闭。
N9K-1(config-if)# 2018 Oct 20 07:23:23 N9K-1 %ETHPORT-5-IF_ADMIN_UP: Interface port-channel100 is admin up .
2018 Oct 20 07:23:23 N9K-1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel100 is down (No operational members)
2018 Oct 20 07:23:23 N9K-1 last message repeated 1 time
2018 Oct 20 07:23:23 N9K-1 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet2/2 is down (Error disabled. Reason:UDLD empty echo)
2018 Oct 20 07:23:23 N9K-1 last message repeated 1 time
2018 Oct 20 07:23:23 N9K-1 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet2/1 is down (Error disabled. Reason:UDLD empty echo)
sh 2018 Oct 20 07:23:25 N9K-1 last message repeated 1 time
线卡因模块1和2的机箱上的L2ACLRedirect诊断测试而失败。
'Show module'
Mod Online Diag Status
--- ------------------
1 Fail————————————cleared the module 1 and 2 error .[show logging nvram]
2 Fail—————————————module 2 reloaded.
3 Pass
Module 1 and 2:
11) L2ACLRedirect-----------------> E
12) BootupPortLoopback: U
客户达到此状态的另一种可能方式是从基于T2 ASIC的机箱迁移到基于Tahoe的机箱的SUP/LC
注意：如果您想了解有关ASIC故障排除的详细信息，请联系思科TAC
CSCvc36411 
从T2升级到基于Tahoe的线卡/FM可能导致诊断故障和TCAM问题
分析
在N9K-2上，当TCAM值设置为0时，会出现此问题
N9K-2# sh hardware access-list tcam region
NAT ACL[nat] size = 0
Ingress PACL [ing-ifacl] size = 0
VACL [vacl] size = 0
Ingress RACL [ing-racl] size = 0
Ingress RBACL [ing-rbacl] size = 0
Ingress L2 QOS [ing-l2-qos] size = 0
Ingress L3/VLAN QOS [ing-l3-vlan-qos] size = 0
Ingress SUP [ing-sup] size = 0
Ingress L2 SPAN filter [ing-l2-span-filter] size =
Ingress L3 SPAN filter [ing-l3-span-filter] size = 0
Ingress FSTAT [ing-fstat] size = 0
span [span] size = 0
Egress RACL [egr-racl] size = 0
Egress SUP [egr-sup] size = 0
Ingress Redirect [ing-redirect] size = 0
要进一步拆除UDLD，但ping操作失败
从N9K-2发出ARP请求
N9K-2# ethanalyzer local interface inband
Capturing on inband
2018-10-23 10:46:47.282551 1.1.1.1 -> 1.1.1.2 ICMP Echo (ping) request
2018-10-23 10:46:47.286072 b0:aa:77:30:75:bf -> ff:ff:ff:ff:ff:ff ARP Who has 1.1.1.1? Tell 1.1.1.2
2018-10-23 10:46:49.284704 1.1.1.1 -> 1.1.1.2 ICMP Echo (ping) request
2018-10-23 10:46:51.286150 b0:aa:77:30:75:bf -> ff:ff:ff:ff:ff:ff ARP Who has 1.1.1.1? Tell 1.1.1.2
2018-10-23 10:46:51.286802 1.1.1.1 -> 1.1.1.2 ICMP Echo (ping) request
2018-10-23 10:46:53.288989 1.1.1.1 -> 1.1.1.2 ICMP Echo (ping) request
2018-10-23 10:46:55.289920 1.1.1.1 -> 1.1.1.2 ICMP Echo (ping) request
2018-10-23 10:46:57.292070 1.1.1.1 -> 1.1.1.2 ICMP Echo (ping) request
2018-10-23 10:46:59.292568 1.1.1.1 -> 1.1.1.2 ICMP Echo (ping) request
2018-10-23 10:46:59.292818 b0:aa:77:30:75:bf -> ff:ff:ff:ff:ff:ff ARP Who has 1.1.1.1? Tell 1.1.1.2
10 packets captured
N9K-1#以太分析器本地接口带内
Capturing on inband
2018-10-23 04:02:40.568119 b0:aa:77:30:75:bf -> ff:ff:ff:ff:ff:ff ARP Who has 1.1.1.1? Tell 1.1.1.2
2018-10-23 04:02:40.568558 cc:46:d6:af:ff:bf -> b0:aa:77:30:75:bf ARP 1.1.1.1 is at cc:46:d6:af:ff:bf
2018-10-23 04:02:48.574800 b0:aa:77:30:75:bf -> ff:ff:ff:ff:ff:ff ARP Who has 1.1.1.1? Tell 1.1.1.2
2018-10-23 04:02:48.575230 cc:46:d6:af:ff:bf -> b0:aa:77:30:75:bf ARP 1.1.1.1 is at cc:46:d6:af:ff:bf————arp reply packet sent by agg1.
N9K-2上的ELAM有来自N9K-1的ARP响应
注意：请联系Cisco TAC以验证ELAM捕获
module-2(TAH-elam-insel6)# reprort
Initting block addresses
SUGARBOWL ELAM REPORT SUMMARY
slot - 2, asic - 1, slice - 0
============================
Incoming Interface: Eth2/2
Src Idx : 0x42, Src BD : 4489
Outgoing Interface Info: dmod 0, dpid 0
Dst Idx : 0x0, Dst BD : 4489
Packet Type: ARP
Dst MAC address: B0:AA:77:30:75:BF
Src MAC address: CC:46:D6:AF:FF:BF
Target Hardware address: B0:AA:77:30:75:BF --------------------------------------- Arp packet captured on Linecard
Sender Hardware address: CC:46:D6:AF:FF:BF
Target Protocol address: 1.1.1.2
Sender Protocol address: 1.1.1.1
ARP opcode: 2
Drop Info:
module-2(TAH-elam-insel6)#
Bug ping仍然失败
N9K-2# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
36 bytes from 1.1.1.2: Destination Host Unreachable
Request 0 timed out
36 bytes from 1.1.1.2: Destination Host Unreachable
Request 1 timed out
36 bytes from 1.1.1.2: Destination Host Unreachable
Request 2 timed out
36 bytes from 1.1.1.2: Destination Host Unreachable
Request 3 timed out
36 bytes from 1.1.1.2: Destination Host Unreachable
N9K-2# show ip arp | inc 1.1.1.1———arp not getting populated
要隔离arp问题，请添加静态arp条目并禁用UDLD
从1.1.1.2到1.1.1.1的静态arp ping开始工作后，如果启用UDLD，它将再次失败
N9K-2(config)# ping 1.1.1.2
PING 1.1.1.2 (1.1.1.2): 56 data bytes
64 bytes from 1.1.1.2: icmp_seq=0 ttl=255 time=0.32 ms
64 bytes from 1.1.1.2: icmp_seq=1 ttl=255 time=0.285 ms
64 bytes from 1.1.1.2: icmp_seq=2 ttl=255 time=0.282 ms
64 bytes from 1.1.1.2: icmp_seq=3 ttl=255 time=0.284 ms
64 bytes from 1.1.1.2: icmp_seq=4 ttl=255 time=0.291 ms
虽然ping工作正常，但启用后仍会在接口上看到UDLD错误
如下所示，无CoPP丢弃
N9K-2# show hardware internal cpu-mac inband active-fm traffic-to-sup
Active FM Module for traffic to sup:
0x00000016———————————————————————————Module 22.
N9K-2# show policy-map interface control-plane module 22 | inc dropp
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
dropped 0 bytes;
Spoiler
面向Sup的活动FM是模块22。要验证在命令下运行
module-30# show mvdxn internal port-status
Switch type: Marvell 98DXN41 - 4 port switch
Port Descr Enable Status ANeg Speed Mode InByte OutByte InPkts OutPkts
-- -------------------- ------ ------ ---- ----- ---- ---------- ---------- ---------- ----------
6 Local AXP CPU Yes UP No 2 6 781502852 1006219901 6868852 3506128
7 This SC BCM EOBC switch Yes UP No 2 6 654791960 430206276 1833465 3523170
8 Other SC BCM EOBC switch Yes DOWN No 2 6 72282 176 3 2
9 This SC EPC switch Yes UP No 2 6 351355874 351309506 1672662 3345683
Switch type: Marvell 98DXN11 - 10 port switch
Port Descr Enable Status ANeg Speed Mode InByte OutByte InPkts OutPkts
-- -------------------- ------ ------ ---- ----- ---- ---------- ---------- ---------- ----------
0 FM6 EPC switch Yes DOWN No 2 6 0 0 0 0
1 FM5 EPC switch Yes DOWN No 2 6 0 0 0 0
2 SUP ALT EPC Yes DOWN No 2 6 0 0 0 0
3 SUP PRI EPC Yes DOWN No 2 6 0 0 0 0
4 FM4 EPC switch Yes DOWN No 2 6 0 0 0 0
5 FM3 EPC switch Yes DOWN No 2 6 0 0 0 0
6 FM2 EPC switch Yes DOWN No 2 6 0 0 0 0
7 FM1 EPC switch Yes DOWN No 2 6 0 0 0 0
8 Other SC EPC switch Yes UP No 2 6 351356399 351310095 1672664 3345687
9 Local SC 4-port switch Yes UP No 2 6 351310031 351356399 3345688 1672664
Rule Rule_name Match_ctr Pol_en Pol_idx inProfileBytes outOfProfileBytes
---- -------------------- -------------------- ------ ------- -------------------- --------------------
解决方案
TCAM值设置为0会导致线路卡中所有控制流量丢弃。
将TCAM值更改为默认udld后，系统会启动并解析arp
添加到N9K-2的配置可解决此问题
配置更改后需要重新加载
N9K-2(config)# hardware access-list tcam region ing-sup 512
Warning: Please reload all linecards for the configuration to take effect
N9K-2(config)# hardware access-list tcam region ing-racl 1536
Warning: Please reload all linecards for the configuration to take effect
N9K-2(config)# hardware access-list tcam region ing-l2 ing-l2-qos ing-l2-span-filter
N9K-2(config)# hardware access-list tcam region ing-l2-qos 256
Warning: Please reload all linecards for the configuration to take effect
N9K-2(config)# hardware access-list tcam region ing-l3-vlan-qos 512
Warning: Please reload all linecards for the configuration to take effect
N9K-2(config)# hardware access-list tcam region ing-l2 ing-l2-qos ing-l2-span-filter
N9K-2(config)# hardware access-list tcam region ing-l2-span-filter 256
N9K-2(config)# hardware access-list tcam region ing-l3-span-filter 256
N9K-2(config)# hardware access-list tcam region span 512
Warning: Please reload all linecards for the configuration to take effect
N9K-2(config)# hardware access-list tcam region egr-racl 1792
Warning: Please reload all linecards for the configuration to take effect
N9K-2(config)# show run | grep tcam
hardware access-list tcam region ing-redirect 0
N9K-2(config)# hardware access-list tcam region ing-redirect 256
Warning: Please reload all linecards for the configuration to take effect
有用的命令
显示硬件访问列表tcam区域
show run | inc TCAM" — 无输出表示TCAM设置为默认设置。
反馈