排除故障ASR1000-ESP10被终止的接口转发流量由于"HAL_PKTMEM-2-OUT_OF_RESOURCES"

简介

本文描述如何排除故障和验证在聚合服务路由器1000 (ASR 1000)的日志消息HAL_PKTMEM-2-OUT_OF_RESOURCES用嵌入式服务处理器10 (ESP10)。

先决条件

要求

Cisco 建议您了解以下主题：

• ASR1k信息包转发

使用的组件

本文档中的信息基于以下软件版本：

• ASR1k 15.1(3)S2以上

本文档中的信息都是基于特定实验室环境中的设备编写的。本文档中使用的所有设备最初均采用原始（默认）配置。如果您使用的是真实网络，请确保您已经了解所有命令的潜在影响。

背景信息

当传送在设备里面时， PAK_PRIORITY是机制设备使用指定数据包的处理。通常是被标记的PAK_PRIORITY例如的数据包是控制协议数据包， ：RIP、OSPF、EIGRP、ISIS、PPP、HDLC等等。

症状

通常此问题提交自己作为的路由器能转发流量在某些界面外面。

这在日志缓冲区记录能被看到：

.Apr 8 18:56:40.808 GMT: %IOSXE-2-PLATFORM: F0: cpp_cp: QFP:00 Thread:069 TS:00006374345833820173 %HAL_PKTMEM-2-OUT_OF_RESOURCES:
.Apr 8 18:57:41.222 GMT: %IOSXE-2-PLATFORM: F0: cpp_cp: QFP:00 Thread:047 TS:00006374406093385973 %HAL_PKTMEM-2-OUT_OF_RESOURCES:
.Apr 8 18:58:43.662 GMT: %IOSXE-2-PLATFORM: F0: cpp_cp: QFP:00 Thread:009 TS:00006374468373382518 %HAL_PKTMEM-2-OUT_OF_RESOURCES

此日志意味着设备用尽了数据包缓冲，由于pak_priority流量的超额预订。
ASR 1k不会丢弃PAK_PRIORITY数据包，进行它容易为了他们能填满不允许的缓冲区其他流量经历。

故障排除

您通过检查接口默认值开始接口的队列与问题：

R1#sh platf hard qfp active infrastructure bqs queue output default interface GigabitEthernet0/0/4
Interface: GigabitEthernet0/0/4 QFP: 0.0 if_h: 19 Num Queues/Schedules: 1
Queue specifics:
Index 0 (Queue ID:0x8a, Name: GigabitEthernet0/0/4)
Software Control Info:
(cache) queue id: 0x0000008a, wred: 0x8b670082, qlimit (bytes): 3281312
parent_sid: 0x278, debug_name: GigabitEthernet0/0/4
sw_flags: 0x08000091, sw_state: 0x00000801, port_uidb: 0
orig_min : 0 , min: 105000000
min_qos : 0 , min_dflt: 0
orig_max : 0 , max: 0
max_qos : 0 , max_dflt: 0
share : 1
plevel : 0, priority: 0
defer_obj_refcnt: 0
Statistics:
tail drops (bytes): 0 , (packets): 0
total enqs (bytes): 969986824 , (packets): 6713421
queue_depth (bytes): 262736736

您能看到队列限制是3281312，但是队列深度是262736736。相当数量数据包超过。当pak_priority数据包在接口时，以高速率到达这能只发生。

然后请检查在QFP (Quantum流处理器)的丢包ASR 1k，您注意有BQSOOR (缓冲队列和日程安排在资源外面)丢包增加。BQS是缓冲，队列，并且日程安排ASIC，这含义设备不能缓冲是到达由于饱和的它的某些数据包。

R1#show plat hardw qfp active statistics drop all | e _0_
-------------------------------------------------------------------------
Global Drop Stats Packets Octets
-------------------------------------------------------------------------
BqsOor                    62918 8700111


R1#show plat hardw qfp active statistics drop all | e _0_
-------------------------------------------------------------------------
Global Drop Stats Packets Octets
-------------------------------------------------------------------------
BqsOor                   62923 8700966


R1#show plat hardw qfp active statistics drop all | e _0_
-------------------------------------------------------------------------
Global Drop Stats Packets Octets
-------------------------------------------------------------------------
BqsOor                   62942 8703894

现在请检查bqs数据包利用率发现使用的缓冲区的百分比。

R1#show platform hardware qfp act bqs 0 packet utilization
Packet buffer memory utilization details:
Total: 256.00 MB
Used : 253.44 MB
Free : 2620.00 KB
Threshold Values:
Out of Memory (OOM) : 255.96 MB, Status: False
Vital (> 98%) : 253.44 MB, Status: True
Out of Resource (OOR) : 217.60 MB, Status: True
Utilization: 99 %

利用率是99%，因此这确认设备用尽缓冲区的资源。

您在缓冲区的哪组中当前需要查找是数据包。

有4个选项：

•通过MQC创建的QoS队列运行命令“Show policy-map int|incl队列深度|限制”
•输出接口的默认队列运行命令“Sho制地图硬qfp操作inf bqs que def全部|incl queue_depth”
•回收用于基础设施的队列运行命令“Sho制地图硬afp操作inf队列回收所有的bqs|incl queue_depth”
•IPC (进程间通信协议)队列运行命令“Sho制地图硬afp操作bqs排队ipc的inf|incl queue_depth”

R1#show platform hardware qfp act inf bqs que out def all | i queue_de
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 262736736
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0

R1#show platform hardware qfp act inf bqs que out recy all | i queue_de
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
R1#show platform hardware qfp act inf bqs que out ipc | i queue_de
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0

您看到数据包在默认队列。

通常此问题可以关联与的PAK_PRIORITY标记的信息包或DDoS攻击风暴也许发送明显的，虽然PAK_PRIORITY为了中断信息包转发，为了此CoPP (Contro平面管制)也许是需要的丢弃不来自有效的来源的数据包。
在，在接口情况下，也会看到暂停输入增加Flow-control能也导致此。

R1#show int gi0/0/4
GigabitEthernet0/0/4 is up, line protocol is up
Hardware is SPA-10X1GE-V2, address is 74de.eeee.cccc (bia 74de.eeee.cccc)
Description: inmumpt005rtwn01-G0/2 Airtel 7779861 300Mbps/1Gbps
Internet address is 10.1.1.1/30
MTU 9000 bytes, BW 300000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
Full Duplex, 1000Mbps, link type is force-up, media type is LX
output flow-control is on, input flow-control is on
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:02, output 00:00:01, output hang never
Last clearing of "show interface" counters 8w5d
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 11
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 0 bits/sec, 0 packets/sec
16653945560 packets input, 6397725725851 bytes, 91 no buffer
Received 339 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
52 input errors, 52 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 2095792 multicast, 166107198 pause input
12240362564 packets output, 3785983938723 bytes, 0 underruns