Introduction
本文描述如何用嵌入式服务处理器10 (ESP10)排除和验证在聚合服务路由器1000 (ASR 1000)的日志消息HAL_PKTMEM-2-OUT_OF_RESOURCES故障。
Prerequisites
Requirements
Cisco 建议您了解以下主题:
Components Used
本文档中的信息基于以下软件版本:
The information in this document was created from the devices in a specific lab environment.All of the devices used in this document started with a cleared (default) configuration.If your network is live, make sure that you understand the potential impact of any command.
背景信息
当传输在设备里面时, PAK_PRIORITY是机制设备使用指定信息包的处理。通常是被标记的PAK_PRIORITY例如的信息包是控制协议信息包, :RIP、OSPF、EIGRP、ISIS、PPP、HDLC等等。
症状
通常此问题提交自己作为的路由器能转发数据流在某些界面外面。
这在日志缓冲器记录能被看到:
.Apr 8 18:56:40.808 GMT: %IOSXE-2-PLATFORM: F0: cpp_cp: QFP:00 Thread:069 TS:00006374345833820173 %HAL_PKTMEM-2-OUT_OF_RESOURCES:
.Apr 8 18:57:41.222 GMT: %IOSXE-2-PLATFORM: F0: cpp_cp: QFP:00 Thread:047 TS:00006374406093385973 %HAL_PKTMEM-2-OUT_OF_RESOURCES:
.Apr 8 18:58:43.662 GMT: %IOSXE-2-PLATFORM: F0: cpp_cp: QFP:00 Thread:009 TS:00006374468373382518 %HAL_PKTMEM-2-OUT_OF_RESOURCES
此日志意味着设备用尽了信息包缓冲,由于pak_priority数据流的超额预订。
ASR 1k不会丢弃PAK_PRIORITY信息包,使容易对他们填满不允许的缓冲区其他数据流经历。
Troubleshoot
您通过检查接口默认值开始接口的队列与问题:
R1#sh platf hard qfp active infrastructure bqs queue output default interface GigabitEthernet0/0/4
Interface: GigabitEthernet0/0/4 QFP: 0.0 if_h: 19 Num Queues/Schedules: 1
Queue specifics:
Index 0 (Queue ID:0x8a, Name: GigabitEthernet0/0/4)
Software Control Info:
(cache) queue id: 0x0000008a, wred: 0x8b670082, qlimit (bytes): 3281312
parent_sid: 0x278, debug_name: GigabitEthernet0/0/4
sw_flags: 0x08000091, sw_state: 0x00000801, port_uidb: 0
orig_min : 0 , min: 105000000
min_qos : 0 , min_dflt: 0
orig_max : 0 , max: 0
max_qos : 0 , max_dflt: 0
share : 1
plevel : 0, priority: 0
defer_obj_refcnt: 0
Statistics:
tail drops (bytes): 0 , (packets): 0
total enqs (bytes): 969986824 , (packets): 6713421
queue_depth (bytes): 262736736
您能看到队列限制是3281312,但是队列深度是262736736。相当数量信息包超过。当pak_priority信息包在接口时,以高速率到达这能只发生。
然后请检查在QFP (Quantum流处理器)的丢包ASR 1k,您注意有BQSOOR (在资源外面的缓冲的排队和安排)丢包增加。BQS是缓冲,排队,并且安排ASIC,这意味着设备不能缓冲是到达由于饱和的它的某些信息包。
R1#show plat hardw qfp active statistics drop all | e _0_
-------------------------------------------------------------------------
Global Drop Stats Packets Octets
-------------------------------------------------------------------------
BqsOor 62918 8700111
R1#show plat hardw qfp active statistics drop all | e _0_
-------------------------------------------------------------------------
Global Drop Stats Packets Octets
-------------------------------------------------------------------------
BqsOor 62923 8700966
R1#show plat hardw qfp active statistics drop all | e _0_
-------------------------------------------------------------------------
Global Drop Stats Packets Octets
-------------------------------------------------------------------------
BqsOor 62942 8703894
现在请检查bqs信息包利用率发现使用的缓冲区的百分比。
R1#show platform hardware qfp act bqs 0 packet utilization
Packet buffer memory utilization details:
Total: 256.00 MB
Used : 253.44 MB
Free : 2620.00 KB
Threshold Values:
Out of Memory (OOM) : 255.96 MB, Status: False
Vital (> 98%) : 253.44 MB, Status: True
Out of Resource (OOR) : 217.60 MB, Status: True
Utilization: 99 %
利用率是99%,因此这确认设备用尽缓冲区的资源。
您当前需要位于哪个组缓冲区是信息包在。
有4个选项:
•通过MQC被创建的QoS队列运行命令“Show policy-map int|incl队列深度|限制”
•输出接口的默认队列运行命令“Sho制地图硬qfp操作inf bqs que def全部|incl queue_depth”
•回收用于基础设施的队列运行命令“Sho制地图硬afp操作inf队列回收所有的bqs|incl queue_depth”
•IPC (进程间通信协议)队列运行命令“Sho制地图硬afp操作bqs排队ipc的inf|incl queue_depth”
R1#show platform hardware qfp act inf bqs que out def all | i queue_de
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 262736736
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
R1#show platform hardware qfp act inf bqs que out recy all | i queue_de
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
queue_depth (packets): 0
R1#show platform hardware qfp act inf bqs que out ipc | i queue_de
queue_depth (bytes): 0
queue_depth (bytes): 0
queue_depth (bytes): 0
您看到信息包在默认队列。
通常此问题可以与也许发送PAK_PRIORITY被标记的信息包或DDoS攻击的风暴产生关联明显的,虽然PAK_PRIORITY为了中断信息包转发,为了此CoPP (Contro平面策略)也许是需要的丢弃不来自一个有效源的信息包。
在,在接口情况下,也会看到暂停输入增加Flow-control能也导致此。
R1#show int gi0/0/4
GigabitEthernet0/0/4 is up, line protocol is up
Hardware is SPA-10X1GE-V2, address is 74de.eeee.cccc (bia 74de.eeee.cccc)
Description: inmumpt005rtwn01-G0/2 Airtel 7779861 300Mbps/1Gbps
Internet address is 10.1.1.1/30
MTU 9000 bytes, BW 300000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
Full Duplex, 1000Mbps, link type is force-up, media type is LX
output flow-control is on, input flow-control is on
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:02, output 00:00:01, output hang never
Last clearing of "show interface" counters 8w5d
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 11
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
30 second input rate 0 bits/sec, 0 packets/sec
30 second output rate 0 bits/sec, 0 packets/sec
16653945560 packets input, 6397725725851 bytes, 91 no buffer
Received 339 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
52 input errors, 52 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 2095792 multicast, 166107198 pause input
12240362564 packets output, 3785983938723 bytes, 0 underruns