Contents

Introduction

Background Information

Issue Summary

Analysis of the Issue

Introduction

This document describes the Label Distribution Protocol (LDP) Neighbor issue due to low memory on ASR9000 routers.

Background Information

LDP is a label distribution protocol that helps generate and share the label information for IGP routes. But before exchanging the label, the router first forms the LDP neighborship. As you know, LDP works on UDP and TCP protocols. For neighbour discovery, LDP uses UDP protocol and to create the neighborship it uses TCP protocol.

The router must have a good amount of memory and CPU to make a TCP connection.

Issue Summary

As evident, you can ping and traceroute.

RP/0/RSP0/CPU0:R1#ping 192.168.12.6 so l14

Thu Jun 27 07:01:31.192 UTC

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.12.6 timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/3 ms

RP/0/RSP0/CPU0:R1# traceroute 192.168.12.6 so l14 numeric

Thu Jun 27 07:01:56.748 UTC

Type escape sequence to abort.

Tracing the route to 192.168.12.6

1  192.168.99.220 [MPLS: Labels 26664/75671 Exp 0] 3 msec  2 msec  2 msec

2  192.168.96.8     [MPLS: Label 75671 Exp 0] 1 msec  1 msec  1 msec

3  192.168.62.151 [MPLS: Label 24201 Exp 0] 2 msec  2 msec  2 msec

4  192.168.12.6     2 msec  2 msec  2 msec

All ping and traceroute are working, but no LDP membership.

RP/0/RSP0/CPU0:R1#  show mpls ldp discovery 192.168.12.6 detail

Thu Jun 27 07:05:43.503 UTC

Local LDP Identifier: 192.168.248.84:0

Discovery Sources:

Targeted Hellos:

192.168.248.84 -> 192.168.12.6 (active), xmit/recv

Hello interval: 5 sec (due in 978 msec)

Quick-start: Enabled

LDP Id:192.168.248.84:0

Hold time: 45 sec (local:90 sec, peer:45 sec)

(expiring in 41.3 sec)

Established: Jun 26 12:02:16.216 (18:58:47 ago)

Session bringup fail reason:

waiting for tcp incall

You can see the TCP Fail reason.

Analysis of the Issue

When Show mpls ldp trace error reverse | in 192.168.12.6  command is checked, it is noticed that there are no LDP neighbors due to low memory.

Jun 27 07:08:29.742 mpls/ldp/err 0/RSP0/CPU0 t1  [ERR][MISC]:8201: VRF(0x60000000):

ldp_nbr_ok_to_connect: Call from 192.168.12.6 rejected - state=2, reason=''MPLS_LDP'

detected the 'resource not available' condition 'A low memory condition prevents new

LDP sessions'

Also, it is noticed that there is minor memory on RSP0 but RSP1 is normal.

RP/0/RSP0/CPU0:R1#show watchdog memory-state

Thu Jun 27 06:16:13.033 UTC

Memory information:

Physical Memory: 6144     MB

Free Memory:      554.988 MB

Memory State:          Minor

RP/0/RSP0/CPU0:R1#show watchdog memory-state location 0/RSP1/CPU0

Thu Jun 27 06:17:06.110 UTC

Memory information:

Physical Memory: 6144     MB

Free Memory:     1208.164 MB

Memory State:         Normal

Because this issue is related to memory, you can verify the memory consumption of different components of the systems and ltrace utilizing the highest memory, even when compared to other RSPs and devices. It is found that RSP0 is utilizes the highest amount of memory for ltrace.

RP/0/RSP0/CPU0:R1#show shmem summary location 0/rSP0/CPU0

Thu Jun 27 07:24:48.601 UTC

Total Shared memory: 2345M

ShmWin: 349M

Image: 71M

LTrace: 690M

AIPC: 73M

SLD: 3M

SubDB: 528K

CERRNO: 148K

GSP-CBP: 165M

EEM: 0

XOS: 15M

CHKPT: 10M

CDM: 9M

XIPC: 4M

DLL: 64K

SysLog: 10M

Miscellaneous: 940M

LTrace usage details:

Used: 690M, Max: 3366M

Current: default(dynamic)

Configured: dynamic with scale-factor: 16 (changes take effect after reload)

You can also see that Ltrace is utilizing a high amount of memory.

Workaround:

1. Reload the standby RP.

2. Wait for it to sync. Confirm that on  sh shmem summary loc 0/RSP1/CPU0, the current profile is scale-factor 16.

3. Switchover.

4. Confirm that memory is normal. Confirm that LDP is back up.

5. Confirm that scale-factor 16 is applied to RSP0 with  sh shmem summary loc 0/RSP0/CPU0.

Collect Logs:

show mpls ldp neighbor

show mpls ldp neighbor detail

show mpls ldp discovery 192.168.12.6 detail

show watchdog memory-state

show watchdog memory-state location 0/RSP1/CPU0

show shmem summ loc 0/RSP1/CPU0

show shmem summ loc 0/RSP0/CPU0

show redundancy

show tech-support mpls ldp

Note: Check the memory usage on another router with the same configuration.