Contents

Introduction

Background Information

Troubleshoot

Identify Application Responsible for Leak

Decode Allocator PC

Investigate MallocLite Memory Statistics

Disable MallocLite

Introduction

This document describes how to troubleshoot MallocLite memory leaks on Cisco IOS®software platforms.

It also specifies the information you should gather before you open a Cisco Technical Assistance Center (TAC) case or reload the device. Collect the outputs mentioned in this document, and attach them to the TAC case in order to help expedite problem resolution.

Background Information

MallocLite is used by the memory manager in order to allocate small, fixed size pieces of memory, known as chunks, for allocations less than or equal to 128 bytes. Small memory allocations do not have the overhead of a block header for every allocation. This feature is supported for processor memory pools only.

Every memory block header takes about 48 bytes of memory, and the smallest block takes about 24 bytes. With a traditional approach in Cisco IOS software for each allocation, you would consume at least 72 (48 + 24) bytes of memory, even if you need to allocate only 8 bytes of actual data.

With MallocLite, this overhead can be reduced by the use of chunks. There is still some overhead, because the chunks have to be managed. However, since the chunks are fixed size, they are managed in a different way than blocks, and the overhead is less.

It is the responsibility of the applications that use the MallocLite memory to free it properly. MallocLite masks the user of the memory.

Troubleshoot

Note: The Cisco CLI Analyzer (registered customers only) supports certain show commands. Use the Cisco CLI Analyzer in order to view an analysis of show command output.

Identify Application Responsible for Leak

It is usually difficult to identify an existing bug if you search only by the malloclite keyword.

This example shows that the *MallocLite* process is holding an abnormal amount of memory:

#show processes memory sorted

Processor Pool Total:  1614282720  Used:  1544726580  Free:   69556140

I/O Pool Total:  313524224   Used:  115564032   Free:  197960192

PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

0   0          0          0  1476043512         0          0 *MallocLite*

You need to identify the exact application that is responsible for leak. Three possible identification methods are:

- Decode allocator PC.

- Investigate MallocLite memory statistics.

- Disable MallocLite.

Decode Allocator PC

Even with MallocLite turned on, you can usually see what function asked for the memory. Output from the show memory allocating-process totals command might show different PC values even though the name reported is MallocLite:

#show memory allocating-process totals

<snip>

Allocator PC Summary for: Processor

Displayed first 2048 Allocator PCs only

PC          Total   Count  Name

0x620BE3C4   42807572     594  MallocLite

0x620ADDD4   13597308     193  MallocLite

0x60738BB0    8909824     122  MallocLite

0x620AE0E0    2060716      31  MallocLite

0x620AE10C    1982780      30  MallocLite

A Cisco TAC engineer can decode the PC values from the top of the list (with the highest total). This helps identify the application that has the memory leak.

Investigate MallocLite Memory Statistics

Among the enhancements added in Cisco IOS software release 15.1T was a new CLI that displays the summary of MallocLite memory allocated by each PC. The show memory lite-chunks command can help you identify applications that are using a large amount of MallocLite blocks.

show memory lite-chunks { statistics | totals } { summary { pool | { all | pool } } }

Refer to the command reference for details of the show memory lite-chunks command.

CLI   : show memory lite-chunks totals

DESC  : Summary of all pools, based on alloc pc.

This cli can be used to find the alloc_pc which is using large amount memory

allocated from all mlite pools

CLI   : show memory lite-chunks statistics

DESC  : Displays number of allocated & free mlite chunks

CLI   : show memory lite-chunks summary pool pool

DESC  : Show summary of particular mlite pool

This cli can be used to find the alloc_pc which is using large amount of memory

in individual mlite pool

CLI   : show memory lite-chunks summary pool all

DESC  : Show individual summary of all mlite pools

CLI   : show memory lite-chunks pool pool

DESC  : Show All chunk elements in the specified pool

CLI   : show memory lite-chunks pool all

DESC  : show all chunk elements in all mlite pools

Examples of output from this command include:

#show memory lite-chunks ?

pool        Malloc lite pool

statistics  Malloc lite statistics

summary     Malloc Lite summary

totals      Malloc Lite Allocating totals

#show memory lite-chunks statistics

Pool       Inuse       Free

8-Bytes         140       1904

20-Bytes         173       1313

44-Bytes         171        791

68-Bytes          24        687

96-Bytes          26        519

128-Bytes          20        410

#show memory lite-chunks totals

PC         Total       Count

26067AE0          2112          33

2269E68C          1932          29

2269FACC          1664          29

2269F964          1664          26

2269FA9C          1580          29

26067FB4          1360          34

23CD2A0C          1036           7

#show memory lite-chunks pool ?

128-Bytes  128 bytes pool

20-Bytes   20 bytes pool

44-Bytes   44 bytes pool

68-Bytes   68 bytes pool

8-Bytes    8 bytes pool

96-Bytes   96 bytes pool

all        all pools

#show memory lite-chunks summary pool 8

8 bytes pool

PC         Total       Count

2269FB10           812          29

23612084           700          25

2269F9F8           700          25

2269F9EC           700          25

Again, the TAC engineer can decode PC values with the highest total and identify the application that is leaking the memory.

Disable MallocLite

The MallocLite feature is enabled by default. In order to investigate the MallocLite leak, you can disable MallocLite:

(config)#no memory lite

The leaked memory will still be under MallocLite until the next reload; however, you can start to monitor further leaks with the show processes memory sorted and show memory allocating-process totals commands. The leaks will now appear under the real process.

If the device runs very low on memory, you must save the configuration and reload the device in order to release the memory:

#wr

#reload

The memory might deplete again over time, so use the show processes memory sorted and show memory allocating-process totals commands in order to monitor memory usage from that point forward.

Note: If you effectively disable MallocLite with the no memory lite command and reload the device, output from the show memory lite-chunks command will be empty.

Refer to the command reference for details of the memory lite command.