Introduction
This document describes how to troubleshoot MallocLite memory leaks on Cisco IOS®software platforms.
It also specifies the information you should gather before you open a Cisco Technical Assistance Center (TAC) case or reload the device. Collect the outputs mentioned in this document, and attach them to the TAC case in order to help expedite problem resolution.
Background Information
MallocLite is used by the memory manager in order to allocate small, fixed size pieces of memory, known as chunks, for allocations less than or equal to 128 bytes. Small memory allocations do not have the overhead of a block header for every allocation. This feature is supported for processor memory pools only.
Every memory block header takes about 48 bytes of memory, and the smallest block takes about 24 bytes. With a traditional approach in Cisco IOS software for each allocation, you would consume at least 72 (48 + 24) bytes of memory, even if you need to allocate only 8 bytes of actual data.
With MallocLite, this overhead can be reduced by the use of chunks. There is still some overhead, because the chunks have to be managed. However, since the chunks are fixed size, they are managed in a different way than blocks, and the overhead is less.
It is the responsibility of the applications that use the MallocLite memory to free it properly. MallocLite masks the user of the memory.
Troubleshoot
Note: The Cisco CLI Analyzer (registered customers only) supports certain show commands. Use the Cisco CLI Analyzer in order to view an analysis of show command output.
Identify Application Responsible for Leak
It is usually difficult to identify an existing bug if you search only by the malloclite keyword.
This example shows that the *MallocLite* process is holding an abnormal amount of memory:
#show processes memory sorted
Processor Pool Total: 1614282720 Used: 1544726580 Free: 69556140
I/O Pool Total: 313524224 Used: 115564032 Free: 197960192
PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 0 0 1476043512 0 0 *MallocLite*
You need to identify the exact application that is responsible for leak. Three possible identification methods are:
- Decode allocator PC.
- Investigate MallocLite memory statistics.
- Disable MallocLite.
Decode Allocator PC
Even with MallocLite turned on, you can usually see what function asked for the memory. Output from the show memory allocating-process totals command might show different PC values even though the name reported is MallocLite:
#show memory allocating-process totals
<snip>
Allocator PC Summary for: Processor
Displayed first 2048 Allocator PCs only
PC Total Count Name
0x620BE3C4 42807572 594 MallocLite
0x620ADDD4 13597308 193 MallocLite
0x60738BB0 8909824 122 MallocLite
0x620AE0E0 2060716 31 MallocLite
0x620AE10C 1982780 30 MallocLite
A Cisco TAC engineer can decode the PC values from the top of the list (with the highest total). This helps identify the application that has the memory leak.
Investigate MallocLite Memory Statistics
Among the enhancements added in Cisco IOS software release 15.1T was a new CLI that displays the summary of MallocLite memory allocated by each PC. The show memory lite-chunks command can help you identify applications that are using a large amount of MallocLite blocks.
show memory lite-chunks { statistics | totals } { summary { pool | { all | pool } } }
Refer to the command reference for details of the show memory lite-chunks command.
CLI : show memory lite-chunks totals
DESC : Summary of all pools, based on alloc pc.
This cli can be used to find the alloc_pc which is using large amount memory
allocated from all mlite pools
CLI : show memory lite-chunks statistics
DESC : Displays number of allocated & free mlite chunks
CLI : show memory lite-chunks summary pool pool
DESC : Show summary of particular mlite pool
This cli can be used to find the alloc_pc which is using large amount of memory
in individual mlite pool
CLI : show memory lite-chunks summary pool all
DESC : Show individual summary of all mlite pools
CLI : show memory lite-chunks pool pool
DESC : Show All chunk elements in the specified pool
CLI : show memory lite-chunks pool all
DESC : show all chunk elements in all mlite pools
Examples of output from this command include:
#show memory lite-chunks ?
pool Malloc lite pool
statistics Malloc lite statistics
summary Malloc Lite summary
totals Malloc Lite Allocating totals
#show memory lite-chunks statistics
Pool Inuse Free
8-Bytes 140 1904
20-Bytes 173 1313
44-Bytes 171 791
68-Bytes 24 687
96-Bytes 26 519
128-Bytes 20 410
#show memory lite-chunks totals
PC Total Count
26067AE0 2112 33
2269E68C 1932 29
2269FACC 1664 29
2269F964 1664 26
2269FA9C 1580 29
26067FB4 1360 34
23CD2A0C 1036 7
#show memory lite-chunks pool ?
128-Bytes 128 bytes pool
20-Bytes 20 bytes pool
44-Bytes 44 bytes pool
68-Bytes 68 bytes pool
8-Bytes 8 bytes pool
96-Bytes 96 bytes pool
all all pools
#show memory lite-chunks summary pool 8
8 bytes pool
PC Total Count
2269FB10 812 29
23612084 700 25
2269F9F8 700 25
2269F9EC 700 25
Again, the TAC engineer can decode PC values with the highest total and identify the application that is leaking the memory.
Disable MallocLite
The MallocLite feature is enabled by default. In order to investigate the MallocLite leak, you can disable MallocLite:
(config)#no memory lite
The leaked memory will still be under MallocLite until the next reload; however, you can start to monitor further leaks with the show processes memory sorted and show memory allocating-process totals commands. The leaks will now appear under the real process.
If the device runs very low on memory, you must save the configuration and reload the device in order to release the memory:
#wr
#reload
The memory might deplete again over time, so use the show processes memory sorted and show memory allocating-process totals commands in order to monitor memory usage from that point forward.
Note: If you effectively disable MallocLite with the no memory lite command and reload the device, output from the show memory lite-chunks command will be empty.
Refer to the command reference for details of the memory lite command.
Feedback