Memory Troubleshooting Guide for Cisco 4000 Series ISRs

DRAM for Cisco 4400 Series ISRs

Cisco 4400 series platforms use 1333MHz DIMMs for memory. The platforms have three DIMM slots, two for main system memory (labeled CP1 and CP2 on the motherboard where CP is for Control Plane), and one for data plane memory (labeled FFP on the motherboard). Both DIMMs in the CP slots must be populated with the same memory size as these systems use interleaving for maximizing performance; unmatched DIMMs are not tested or supported by Cisco. Table 1-1 indicates the DIMM size options available.

Table 1-2 indicates the supported memory options on Cisco 4400 Series ISRs.

DRAM for Cisco 4300 Series ISRs

Cisco 4300 Series platforms use 1600MHz DIMMs for memory. The platforms have one or two DIMM slots for main system memory. For platforms with two DIMM slots, both slots must be populated with the same size memory as these systems use interleaving for maximizing performance; unmatched DIMMs are not tested or supported by Cisco. Table 1-3 indicates the DIMM size options available on Cisco 4300 Series ISRs.

Table 1-4 indicates the supported memory options on Cisco 4300 series ISRs.

Note Cisco ISR 4351 and 4331 devices can operate with only one DIMM (minimum 4 GB), installed in slot 0. Older ROMMON versions may flash a warning message for both DIMMs to be installed before boot. To override this warning, upgrade to ROMMON 16.7(3r) or higher.

System Memory Usage in Cisco IOS Daemon

Cisco IOSd runs as its own process and it allocates memory from the system. It uses its own memory manager for features running within Cisco IOSd. In general, Cisco IOSd is allowed to use a maximum of approximately ½ of the total system memory. However, it uses less system memory depending on the configuration. IOS will consume system memory as needed to satisfy memory requests for features within Cisco IOSd. To do this, it uses a feature referred to as Cisco IOSd Dynamic Heap (dHeap). With dHeap, Cisoc IOS will request memory from the operating system based on demand as indicated by the dHeap output indicated in the show memory command.

To find out the total amount of system memory that is in use by Cisco IOSd, use the Cisco IOS XE system memory commands.

To get more detailed information on Cisco IOS process memory use, use the show process memory command. Note that the following commands display only information that is within Cisco IOSd, and it does not indicate memory use in the Cisco IOS XE operating system.

The show processes command displays information about the active processes. Use the show processes memory command to show the amount of memory used within Cisco IOSd.

System Memory Usage in Cisco IOS XE Software

To inspect the system memory usage in Cisco IOS XE software, the following show commands can be used. As mentioned earlier, IOS-XE relies on the Linux kernel which provides the system memory management. For more information on Linux memory management, it is recommended to learn how Linux memory management works. Also, you can refer the book Understanding how the Linux Virtual Memory Manager Works available on www.kernel.org. To see current system memory usage on Cisco IOS XE, use the show platform software status control-processor brief command.

Memory used for the above command includes filesystem cache, much of which can be used when the system memory requires it. In general, used memory does not reflect how much more memory can be used by the new feature configuration; a more accurate estimate from this command comes from using the committed memory (in this example, about 12% of the system memory is in use).

The following command displays a high level view of the system utilization (which is available from Cisco IOS XE release 3.14 or later).

Memory Usage for Display Process

To display system memory utilization for individual processing running in Cisco IOS XE operating system, use the show processes memory platform sorted command which is available from Cisco IOS XE release 3.14 or later. You can sort the processes memory using the Resident Set Size (RSS). For previous releases, the show platform software process memory R0 all sorted command is also available to obtain per-process memory use in the system.

For example, in the above command, you can see that the amount of system memory used by Cisco IOSd is 4979144kB as represented in the RSS (Resident Set Size) for the linux_iosd process. The Total memory column refers to the VSZ (Virtual Set Size) for each process.

Memory Usage for Monitoring Process

To interactively monitor system memory usage for each process running in Cisco IOS XE, use the monitor platform software process rp active command. After the screen appears, you can type shift + M to sort displayed processes with memory usage. RES (same as RSS) indicates the non-swapped physical memory and SHR indicates the amount of shared memory used by a process.

Memory Usage for Data Plane

To display the information about the memory usage of the data plane or QFP, use the show platform hardware qfp active infrastructure exmem statistics command.

For the Cisco 4400 Series ISR platforms, the data plane DRAM is located on a 2GB DIMM, which is physically separate from the system DRAM memory as describe above. Most of the data plane DRAM is used for system purposes. About 750 MB is used for buffering and another 750 MB is used to store the microcode that runs the data plane packet forwarding process. The remaining 512 MB is used for EXMEM, which is the dynamic memory that allows the actual packet handling and for features to scale.

Note The displayed memory usage information of the data plane on Cisco 4400 Series ISR platforms differs depending on the IOS version that runs on the platform.

The display information of Cisco 4400 Series ISR platforms that are running IOS versions earlier than 16.x reflects the total 2GB DRAM as shown below.

Example: Cisco 4451 ISR with IOS version 3.16

When you check the DRAM usage in this command, 75% of the DRAM memory is used. However that includes the 750 MB for fixed part of microcode and another 750 MB for buffering. The remaining is the dynamic EXMEM part, which is used for features. In this case, there is 499 MB EXMEM available out of the total 512 MB. So, this router’s data plane memory utilization is only about 2%.

For Cisco 4400 Series ISR platforms that are running IOS versions 16.x and later, this command reflects only the actual EXMEM usage. This is the same for Cisco 4300 Series ISR, in which the data plane is not located on a separate DIMM. However, Cisco 4300 Series ISR uses a portion of the overall system memory.

Example: Cisco 4451 ISR with IOS Version 16.5.1

Troubleshooting Commands

The following commands helps to troubleshoot the resource utilizatio:

• show platform hardware qfp active datapath utilization summary
• show platform hardware resource datapath

Show Platform Hardware QFP Active Datapath Utilization Summary

This command reveals the load on the Cisco 4300 and 4200 Series ISRS. The show platform hardware qfp active datapath utilization command includes the information on IO/crypto core utilization. If the row Processing: Load has high values, it indicates the utilization is high and needs further troubleshoot to see if it is caused due to features configured on the router or high traffic rate.

If the system indicates that the qfp memory utilization is high, the following message is displayed:

Also, when the IOSd reaches a low memory level, the following message is displayed:

*Jan 18 07:23:39.090: %SYS-4-FREEMEMLOW: Free Memory has dropped below low watermark. Pool: Processor Free: 235388376 Threshold: 394129408 Top Allocator Name: BGP net chunk, PC: iosd_shr_m_uk9_ROUTING_crb:7FF3B55F3000+907D18, Size: 212582048, Count: 3239 Largest block: 426644

*Jan 18 07:23:39.090: %SYS-4-FREEMEMLOW: Free Memory has dropped below low watermark. Pool: Processor Free: 235388376 Threshold: 394129408 Top Allocator Name: BGP net chunk, PC: iosd_shr_m_uk9_ROUTING_crb:7FF3B55F3000+907D18, Size: 212582048, Count: 3239 Largest block: 426644

The memory consumption details are saved in the bootflash memory.

Related Documents

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

© 2020 Cisco Systems, Inc. All rights reserved.