Introduction
This document helps you understand average memory utilization on Nexus 3000 device.
Contributed by Narottama Sahoo, Santhosh Shankar, Cisco TAC Engineers
Background
The memory utilization in nexus 3000 likely vary based on the version running on the device.
In newer versions due to new feature addition the memory usage will likely be high when compared to older version
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
Components Used
The information in this document is based on these software and hardware versions:
- Nexus 3000 series platform
- Code version 6.x and 7.x
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Troubleshooting tips
Below are some sample outputs from my lab (minimally configured N3548) demonstrates baseline memory utilization on the platform.
N3K-3548.42# sh system resources | egrep "used"
Memory usage: 4117860K total, 2250220K used, 1867640K free
N3K-3548.42# sh system internal kernel memory global | be NAME | cut -c 1-42
NAME |TOTAL |
mtc_usd | 462744 231212 223972 |
fwm | 277132 97788 84907 |
afm | 218180 53060 40829 |
netstack | 380252 45592 38952 |
clis | 219208 41924 32060 |
m2rib | 182944 35812 31502 |
--More--
All running processes will consume some memory for text/heap/stack/data/shared-lib. The above CLI reports proportional memory utilization per process (takes into account memory pages shared between processes). By adding the proportional memory usage of all processes, we can get a representation of userspace processes memory usage. On lab switch, the baseline utilization sum for all running processes was approximately 42% of overall utilization (943276 kB).
N3K-3548.42# sh system internal kernel meminfo | egrep "ShmFS|^Cached"
Cached: 1174236 kB
ShmFS: 1169840 kB
NX-OS binaries / configurations / etc are extracted from kickstart/system images during boot. These reside in temporary file systems not backed by persistent storage and are accounted for by linux as ShmFS.
In addition to ShmFS usage, linux will temporarily cache file contents being read/referenced by processes. The Cached counter is a sum of ShmFS and cached files. From the above output, you can see that Cached usage accounts for approximately 52% of overall utilization (1174236 kB).
N3K-3548.42# sh system internal kernel meminfo | egrep "Slab"
Slab: 85376 kB
Finally, the kernel also uses some memory for internal data structures. Slab accounts for kernel and kernel_module data structures. From the above output, Slab usage accounts for approximately 4% of overall utilization (85376 kB).
There is some additional memory utilization that is not broken out by the linux kernel, but it only accounts for ~50mb. For more information, please refer linux architecture documentation.
As a proportion of baseline utilization, temporary file systems are the largest memory consumer. This is expected behavior and common among all NX-OS based platforms.
For NX-OS running 5.X average utlilization will be around 40%
For NX-OS running 6.X average utilization ranges between 60-70%
For NX-OS running 7.X average utlilization will bearound 80%
Therefore, the increased memory usage is likely a factor of new features in each release and a larger base footprint for the OS.
Sample output collected without any/default config on the device
Version: 6.0(2)U3(7)
switch# show system resources module all
CPU Resources:
-----------------------------------------------------------
CPU utilization: Module 5 seconds 1 minute 5 minutes
-----------------------------------------------------------
1 7 6 9
-----------------------------------------------------------
Processor memory: Module Total(KB) Free(KB) % Used
-----------------------------------------------------------
1 3785276 1450540 61 (Usage when running 6.x version)
Version:7.0.3.I2.2a
switch# show system resources module all
CPU Resources:
----------------------------------------------------------
CPU utilization: Module 5 seconds 1 minute 5 minutes
-----------------------------------------------------------
1 6 8 8
-----------------------------------------------------------
Processor memory: Module Total(KB) Free(KB) % Used
-----------------------------------------------------------
1 3903900 795600 79 (Usage when running 7.x version)
Recommendations:
Always refer Release notes to understand the system requirement for every software. The hardware and software requirement espescially the memory vary while upgrading to a newer version of code.
Useful links:
Release notes
Feedback