The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
To use these commands in System Admin VM, you must be in a user group associated with appropriate command rules and data rules. If the user group assignment is preventing you from using a command, contact your AAA administrator for assistance.
To unobtrusively debug a live process or a live thread in a process, use the follow process command in XR EXEC modeSystem Admin EXEC mode.
follow process [ pid | location node-id ]
pid |
Follows the process with the process ID (PID) specified for the pid argument. |
location node-id |
Follows the target process on the designated node. The node-id argument is expressed in the rack/slot notation. |
Entering the follow process command without any keyword displays the stack information of the live processes with all the threads, heap memory usage, and register values.
XR EXEC mode
System Admin EXEC mode
Release |
Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
Use this command to unintrusively debug a live process or a live thread in a process. This command is particularly useful for debugging deadlock and livelock conditions, for examining the contents of a memory location or a variable in a process to determine the cause of a corruption issue, or in investigating issues where a thread is stuck spinning in a loop. A livelock condition is one that occurs when two or more processes continually change their state in response to changes in the other processes.
The following actions can be specified with this command:
Follow all live threads of a given process or a given thread of a process and print stack trace in a format similar to core dump output.
Display register values and status information of the target process.
Take a snapshot of the execution path of a thread asynchronously to investigate performance-related issues by specifying a high number of iterations with a zero delay.
sysadmin-vm:0_RP0# follow process 1 location 0/RP0
Location : 0/RP0
************************************************************
2013-09-20 01:57:30
Text address Size Library name
--------------------------------------------
00007f4b8a66c000 48 r-x-- libnss_files-2.12.so
00007f4b8a879000 1444 r-x-- libc-2.12.so
00007f4b8abec000 48 r-x-- libpci.so
00007f4b8adf9000 32 r-x-- librt-2.12.so
00007f4b8b002000 248 r-x-- libdbus-1.so.3.4.0
00007f4b8b241000 96 r-x-- libpthread-2.12.so
00007f4b8b45e000 128 r-x-- ld-2.12.so
--------------------------------------------
#0 0x00007f4b8a955c83 in select+0x13 from /lib64/libc-2.12.so
#1 0x000000000041f974 in ?? () from /sbin/init
#2 0x0000000000404b9d in ?? () from /sbin/init
#3 0x00007f4b8a897cce in __libc_start_main+0xfe from /lib64/libc-2.12.so
#4 0x0000000000404659 in ?? () from /sbin/init
To terminate or restart a process, use the process command in the System Admin EXEC mode.
process { crash | restart } executable-name { IID location node-id | location node-id }
crash |
Ends a process. All active services hosted by the process that have high availability enabled are switched off and the process restarts. |
restart |
Restarts a process. |
executable-name |
Executable name of the process to be crashed or restarted. Supplying an executable name for the executable-name argument performs the action for all the simultaneously running instances of the process, if applicable. |
IID |
Process instance ID of the process to be crashed or restarted. Supplying a process ID for the IID argument performs the action for only the process instance associated with the process ID. |
location node-id |
Crashes or restarts a process on the designated node. The node-id argument is entered in the rack/slot notation. |
None
System Admin EXEC
Release |
Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
Under normal circumstances, processes are started and restarted automatically by the operating system as required. If a process crashes, it is automatically restarted.
Use this command to manually stop or restart individual processes.
Caution | Manually stopping or restarting a process can seriously impact the operation of a router. Use these commands only under the direction of a Cisco Technical Support representative. |
The process restart command restarts a process, such as a process that is not functioning optimally.
sysadmin-vm:0_RP0# process restart syslogd_helper location 0/3
proc-action-status User root (127.0.0.1) requested restart for process syslogd_helper(0) at 0/3 'Sending signal 15 to process syslogd_helper(IID 0) pid=1801'
To display the current state of the disk storage media, use the show media command in System Admin EXEC mode.
show media location { node-id | all }
location{node-id | all} |
(Optional) Specifies the node where the file system is located. The node-id argument is expressed in the rack/slot notation. Use the all keyword to indicate all nodes. |
The disk storage media for the active RP is displayed.
System Admin EXEC
Release |
Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
Use the show media command to view the status of the storage media on your system.
sysadmin-vm:0_RP0#show media
Partition Size Used Percent Avail
rootfs: 2.0G 471M 26% 1.4G
log: 494M 84M 18% 385M
config: 494M 24M 5% 445M
disk0: 965M 31M 4% 886M
harddisk: 20G 185M 1% 19G
---------------------------------------------------
rootfs: = root file system (read-only)
log: = system log files (read-only)
config: = configuration storage (read-only)
Field | Description |
---|---|
Partition | Partition on the disk. |
Size | Size of the partition. |
Used | Partition size used. |
Percent | Percentage used. |
Avail | Available free partition space. |
To display the available physical memory and memory usage information of processes on the router, use the show memory command in System Admin EXEC and XR EXEC mode.
show memory [ location node-id | pid pid [ location node-id ] | summary [ location node-id ] ]
location node-id |
Displays the available physical memory from the designated node. The node-id argument is entered in the rack/slot notation. |
pid pid |
Displays memory usage of the specified process. |
summary |
Displays a summary of the physical memory and memory usage information. |
None
System Admin EXEC
XR EXEC
Release |
Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
To display detailed memory information for the entire router, enter the show memory command without any parameters.
sysadmin-vm:0_RP0#show memory location 0/RP0
Tue Aug 20 00:49:41.649 UTC
************************************************************
Location : 0/RP0
************************************************************
Tue Aug 20 00:49:41 UTC 2013
1: /sbin/init
Address Kbytes RSS Anon Locked Mode Mapping
0000000000400000 204 - - - r-x-- init
0000000000632000 4 - - - rw--- init
Address - Memory Address
Kbytes - Memory Size
RSS - Resident Set Size (portion of mem in RAM)
Anon - Non-shared Anonymous
Locked - locked memory
Mode - Read/Write/Executable mode
Mapping - process Mapping
To display details about heap memory usage for all processes on the router at different moments in time and compare the results, use the show memory compare command in System Admin EXEC and XR EXEC mode.
show memory compare { start | end | report }
start |
Takes the initial snapshot of heap memory usage for all processes on the router and sends the report to a temporary file named /tmp/memcmp_start.out. |
end |
Takes the second snapshot of heap memory usage for all processes on the router and sends the report to a temporary file named /tmp/memcmp_end.out. This snapshot is compared with the initial snapshot when displaying the heap memory usage comparison report. |
report |
Displays the heap memory comparison report, comparing heap memory usage between the two snapshots of heap memory usage. |
None
System Admin EXEC
XR EXEC
Release |
Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
Use the show memory compare command to display details about the heap memory usage of all processes on the router at different moments in time and compare the results. This command is useful for detecting patterns of memory usage during events such as restarting processes or configuring interfaces.
Use the following steps to create and compare memory snapshots:
Enter the show memory compare command with the start keyword to take the initial snapshot of heap memory usage for all processes on the router.
Perform the test you want to analyze.
Enter the show memory compare command with the end keyword to take the snapshot of heap memory usage to be compared with the initial snapshot.
Enter the show memory compare command with the report keyword to display the heap memory usage comparison report.
This example shows sample output from the show memory compare command with the report keyword:
sysadmin-vm:0_RP0# show memory compare start Tue Aug 20 11:50:45.860 UTC sysadmin-vm:0_RP0# show memory compare end Tue Aug 20 11:50:57.311 UTC sysadmin-vm:0_RP0# show memory compare report PID NAME MEM BEFORE MEM AFTER DIFFERENCE MALLOCS --------------------------------------------------------------------- 21416 malloc_dump 34731 34731 0 0 21414 sh 39652 39640 -12 0 21411 show_memory_common 984 984 0 0 8340 ntpd 69033 69033 0 0 5172 inst_mgr 1800118 1800118 0 0 5166 fsdbagg 14907247 14907247 0 0 5175 fsdb_server 15475470 15475470 0 0 5177 led_mgr 3347339 3347339 0 0 5176 envmon_ui 889094 889094 0 0 5169 esdma 8954927 8954927 0 0 5164 fit_mgbl 952067 952067 0 0 5174 fab_fgid_service 9014924 9014924 0 0 5173 confd_helper 8018190 8018190 0 0 5171 debug_agent 8146830 8146830 0 0 5170 gaspp_mgbl 1285020 1285020 0 0 5168 ael_mgbl 787101 787101 0 0 5165 fpdserv 1149685 1149685 0 0 5167 ssh_key_server 661086 661086 0 0 2052 sfe_driver 35005323 35005323 0 0 2066 zen 5083246 5083246 0 0 2017 ccc_driver 8872747 8882315 9568 1 2053 shelf_mgr 30666121 30666121 0 0 2031 esd 6335087 6334783 -304 -2 2049 sdr_mgr 4366258 4366258 0 0 2025 dumper 616144 616144 0 0 2035 inst_agent 1820469 1820469 0 0 2062 syslogd_relay 657904 657904 0 0 2030 envmon 7853186 7853330 144 2 2041 ntp_helper 701348 701348 0 0 2539 ssh 202441 202441 0 0 2015 bios_fpd 2950893 2950893 0 0 2042 obfl_mgr 2686006 2686006 0 0 2018 cm 13755230 13755230 0 0 2047 obfl_show 686286 686286 0 0 2024 ds 7826821 7826821 0 0 2060 syslogd_helper 912664 912664 0 0 2014 aaad 804327 804327 0 0 2019 debug_client 577975 577975 0 0 2016 calv_alarm_mgr 2077250 2077250 0 0 2065 wdmon 3557984 3558056 72 1 2064 vm_manager 3149588 3149588 0 0 2037 mlap 1520260 1520260 0 0 2056 ssh_key_client 612824 612824 0 0 2055 ship_server 778066 778066 0 0 2063 timezone_config 711110 711110 0 0 1744 pm 7875584 7875584 0 0
Field |
Description |
---|---|
PID |
Process ID. |
name |
Process name. |
mem before |
Heap memory usage at start (in bytes). |
mem after |
Heap memory usage at end (in bytes). |
difference |
Difference in heap memory usage (in bytes). |
mallocs |
Number of unfreed allocations made during the test period. |
restarted |
Indicates if the process was restarted during the test period. |
To display information about the heap space for a process, use the show memory heap command in System Admin EXEC and XR EXEC mode.
show memory heap pid
pid |
Process ID |
None
System Admin EXEC
XR EXEC
Release |
Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
sysadmin-vm:0_RP0#show memory heap 1933
Tue Aug 20 01:06:11.282 UTC
statistics (1933:vm_manager)
Global data:
current usage: 3147787 bytes
Wrapper uses: 109560 bytes(hash:32728)
total high wm: 7342424 bytes
current objs: 2401 entry
malloc_db/malloc: 79946 times / 79946 times
calloc_db/calloc: 1067 times / 1067 times
realloc_db/realloc: 26342 times / 26342 times
realloc_null: 25644 times
realloc_db_miss : 0 times
realloc_relocate: 39 times
free_db/free: 104256 times / 104722 times
free_null: 466 times
free_db_miss: 0 times
error: 0 times
To display information about active processes, use the show processes command in System Admin EXEC mode.
show processes { process-name { [ detail | run ] location node-id | location node-id } | aborts location node-id | all location node-id | blocked [ PID | extended | location node-id ] | family [ PID | location node-id ] | files [ PID | details | location node-id ] | location [ all | node-id ] | mandatory location node-id | memory [ PID | location node-id ] | services { service-name | active | all | run | standby } location node-id | signal [ PID | location node-id ] | startup location node-id | threadname [ PID | location node-id ] }
process-name |
Name of the executable. |
detail |
Displays detailed information of the process. |
run |
Displays information of running processes. |
location node-id |
Displays information about the active processes from a designated node. The node-id argument is entered in the rack/slot notation. |
aborts |
Displays process abort information. |
all |
Displays summary process information for all processes. |
blocked |
Displays details about reply, send, and mutex blocked processes. |
PID |
Displays process ID. |
extended |
Displays blocked processes in detail. |
family |
Displays the process session and family information. |
files |
Displays information about open files and open communication channels. |
mandatory |
Displays process data for mandatory processes. |
memory |
Displays information about the text, data, and stack usage for processes. |
services service name |
Displays service data for processes. |
active |
Displays active services data. |
standby |
Displays standby services data. |
signal |
Displays the signal options for blocked, pending, ignored, and queued signals. |
startup |
Displays process data for processes created at startup. |
threadname |
Displays thread names. |
None
System Admin EXEC
Release |
Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
Use the show processes command to display process level information across the system.
The show processes command with the memory keyword displays details of memory usage for a given process as shown in the following example:
sysadmin-vm:0_RP0# show process memory
PID Text Data Stack Dynamic Process
=========================================================
1 204 KB 204 KB 136 KB 14932 KB init
12680 16 KB 48 KB 136 KB 3852 KB sleep
12747 32 KB 8432 KB 136 KB 24776 KB cmdptywrapper
12751 12 KB 8508 KB 136 KB 74040 KB show_processes_
12754 724 KB 8456 KB 136 KB 25832 KB sh
1299 724 KB 208 KB 136 KB 11280 KB oom.sh
1305 724 KB 208 KB 136 KB 11280 KB oom.sh
1443 476 KB 540 KB 136 KB 14984 KB dhclient
1486 28 KB 188 KB 136 KB 6104 KB syslogd
1490 20 KB 3056 KB 136 KB 6864 KB klogd
1545 224 KB 204 KB 136 KB 13172 KB lldpad
1557 308 KB 204 KB 136 KB 12844 KB dbus-daemon
1588 412 KB 444 KB 136 KB 23252 KB sshd
1593 412 KB 444 KB 136 KB 23252 KB sshd
1602 192 KB 372 KB 136 KB 11120 KB xinetd
1618 40 KB 692 KB 524 KB 7008 KB crond
1630 792 KB 49720 KB 136 KB 83164 KB libvirtd
1711 116 KB 636 KB 136 KB 4540 KB udevd
1712 116 KB 636 KB 136 KB 4540 KB udevd
1722 324 KB 16164 KB 136 KB 148164 KB pm
Field |
Description |
---|---|
PID |
Process ID. |
Text |
Size of text region (process executable). |
Data |
Size of data region (initialized and uninitialized variables). |
Stack |
Size of process stack. |
Dynamic |
Size of dynamically allocated memory. |
Process |
Process name. |
To display real-time view of running processes in different locations, use the top command in the System Admin EXEC and XR EXEC modes.
top [ dumbtty | location node-id [dumbtty] ]
dumbtty |
Displays the output of the command as if on a dumb terminal (the screen is not refreshed). |
location location |
Specifies the target location. The node-id argument is expressed in rack/slot notation. |
None
System Admin EXEC
XR EXEC
Release | Modification |
---|---|
Release 5.0.0 |
This command was introduced. |
The top command provides a real-time list of CPU intensive tasks running in the system. To terminate the display and return to the system prompt, enter the Ctrl+C keys. Using the dumbtty option does not overwrite the logs but instead updates the real-time list one after the other.
sysadmin-vm:0_RP0#top location 0/0 dumbtty
Tue Aug 20 01:09:28.534 UTC
************************************************************
Location : 0/0
************************************************************
top - 01:09:29 up 3:35, 1 user, load average: 0.00, 0.00, 0.00
Tasks: 170 total, 2 running, 168 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.5%us, 0.4%sy, 0.0%ni, 98.9%id, 0.1%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 916860k total, 374500k used, 542360k free, 12080k buffers
Swap: 14444k total, 6200k used, 8244k free, 31736k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1764 root 20 0 140m 3844 2256 S 2.0 0.4 0:13.18 syslogd_helper
1 root 20 0 14932 1080 1000 S 0.0 0.1 0:01.83 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.27 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
6 root 20 0 0 0 0 S 0.0 0.0 0:00.61 events/0
7 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuset
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khelper