Collect DM Logs From Cellular Gateway and PIM Module

Introduction

This document describes the process of DM Diagnostic Monitor (DM) logs collection for Cellular Gateways and Pluggable Interfaces Modules (PIM).

What are DM Logs

DM - Diagnostic Monitor - logs are used to capture data transactions between the modem and the network over the RF interface, which helps to troubleshoot 4G and 5G data connectivity or performance issues.

Basic 5G Call Flow

It is important to understand the basic call flow, before going into DM logs collection and analysis.

Basic 5G Call Flow

1. Device Initialization and Cell Search:

• The 5G device powers on and begins searching for available 5G cells.
• It scans for synchronization signals broadcast by nearby base stations (gNBs).

2. Initial Access:

• Once a suitable cell is found, the device attempts to synchronize with the gNB timing and frequency.
• It transmits a message (RRC Connection Request) to the gNB, requesting a connection.

3. RRC Connection Establishment:

• The gNB receives the request and establishes a Radio Resource Control (RRC) connection with the device.
• This connection allows for secure and reliable communication between the device and the network.

4. Registration:

• The device authenticates itself with the network (using SIM card information).
• It registers its presence with the network and requests access to services.

5. Data Transmission:

• Once registered, the device can now send and receive data over the 5G network.
• This could include voice calls, data traffic (browsing, streaming), or other applications.

6. Connection Release:

• When the user ends the call or disconnects from the network, the device releases the connection with the gNB.

Key Points:

• RRC Connection: This is a critical step that enables secure and reliable communication between the device and the network.
• Data Plane: Once the RRC connection is established, the device can use the data plane to transmit and receive data.
• Control Plane: The control plane is used for signaling and control functions, such as establishing the connection and managing resources.

DM Logs Collection For Cellular Gateway

First of all, shutdown the modem before enabling DM Logs, this can be done using the AT command AT+CFUN=0

CellularGateway# cellular 1 modem-at-commands
Value for '' (<string>): AT+CFUN=0

This command takes a few second and freezes the SSH session, have another SSH session opened to the Cellular Gateway.

Verify that it has been applied with the AT command AT+CFUN?

CellularGateway# cellular 1 modem-at-commands
Value for '' (<string>): AT+CFUN?

cellular_modem_at_cmd: modem-at-response AT+CFUN?
+CFUN: 0
OK

Setup DM Logs size, rotation and autostop.

CellularGateway# config terminal
Entering configuration mode terminal
CellularGateway(config)# controller cellular 1
CellularGateway(config-cellular-1)# dm-log rotation
CellularGateway(config-cellular-1)# dm-log max_dm_log_size 60
CellularGateway(config-cellular-1)# dm-log autostop-event MODEM_STATE_DNS_ACQUIRED
CellularGateway(config-cellular-1)# dm-log autostop-timer 2

These settings can be adapted based on the log collection flow, optionally a filter can be applied if the issue is related to cellular performance.

Enable DM Logs

CellularGateway(config-cellular-1)# dm-log enable
CellularGateway(config-cellular-1)# commit

Enable the modem using AT+CFUN=1 AT Command.

CellularGateway# cellular 1 modem-at-commands
Value for '' (<string>): AT+CFUN=1

cellular_modem_at_cmd: modem-at-response AT+CFUN=1
OK

CellularGateway#
CellularGateway# cellular 1 modem-at-commands
Value for '' (<string>): AT+CFUN?

cellular_modem_at_cmd: modem-at-response AT+CFUN?
+CFUN: 1
OK

Check the files created.

CellularGateway# gw-action:request file list
-rw-r--r-- 97272745 Dec 11 09:05 dmlog-slot0-20241211-090453.tar.gz
-rw-r--r-- 10550241 Dec 25 10:58 dmlog-slot0-20241225-105827.tar.gz
drwx------ 16384 Sep 27 18:29 lost+found
drwxr-xr-x 4096 Dec 25 11:03 storage
drwxr-xr-x 4096 Sep 27 18:29 tmp

These files can be transferred by initiating a copy from the next hop.

5GSwitch#copy tftp://192.168.1.1/dmlog-slot0-20241211-090453.tar.gz flash:
Destination filename [dmlog-slot0-20241211-090453.tar.gz]?
Accessing tftp://192.168.1.1/dmlog-slot0-20241211-090453.tar.gz...
Loading dmlog-slot0-20241211-090453.tar.gz from 192.168.1.1 (via GigabitEthernet1/0/3): !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Disable DM Logs once debugging is complete.

CellularGateway# config terminal
Entering configuration mode terminal
CellularGateway(config-cellular-1)# no dm-log enable
CellularGateway(config-cellular-1)# commit

DM Logs Collection For PIM Modules

First, shutdown the cellular interface and the radio.

ISR2#configure terminal
ISR2(config)#interface cellular 0/2/0
ISR2(config-if)#shutdown
ISR2(config-if)#exit
ISR2(config)#controller cellular 0/2/0
ISR2(config-controller)#lte radio off

Setup DM logging parameters, including rotation, file size, auto stop and if a filter is needed.

ISR2(config)#controller cellular 0/2/0
ISR2(config-controller)#lte modem dm-log rotation
ISR2(config-controller)#lte modem dm-log filesize 20
ISR2(config-controller)#lte modem dm-log autostop timer 20

Enable DM Logs

ISR2(config-controller)#lte modem dm-log enable

Enable the radio and the cellular interface.

ISR2(config)#controller cellular 0/2/0
ISR2(config-controller)#no lte radio off
ISR2(config)#interface cellular 0/2/0
ISR2(config-if)#no shutdown

Verify DM Logs stats and files.

ISR2#show cellular 0/2/0 logs dm-log
Integrated DM logging is on
Output path = bootflash:
Filter Type = MC73xx generic
Filter Name = Telit Default log mask.bin
Maximum log size = 64 MB
Maximum file size = 20 MB
Log rotation = Enabled

ISR2#dir bootflash: | inc dm
24 -rw- 4236574 Jan 3 2025 11:39:33 +00:00 dmlog-slot2-20250103-113930.tar.gz

Once debugging is completed, disable DM logs.

ISR2(config)#controller cellular 0/2/0

ISR2(config-controller)#no lte modem dm-log enable

Using EEM Scripts to disable DM Logging on PIM Modules

We can use an EEM Script to disable DM logging based on the specific log message or event, the most common use case is the cellular interface or tunnel interface flap. 

event manager applet Cellular-Capture authorization bypass
event syslog pattern "Syslog Messege Needed to stop logging" maxrun 600
action 100 cli command "enable"
action 101 cli command "terminal length 0"
action 102 cli command "term exec prompt timestamp"
action 103 cli command "show cellular 0/2/0 all | append flash:Out_info"
action 104 cli command "show controller cellular 0/2/0 | append flash:Out_info"
action 105 cli command "show dialer | append flash:Out_info"
action 106 cli command "show int cellular 0/2/0 | append flash:Out_info"
action 112 syslog msg "Information loaded"
action"113"wait"180
action 114 cli command "show log | append flash:Out_info"
action 115 cli command "enable"
action 116 cli command "conf t"
action 117 cli command "controller cellular 0 1"
action 118 cli command "no lte modem dm-log enable"
action 119 syslog msg "DM-logs disabled"
action 120 cli command "en

Conclusion

DM Logs analysis requires TAC assistance as the files collected require decoding, please open a TAC case in case of an issues where DM logging is required.