Troubleshooting Information

Feature Summary and Revision History

Summary Data

Table 1. Summary Data

Applicable Product(s) or FunctionalArea

SMF

Applicable Platform(s)

SMI

Feature Default Setting

Not Applicable

Related Changes in this Release

Not Applicable

Related Documentation

Not Applicable

Revision History

Table 2. Revision History
Revision Details Release

Added Troubleshooting information for RADIUS, Roaming, and Option-3x features.

2021.01.0

First introduced.

Pre-2020.02.0

clear Commands

clear subscriber

Table 3. clear subscriber Command Output Description

Field

Description

all

Clears all the sessions.

amf

Clears subscriber based on AMF address.

chf

Clears subscriber based on CHF address.

dnn

Clears subscriber based on DNN value.

gtp-peer

Clears subscriber based on GTP-PEER address.

ipv4-pool

Clears subscriber based on IPv4 pool name.

ipv4-range

Clears subscriber based on IPv4 address-range.

ipv6-pool

Clears subscriber based on IPv6 pool name.

ipv4-range

Clears subscriber based on IPv6 prefix-range.

ipv6-range

Clears subscriber based on IPv6 prefix-range.

namespace

Clears subscriber based on the respective namespace. Default: none.

pcf

Clears subscriber based on PCF address.

policy

Clears subscriber based on policy information.

purge

Clears true, if purged locally.

sgw

Clears subscriber based on the S-GW address.

smf

Clears subscriber based on the SMF address.

supi

Clears subscriber based on the SUPI value.

|

The output modifiers.

clear subscriber supi imsi <imsi_value>

Table 4. clear subscriber supi imsi <imsi_value> Command Output Description

Field

Description

ebi

Clears subscriber based on EPS bearer ID.

imsi

Clears subscriber based on IMSI.

purge

Clears true, if purged locally.

|

Output modifier.

clear subscriber supi imsi <imsi_value> psid <psid_value>

Table 5. clear subscriber supi imsi <imsi_value> psid <psid_value> Command Output Description

Field

Description

ebi

Clears subscriber based on EPS bearer ID.

imsi

Clears subscriber based on IMSI.

psid

Clears subscriber based on Service ID.

purge

Clears true, if purged locally.

|

Output modifier.

show Commands

show helm

The show helm command displays the version information for the SMF system image.

show peers all

This command displays the list of external inbound and outbound connections established by SMF.

Table 6. show peers all Command Output Description

Field

Description

additionalDetails

Displays the additional details.

connectedTime

Displays the connected time.

direction

Displays the direction.

podInstance

Displays the pod instance.

rpc

Displays the interface RPC.

type

Displays the interface type.

|

Displays the output modifiers.

show subscriber

This commands displays the existing show subscriber CLI output with the newly added CLI output.

Table 7. show subscriber Command Output Description

Field

Description

all

Displays the information for all SUPIs or IMEIs.

amf

Displays the AMF address.

chf

Displays the CHF address.

count

Displays the number of sessions.

debug

Displays the debugging information.

dnn

Displays the DNN value.

gtp-peer

Displays the GTP-peer address.

namespace

Displays the product namespace under which to search. Default: none.

pcf

Displays the PCF address.

rat

Displays the RAT type as 4G or 5G.

supi

Displays the SUPI value.

udm

Displays the UDM address.

upf

Displays the UPF address.

|

The output modifiers.

show subscriber count

This command displays the CLI options for the count CLI command.

Table 8. show subscriber count Command Output Description

Field

Description

all

Displays all the SUPIs.

amf

Displays the AMF address.

chf

Displays the CHF address.

dnn

Displays the DNN value.

gtp-peer

Displays the GTP-peer address.

pcf

Displays the PCF address.

rat

Displays the RAT type as 4G or 5G.

supi

Displays the specific SUPI value.

udm

Displays the UDM address.

upf

Displays the UPF address.

|

Displays the output modifiers.

show subscriber count all

This command displays the total number of sessions for all the SUPIs.

Table 9. show subscriber count all Command Output Description

Field

Description

|

Displays the output modifiers.

show subscriber count chf <chf_address>

This command displays the total number of sessions for the specified parameters.

Table 10. show subscriber count chf <chf_address> Command Output Description

Field

Description

amf

Displays the AMF address.

dnn

Displays the DNN value.

gtp-peer

Displays the GTP-peer address.

pcf

Displays the PCF address.

rat

Displays the RAT type as 4G or 5G.

udm

Displays the UDM address.

upf

Displays the UPF address.

|

Displays the output modifiers.

show subscriber count chf <chf_address> dnn <dnn_value>

This command displays the total number of sessions for the specified parameters.

Table 11. show subscriber count chf <chf_address> dnn <dnn_value> Command Output Description

Field

Description

amf

Displays the AMF address.

gtp-peer

Displays the GTP-peer address.

pcf

Displays the PCF address.

rat

Displays the RAT type as 4G or 5G.

udm

Displays the UDM address.

upf

Displays the UPF address.

|

Displays the output modifiers.

show subscriber count supi <supi_value>

This command displays the total number of sessions for the specific SUPI value.

Table 12. show subscriber count supi <supi_value> Command Output Description

Field

Description

|

Displays the output modifiers.

show subscriber debug-info supi <supi_value>

This command displays the debug information for the specific SUPI value where the PSID value is optional.

Table 13. show subscriber debug-info supi <supi_value> Command Output Description

Field

Description

psid

Displays the Provider Service Identifier.

show subscriber debug-info supi <supi_value> psid <psid_value>

This command displays the debug information for the specific SUPI and PSID combination.

Table 14. show subscriber debug-info supi <supi_value> psid <psid_value> Command Output Description

Field

Description

|

Displays the output modifiers.

Note

Currently, the SMF does not validate the serving PLMN received on N11 interface. Also, the SMF does not validate the UE PLMN when the N11 smContextCreate or GTP createSessionRequest is received.

The PLMN value received in the NSSAI included in the PCO request remain the same as the PLMN configured on the SMF.

show subscriber supi <supi_value> psid <psid_value> full

This command displays detailed subscriber information.

Table 15. show subscriber supi <supi_value> psid <psid_value> full Command Output Description

Field

Description

sessTimeStamp

Connected time of the session.

callDuration

Call duration.

commonId

Call ID equivalent for the session (common ID).

ipPool, ipv6Pool

IP pool from which the address has been allocated.

linkedEbi

Linked EBI for a session.

snssai

sNssai details.

smfIwkEpsInd

SMF EPS IWK decision based on AMF and UDM data.

TotalNumberOfPdrs

Number of associated PDRs.

TotalNumberOfFars

Number of associated FARs.

TotalNumberOfQers

Number of associated QERs.

TotalNumberOfUrrs

Number of associated URRs.

upfSeid

Remote SEID for a particular UPF session.

epsInterworking Indication

EPS interworking indication status of AMF.

ebi

ERAB ID allocated for each flow.

revalidationTime

Revalidation timer information for a session.

SMF Monitoring

This section provides information on CLI commands for monitoring the health of 5G SMF.

SMF Data Collection Method

The SMF application CLI data is collected at the following levels:

  • SMF Application CLI:

    • SMF Ops-Center CLI

    • CEE Ops-Center CLI

Application CLI

The following table lists the CLI commands to be executed on the SMF Ops-Center for RADIUS.

CLI Command

Description

Comments

show subscriber supi supi_id full

Displays detailed subscriber information.

Additional information for accounting, authentication, and RADIUS URR is added.

show ipam dp dp-name ipv4-addr

Displays IPAM data plane allocation for a specific UPF.

Additional information added to display VRF tag and static pool.

show ipam dp dp-name ipv6-prefix

Displays range of IPv6 pool allocated to the UPF.

Additional information added to display VRF tag and static pool.

show radius

Displays RADIUS endpoint accounting and authentication statistics.

show radius dyn-auth

Displays statistics for RADIUS Disconnect/COA from RADIUS endpoint.

CoA is not supported in the SMF releases 2020.04 and 2021.01.

show radius acct-server

Displays RADIUS endpoint accounting statistics.

Per server details can also be fetched.

show radius auth-server

Displays RADIUS endpoint authentication statistics.

Per server details can also be fetched.

show endpoint

Displays internal and external connections.

Additional RADIUS-specific endpoint information is added.

show peers

Displays the list of external inbound and outbound connections established by the SMF.

Additional information for RADIUS Authentication and Accounting server is added.

CEE Ops-Center CLI

The following table lists the CLI commands to be executed on the CEE Ops-Center.

CLI Command

Description

Comments

show alerts active

Displays active alerts.

New alerts are added for RADIUS.

show bulk-stats current

Displays bulk statistics.

New bulk statistics are added for RADIUS.

SMF Data Collection - CLI Output

SMF Ops-Center CLI Commands

The CLI execution fails due to various reasons. In such cases, the output of the CLI contains the word "Error". The Validation Criteria looks for "Error" in a case-insensitive search.

The following are examples.

************** LASMF006 - show sessions commit-pending *************
Error: 'sessions commit-pending': Error in callback: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $
 
************** LASMF006 - show running-status **************
Error: application timeout
Subscriber Details for RADIUS-specific Information

The show subscriber supi supi_id full CLI command displays the subscriber details for RADIUS-specific use cases. 

[unknown] smf# show subscriber supi imsi-123456789012345 full  
subscriber-details 
{
…
"alwaysOn": "None",
        "dcnr": "None",
        "wps": "Wps Session",
        "ratType": "NR",
        "idleTimeout": 600,      << can be overwritten from Radius in Auth Resp 
        "sessTimeout": 1200,     << can be overwritten from Radius in Auth Resp 
        "radiusEpInfo": "198.18.1.20:1812", 
        "authAlg": "pap-default", 
        "authStatus": "Authenticated" 
…
…
        "accountingEnabled": "true", 
        "n40ChargingEnabled": "true",
        "acctSessId": "198.15.1.40016777221" 
…
…
"upfServData": {
        "numberOfTunnels": 2,
        "smfSeid": 72057615828912656,
        "UPState": "Activated",
        "urrInfo": [
          {
            "id": 2147483657,
            "chgName": "radiusurr",
            "method": {
              "duration": "false",
              "volume": "true",
              "event": "false"
            },
Subscriber Details for Roaming-specific Information

The show subscriber supi supi_id nf-service smf full CLI command displays the roaming status of a UE. 

[unknown-smf] smf# show subscriber supi imsi-123456789012345 nf-service smf full 
subscriber-details 
{
…
        "authStatus": "Unauthenticated",
        "roamingStatus": "Vistor LBO",    <<< In-Roamer UE Roaming Status
        "uePlmnId": {
          "mcc": "123",
          "mnc": "456"
        }

…
     "authStatus": "Unauthenticated",
        "roamingStatus": "Roamer",      <<< Out-Roamer UE Roaming Status
        "uePlmnId": {
          "mcc": "123",
          "mnc": "456"
        }
…
        "authStatus": "Unauthenticated",
        "roamingStatus": "Homer",
        "uePlmnId": {
          "mcc": "123",
          "mnc": "456"
        }
Subscriber Details with DCNR and Presence Reporting Enabled

The show subscriber nf-service smf supi supi_id full CLI command displays the DCNR active session with presence reporting enabled for the Option-3x feature. 

[unknown] smf# show subscriber nf-service smf supi imsi-310260789012345 full

subscriber-details 
{
  “subResponses”: [
    {
      “status”: true,
      “genericInfo”: {
        “supi”: “imsi-310260789012345”,
        “pei”: “imei-123456786666660”,
        “pduSessionId”: 5,
        “pduSesstype”: “Ipv4PduSession”,
        “accessType”: “3GPP_ACCESS”,
        “dnn”: “fast.t-mobile.com”,
        “plmnId”: {
          “mcc”: “123”,
          “mnc”: “456”
        },
…
        “alwaysOn”: “None”,
        “dcnr”: “Enabled”,
        “wps”: “Non-Wps Session”,
        “ratType”: “EUTRA”,
        “ueType”: “NR Capable UE”,
        “iwkEpsInd”: true,
        “sessTimeStamp”: “2021-01-12 12:40:39.931012285 +0000 UTC”,
        “callDuration”: “4m25.36784895s”,
        “ipPool”: “poolv4”,
        “commonId”: 16777223,
        “linkedEbi”: 5,
        “smfIwkEpsInd”: true,
        “snssai”: {
          “sd”: “Abf123”,
          “sst”: 1
        },
        “authStatus”: “Unauthenticated”,
        “roamingStatus”: “Roamer”,
        “uePlmnId”: {
          “mcc”: “310”,
          “mnc”: “260”
        }
      },
      “policySubData”: {
        “TotalDynamicRules”: 2,
        “TotalFlowCount”: 2,
        “TotalNonGBRFlows”: 1,
        “TotalGBRFlows”: 1,
        …
        “presenceReporting”: “Enabled”,
        “praList”: [
          {
            “praId”: “0x80000b”,
            “presenceState”: “Inactive”
          },
          {
            “praId”: “0x800000”,
            “presenceState”: “InArea”
          },
          {
            “praId”: “0x80000a”,
            “presenceState”: “OutOfArea”
          }
        ]
      },
      …
      }
    }
  ]
}
Range of IPv4 Allocated to UPF

The show ipam dp dp_name ipv4-addr CLI command displays the IP pool chunks allocated to UPF. This includes the VRF tag information and details such as whether the pool defined is a static or dynamic pool. 

[unknown] smf# show ipam dp 198.18.1.3 ipv4-addr  

==============================================================================================
Flag Indication: S(Static) O(Offline)
N/P  Indication: N(Native InstId) P(Peer InstId)
==============================================================================================
StartAddress       EndAddress      AllocContext  Route            N/P    Utilization   Flag 
==============================================================================================
16.0.0.0           16.0.0.31       v4pool3(vrf3@ISP)16.0.0.0/27      -                    S
16.0.0.0           16.0.0.31       v4pool4(vrf4@ISP)16.0.0.0/27      -                    S
16.0.0.32          16.0.0.63       v4pool3(vrf3@ISP)16.0.0.32/27     -                    S
16.0.0.32          16.0.0.63       v4pool4(vrf4@ISP)16.0.0.32/27     -                    S
16.0.0.64          16.0.0.95       v4pool3(vrf3@ISP)16.0.0.64/27     -                    S
16.0.0.64          16.0.0.95       v4pool4(vrf4@ISP)16.0.0.64/27     -                    S
16.0.0.96          16.0.0.127      v4pool3(vrf3@ISP)16.0.0.96/27     -                    S
16.0.0.96          16.0.0.127      v4pool4(vrf4@ISP)16.0.0.96/27     -                    S
16.0.0.128         16.0.0.159      v4pool4(vrf4@ISP)16.0.0.128/27    -                    S
16.0.0.160         16.0.0.191      v4pool4(vrf4@ISP)16.0.0.160/27    -                    S
16.0.0.192         16.0.0.223      v4pool4(vrf4@ISP)16.0.0.192/27    -                    S
16.0.0.224         16.0.0.255      v4pool4(vrf4@ISP)16.0.0.224/27    -                    S
===================================================================================
Range of IPv6 Allocated to UPF

The show ipam dp dp_name ipv6-prefix CLI command displays the IP pool chunks allocated to UPF. This includes the VRF tag information and details such as whether the pool defined is a static or dynamic pool. 

[unknown] smf# show ipam dp 198.18.1.3 ipv6-prefix  

===================================================================================
Flag Indication: S(Static) O(Offline)
N/P  Indication: N(Native InstId) P(Peer InstId)
=============================================================================================================================
StartAddress          EndAddress             AllocContext       Route                    N/P    Utilization   Flag 
===================================================================================
3001:db0::            3001:db0:0:3fff::      v6pool4(vrf4@ISP)  3001:db0::/50            -                    S
3001:db0::            3001:db0:0:3fff::      v6pool3(vrf3@ISP)  3001:db0::/50            -                    S
3001:db0:0:4000::     3001:db0:0:7fff::      v6pool4(vrf4@ISP)  3001:db0:0:4000::/50     -                    S
3001:db0:0:4000::     3001:db0:0:7fff::      v6pool3(vrf3@ISP)  3001:db0:0:4000::/50     -                    S
===================================================================================
[unknown] smf#
RADIUS Endpoint Authentication and Accounting Statistics

The show radius CLI command displays statistics for RADIUS Authentication and Accounting from RADIUS endpoint. 

[unknown] smf# show radius 
radius
--------------------------------------------------------
 Server: 198.18.1.20, port: 1812, status: up, port-type: Auth
 3 requests, 0 pending, 0 retransmits
 2 accepts, 0 rejects, 1 timeouts
 0 bad responses, 0 bad authenticators
 0 unknown types, 0 dropped, 1 ms latest rtt
--------------------------------------------------------
--------------------------------------------------------
 Server: 10.84.115.45, port: 1813, status: up, port-type: Acct
 3 requests, 0 pending, 6 retransmits
 0 responses, 3 timeouts
 0 bad responses, 0 bad authenticators
 0 unknown types, 0 dropped, 0 ms latest rtt
--------------------------------------------------------
 Server: 198.18.1.20, port: 1813, status: up, port-type: Acct
 5 requests, 0 pending, 3 retransmits
 3 responses, 2 timeouts
 0 bad responses, 0 bad authenticators
 0 unknown types, 0 dropped, 6 ms latest rtt
--------------------------------------------------------
[unknown] smf# 

[unknown] smf# show radius acct-server  
--------------------------------------------------------
 Server: 10.84.115.45, port: 1813, status: up, port-type: Acct
 3 requests, 0 pending, 6 retransmits
 0 responses, 3 timeouts
 0 bad responses, 0 bad authenticators
 0 unknown types, 0 dropped, 0 ms latest rtt
--------------------------------------------------------
 Server: 198.18.1.20, port: 1813, status: up, port-type: Acct
 5 requests, 0 pending, 3 retransmits
 3 responses, 2 timeouts
 0 bad responses, 0 bad authenticators
 0 unknown types, 0 dropped, 6 ms latest rtt
--------------------------------------------------------
[unknown] smf#  
[unknown] smf# show radius auth-server  
--------------------------------------------------------
 Server: 198.18.1.20, port: 1812, status: up, port-type: Auth
 3 requests, 0 pending, 0 retransmits
 2 accepts, 0 rejects, 1 timeouts
 0 bad responses, 0 bad authenticators
 0 unknown types, 0 dropped, 1 ms latest rtt
--------------------------------------------------------
[unknown] smf#
RADIUS Endpoint Disconnect Message and CoA Statistics

The show radius-dyn-auth CLI command displays statistics for RADIUS Disconnect Message and CoA from RADIUS endpoint. 

[unknown] smf# show radius-dyn-auth
radius-dyn-auth
--------------------------------------------------------
 IP: 198.18.1.20
 ------------------
 COA:
 0 total-requests       0 inprocess-requests
     0 retry-request-drops  0 invalid-requests
     0 bad-authenticators   0 internal-errors
 0 ack-sent             0 nak-sent
 ------------------
 DISCONNECT:
 2 total-requests       0 inprocess-requests
     0 retry-request-drops  0 invalid-requests
     0 bad-authenticators   0 internal-errors
 1 ack-sent             1 nak-sent
 ------------------
 UnknownTypesRcvd: 0
--------------------------------------------------------
[unknown] smf#
External Inbound and Outbound Connections

The show peers all CLI command fetches the list of external inbound and outbound connections established by the SMF. 

[unknown] smf# show peers all | include Radius 
RadiusServer  -     10.84.115.45:1813   Outbound   radius-ep-0    Udp   18 hours   Radius  Status: Active,Type: Acct  1 
RadiusServer  -     198.18.1.20:1812    Outbound   radius-ep-0    Udp   17 hours   Radius  Status: Active,Type: Auth  1 
RadiusServer  -     198.18.1.20:1813    Outbound   radius-ep-0    Udp   17 hours   Radius  Status: Active,Type: Acct  1 
[unknown] smf#
Internal and External Connections

The show endpoint info CLI command fetches the list of internal and external connections established by the SMF. 

[unknown] smf# show endpoint all | include Radius 
Radius:198.18.1.4:     198.18.1.4:3799     Udp   Started  RADIUS     false     18 hours  <none>   1
[unknown] smf#
Status of Pods

The show running-status CLI command fetches the current status of pods. This is analogous to the K8 kubectl get pods –n <> CLI command. 

[unknown] smf# show running-status | include radius 
radius-ep-0      Started      19 hours  
[unknown] smf#
Configuration Errors

The show config-error CLI command displays the validation criteria — Pass (if there are no entries) or Failed. 

[unknown] smf# show config-error | include radius 
[unknown] smf#

CEE Ops-Center CLI Commands

RADIUS Alerts

The RADIUS endpoint for MVNO/PAPN flow supports new alerts. Some basic alerts are listed below. The same alerts can be enhanced based on RAT or as required by the users.

RADIUS EP Down Alert

Use the following example to configure alerts related to RADIUS EP Down.

alerts rules group RadiusEP
 rule Radius_Server_Down 
  expression "sum by (namespace, radSvrIP, radSvrPort) (Radius_Server_Status{radSvrPortType=~\"Auth|Acct\"} < 1)"
  duration   15m
  severity   major
  type       "Processing Error Alarm"
  annotation summary
  value "\"Radius Server: {{ $labels.radSvrIP }}, Port: {{ $labels.radSvrPort }} in namespace: {{ $labels.namespace }} is DOWN for more than 15min.\""
  exit
 exit
RADIUS Accounting Establishment Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Accounting Establishment Failure threshold.

alerts rules group RadiusEP
 rule Radius_Acct_Establish_SR 
   expression "sum by (namespace) (increase(Radius_Requests_Statistics{radMsgCode=\"RadiusAcctReq\", procType=\"PDU Session Establishment\", radPacketType=\"Rx\", radResult=\"Success\"}[5m])) / sum by (namespace) (increase(Radius_Requests_Statistics{radMsgCode=\"RadiusAcctReq\", procType=\"PDU Session Establishment\", radPacketType=\"Tx\"}[5m])) < 0.80"
   severity   major
   type       "Communications Alarm"
   annotation summary
     value "This alert is fired when the percentage of successful Radius Accounting Establish responses received is lesser than threshold"
   exit
  exit
RADIUS Accounting Release Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Accounting Release Failure threshold. 

rule Radius_Acct_Release_SR 
   expression "sum by (namespace) (increase(Radius_Requests_Statistics{radMsgCode=\"RadiusAcctReq\", procType=\"PDU Session Release\", radPacketType=\"Rx\", radResult=\"Success\"}[5m])) / sum by (namespace) (increase(Radius_Requests_Statistics{radMsgCode=\"RadiusAcctReq\", procType=\"PDU Session Release\", radPacketType=\"Tx\"}[5m])) < 0.80"
   severity   major
   type       "Communications Alarm"
   annotation summary
     value "This alert is fired when the percentage of successful Radius Accounting Release responses received is lesser than threshold"
   exit
  exit
RADIUS Authentication Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Authentication Failure threshold.

rule Radius_Auth_SR 
   expression "sum by (namespace) (increase(Radius_Requests_Statistics{radMsgCode=\"SecondaryAuthenReq\", procType=\"PDU Session Establishment\", radPacketType=\"Rx\", radResult=\"Success\"}[5m])) / sum by (namespace) (increase(Radius_Requests_Statistics{radMsgCode=\"SecondaryAuthenReq\", procType=\"PDU Session Establishment\", radPacketType=\"Tx\"}[5m])) < 0.80"
   severity   major
   type       "Communications Alarm"
   annotation summary
     value "This alert is fired when the percentage of successful Radius Authentication Request responses received is lesser than threshold"
   exit
  exit
RADIUS Disconnect Message Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Disconnect Message Failure threshold. 

rule Radius_Disconnect_Message_SR 
   expression "sum by (namespace) (increase(Radius_CoaDM_Requests_Statistics{radMsgCode=\"DisconnectACK\", radPacketType=\"Tx\", radResult=\"Success\"}[5m])) / sum by (namespace)(increase(Radius_CoaDM_Requests_Statistics{radMsgCode=\"DisconnectRequest\", radPacketType=\"Rx\"}[5m])) < 0.80"
   severity   major
   type       "Communications Alarm"
   annotation summary
     value "This alert is fired when the percentage of successful Disconnect Message (DM) responses sent is lesser than threshold"
   exit
  exit
exit
RADIUS Server RTT Alert

Use the following example to configure alerts related to RADIUS server RTT.

rule Radius_Server_RTT 
  expression "sum by (namespace, radSvrIP, radSvrPort) (Radius_Server_Rtt_ms{radSvrPortType=~\"Auth|Acct\"} > 5)"
duration   15m
  severity   warning
  type       "Communications Alarm"
  annotation summary
   value "\"RTT for Radius Server: {{ $labels.radSvrIP }}, Port: {{ $labels.radSvrPort }} in namespace: {{ $labels.namespace }} is more than 5 ms.\""
  exit
 exit
RADIUS Accounting Start Initial Message Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Accounting Start Initial Message Failure threshold.

rule Radius_Acct_Start_SR 
   expression "sum by (namespace) (increase(radius_accounting_message_stats{procedure_type=\"radius_initial\", status=\"success\"}[5m])) / sum by (namespace) (increase(radius_accounting_message_stats{procedure_type=\"radius_initial\", status=\"attempted\"}[5m])) < 0.80"
   severity   major
   type       "Processing Error Alarm"
   annotation summary
     value "This service based alert is fired when the percentage of successful Radius Accounting Start successful response received is lesser than threshold"
   exit
  exit
RADIUS Accounting Interim/Update Message Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Accounting Interim/Update Message Failure threshold.

rule Radius_Acct_Interim_SR 
   expression "sum by (namespace) (increase(radius_accounting_message_stats{procedure_type=\"radius_update\", status=\"success\"}[5m])) / sum by (namespace) (increase(radius_accounting_message_stats{procedure_type=\"radius_update\", status=\"attempted\"}[5m])) < 0.80"
   severity   major
   type       "Processing Error Alarm"
   annotation summary
     value "This service based alert is fired when the percentage of successful Radius Accounting Interim Update successful response received is lesser than threshold"
   exit
  exit
RADIUS Accounting Stop/Terminate Message Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Accounting Stop/Terminate Message Failure threshold. 

rule Radius_Acct_Stop_SR 
   expression "sum by (namespace) (increase(radius_accounting_message_stats{procedure_type=\"radius_terminate\", status=\"success\"}[5m])) / sum by (namespace) (increase(radius_accounting_message_stats{procedure_type=\"radius_terminate\", status=\"attempted\"}[5m])) < 0.80"
   severity   major
   type       "Processing Error Alarm"
   annotation summary
     value "This service based alert is fired when the percentage of successful Radius Accounting Stop successful response received is lesser than threshold"
   exit
  exit
RADIUS Authentication Type Message Failure Threshold Alert

Use the following example to configure alerts related to RADIUS Authentication Type Message Failure threshold.

rule Radius_Auth_Type_SR 
   expression "sum by (namespace, radius_auth_algorithm) (increase(radius_authentication_message_stats{radius_auth_algorithm=\"radius_auth_algorithm_.*\", status=\"success\"}[1m])) / sum by (namespace) (increase(radius_authentication_message_stats{radius_auth_algorithm=\"radius_auth_algorithm_.*\", status=\"attempted\"}[1m])) < 0.80"
   severity   major
   type       "Processing Error Alarm"
   annotation summary
     value "This alert is fired when the percentage of successful Radius Auth Type response received is lesser than threshold"
   exit
  exit
Roamer UE Alerts

This section describes the alerts supported for roamer UEs. These alerts can be enhanced per RAT based or as per the intent of the end user.

In-roamer UE Failure Threshold Alert

Use the following example to configure alerts related to In-roamer UE Failure Threshold.

alerts rules group RoamerUEs
 rule In-Roamer_SR
  expression "sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", roaming_status=\"visitor-lbo\", rat_type!=\"\", status=\"Success\"}[5m])) / sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", roaming_status=\"visitor-lbo\", rat_type!=\"\", status=\"attempted\"}[5m])) < 0.10"
  severity major
  type "Communications Alarm"
  annotation summary
   value "This alert is fired when the percentage of successful InRoamer is lesser than threshold"
  exit
exit
Out-roamer UE Failure Threshold Alert

Use the following example to configure alerts related to Out-roamer UE Failure Threshold.

rule Radius_Acct_Release_SR
   rule Out-Roamer_SR
  expression "sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", roaming_status=\"roamer\", rat_type!=\"\", status=\"Success\"}[5m])) / sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", roaming_status=\"roamer\", rat_type!=\"\", status=\"attempted\"}[5m])) < 0.10"
  severity major
  type "Communications Alarm"
  annotation summary 
   value "This alert is fired when the percentage of successful InRoamer is lesser than threshold"
  exit
exit
Option-3x: DCNR Enabled UE Alerts

This section describes the alerts supported for DCNR enabled UEs with presence-reporting enabled. These alerts can be enhanced per 4G procedure based or as per the intent of the end user.

Examples of DCNR statistics/gauges: pdn_sess_create, pdn_inter_sgw_handover, pdn_mbr, pcf_req_ded_brr_mod, pcf_req_ded_brr_create, pcf_req_ded_brr_delete, delete_session_request, smf_initiated_pdn_detach, ue_req_pdn_sess_rel, and so on.

DCNR UE Attach Failure Threshold Alert

Use the following example to configure alerts related to DCNR UE Attach Failure Threshold.

alerts rules group DCNRUEs
 rule DCNR_UE_SR
  expression "sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", dcnr=\"enable\", rat_type!=\"EUTRA\", status=\"success\", procedure_type=\"pdn_sess_create\"}[5m])) / sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", dcnr =\"enable\", rat_type!=\"EUTRA\", status=\"attempted\", procedure_type=\"pdn_sess_create\"}[5m])) < 0.10"
  severity   major
  type       "Communications Alarm"
  annotation summary
   value "This alert is fired when attach success rate of DCNR enabled UE lesser than threshold"
  exit
 exit
DCNR UE Attach Failure Threshold Alert with Presence Reporting

Use the following example to configure alerts related to DCNR UE Attach Failure Threshold with presence reporting enabled. 

rule DCNR_UE_PRA_ENABLE_SR
  expression "sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", dcnr=\"enable\", rat_type!=\"EUTRA\", status=\"success\", procedure_type=\"pdn_sess_create\", pra=\"enable\"}[5m])) / sum by (namespace) (increase(smf_service_stats{app_name=\"smf\", dcnr =\"enable\", rat_type!=\"EUTRA\", status=\"attempted\", procedure_type=\"pdn_sess_create\", pra=\"enable\"}[5m])) < 0.10"
  severity   major
  type       "Communications Alarm"
  annotation summary
   value "This alert is fired when attach success rate of DCNR enabled UE and presence reporting enabled lesser than threshold "
  exit
 exit
show alerts

This section provides the sample output for different variants of the show alerts CLI command.

show alerts | include radius
alerts history radius_test cfb253587397
alerts history radius_test 911f84aff47c
alerts history radius_test 3ed7a5112905
alerts history radius_test 292af807b299
 source      radius-ep-n0-0
 labels      [ "namespace: smf" "pod: radius-ep-n0-0" ]
 annotations [ "summary: Container:  of pod: radius-ep-n0-0 in namespace: smf has been restarted." ]
 source      radius-ep-n0-0
 labels      [ "name: k8s_radius-ep_radius-ep-n0-0_smf_7f9e968a-39dc-11eb-ba84-0050569cb367_0" "namespace: smf" "pod: radius-ep-n0-0" ]
 annotations [ "summary: Container: k8s_radius-ep_radius-ep-n0-0_smf_7f9e968a-39dc-11eb-ba84-0050569cb367_0 of pod: radius-ep-n0-0 in namespace: smf has been restarted." ]
 source      radius-ep-n0-0
 labels      [ "name: k8s_POD_radius-ep-n0-0_smf_7f9e968a-39dc-11eb-ba84-0050569cb367_0" "namespace: smf" "pod: radius-ep-n0-0" ]
 annotations [ "summary: Container: k8s_POD_radius-ep-n0-0_smf_7f9e968a-39dc-11eb-ba84-0050569cb367_0 of pod: radius-ep-n0-0 in namespace: smf has been restarted." ]
alerts history radius_test 1c17e31c13f9
alerts history radius_test ffaabfce0929
 source      radius-ep-n0-0
 labels      [ "name: k8s_POD_radius-ep-n0-0_smf_cd16807a-2f0b-11eb-ba84-0050569cb367_0" "namespace: smf" "pod: radius-ep-n0-0" ]
 annotations [ "summary: Container: k8s_POD_radius-ep-n0-0_smf_cd16807a-2f0b-11eb-ba84-0050569cb367_0 of pod: radius-ep-n0-0 in namespace: smf has been restarted." ]
 source      radius-ep-n0-0
 labels      [ "namespace: smf" "pod: radius-ep-n0-0" ]
 annotations [ "summary: Container:  of pod: radius-ep-n0-0 in namespace: smf has been restarted." ]
 source      radius-ep-n0-0
 labels      [ "name: k8s_radius-ep_radius-ep-n0-0_smf_cd16807a-2f0b-11eb-ba84-0050569cb367_0" "namespace: smf" "pod: radius-ep-n0-0" ]
 annotations [ "summary: Container: k8s_radius-ep_radius-ep-n0-0_smf_cd16807a-2f0b-11eb-ba84-0050569cb367_0 of pod: radius-ep-n0-0 in namespace: smf has been restarted." ]
[unknown] cee#
show alerts active detail | include Radius
alerts active detail Radius_Server_Down 0fe030aba3ce
 summary  "Radius Server: 198.18.1.20, Port: 1813 in namespace: smf is DOWN for more than 15min."
alerts active detail Radius_Server_Down 6f41c340311c
 summary  "Radius Server: 10.84.115.45, Port: 1813 in namespace: smf is DOWN for more than 15min."
alerts active detail Radius_Server_Down 8a290c5ed1de
 summary  "Radius Server: 198.18.1.20, Port: 1812 in namespace: smf is DOWN for more than 15min."
[unknown] cee#
[unknown] cee# 
alerts active detail Radius_Server_Down 0fe030aba3ce
 severity major
 type     "Processing Error Alarm"
 startsAt 2020-12-11T13:30:16.874Z
 source   System
 summary  "Radius Server: 198.18.1.20, Port: 1813 in namespace: smf is DOWN for more than 15min."
 labels   [ "namespace: smf" "radSvrIP: 198.18.1.20" "radSvrPort: 1813" ]
alerts active detail Radius_Server_Down 6f41c340311c
 severity major
 type     "Processing Error Alarm"
 startsAt 2020-12-11T13:30:16.874Z
 source   System
 summary  "Radius Server: 10.84.115.45, Port: 1813 in namespace: smf is DOWN for more than 15min."
 labels   [ "namespace: smf" "radSvrIP: 10.84.115.45" "radSvrPort: 1813" ]
alerts active detail Radius_Server_Down 8a290c5ed1de
 severity major
 type     "Processing Error Alarm"
 startsAt 2020-12-11T13:30:16.874Z
 source   System
 summary  "Radius Server: 198.18.1.20, Port: 1812 in namespace: smf is DOWN for more than 15min."
 labels   [ "namespace: smf" "radSvrIP: 198.18.1.20" "radSvrPort: 1812" ]

[unknown] cee# show alerts active summary | include RTT 
Radius_Server_RTT      1d0353b3db82  major     12-11T15:10:16  System                 RTT for Radius Server: 198.18.1.20, Port: 1812 in namespace: smf is more than 5 ms.                                                                
[unknown] cee#
show alerts active summary | include RTT
Radius_Server_RTT      1d0353b3db82  major     12-11T15:10:16  System                 RTT for Radius Server: 198.18.1.20, Port: 1812 in namespace: smf is more than 5 ms.                                                                
[unknown] cee#
show alerts active summary | include Radius
Radius_Server_RTT      1d0353b3db82  major     12-11T15:10:16  System                 RTT for Radius Server: 198.18.1.20, Port: 1812 in namespace: smf is more than 5 ms.                                                                
Radius_Acct_Establish  520d9943d53f  major     12-11T15:05:16  System                 This alert is fired when the percentage of successful Radius Accounting Establish responses received is lesser than threshold                      
Radius_Server_Down     0fe030aba3ce  major     12-11T13:30:16  System                 Radius Server: 198.18.1.20, Port: 1813 in namespace: smf is DOWN for more than 15min.                                                              
Radius_Server_Down     6f41c340311c  major     12-11T13:30:16  System                 Radius Server: 10.84.115.45, Port: 1813 in namespace: smf is DOWN for more than 15min.                                                             
Radius_Server_Down     8a290c5ed1de  major     12-11T13:30:16  System                 Radius Server: 198.18.1.20, Port: 1812 in namespace: smf is DOWN for more than 15min.
DCNR UE Bulk Statistics

Use the following SMF service bulk statistics to monitor the failures or issues associated with DCNR UEs.

Table 16. DCNR UE
Bulk Statistics Name

Query

 Description

4G_DCNRUE_Attempted

bulk-stats query 4G_DCNRUE_Attempted expression "sum(smf_service_stats {dcnr='enable',status='attempted', rat_type='EUTRA'}) by (namespace)" exit

4G_DCNRUE_Success

bulk-stats query 4G_DCNRUE_Success expression "sum(smf_service_stats {dcnr='enable',status='success', rat_type='EUTRA'}) by (namespace)" exit

4G_PRA_ENABLE_Attempted

bulk-stats query 4G_PRA_ENABLE_Attempted expression "sum(smf_service_stats {pra='enable',status='attempted', rat_type='EUTRA', procedure_type!='create_session_request'}) by (namespace)" exit

4G_PRA_ENABLE_Success

bulk-stats query 4G_PRA_ENABLE_Success expression "sum(smf_service_stats {pra='enable',status='success', rat_type='EUTRA', procedure_type!='create_session_request'}) by (namespace)" exit

RADIUS Bulk Statistics

Use the following bulk statistics to monitor the failures or issues associated with RADIUS authentication, RADIUS accounting, and Disconnect Message requests.

Table 17. RADIUS Accounting Message (Per endpoint)
Bulk Statistics Name

Query

 Description

Radius_EP_Acct_Establish _Attempt

bulk-stats query Radius_EP_Acct_Establish_Attempt expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDN Connect|PDU Session Establishment', radPacketType='Tx'}) by (namespace,dnn)"

label dnn

exit

 Total number of Accounting Requests attempted by MVNO subscribers during session establishment.

Radius_EP_Acct_Establish _Success

bulk-stats query Radius_EP_Acct_Establish_Success expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDN Connect|PDU Session Establishment', radPacketType='Rx', radResult='Success',dnn!=""}) by (namespace,dnn)"

label dnn

exit

 Total number of successful Accounting Requests from MVNO subscribers during session establishment.

Radius_EP_Acct_Establish_Failure

bulk-stats query Radius_EP_Acct_Establish_Failure expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDN Connect|PDU Session Establishment', radPacketType='Rx', radResult!='Success'}) by (namespace,dnn)"

label dnn

exit

 Total number of failed Accounting Requests from MVNO subscribers during session establishment.

Radius_EP_Acct_Release_Attempt

bulk-stats query Radius_EP_Acct_Release_Attempt expression "sum (Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDU Session Release.*|PDN Disconnect', radPacketType='Tx'}) by (namespace,dnn)"

label dnn

exit

 Total number of Accounting Requests attempted by MVNO subscribers during session release.

Radius_EP_Acct_Release_Success

bulk-stats query Radius_EP_Acct_Release_Success expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDU Session Release.*|PDN Disconnect', radPacketType='Rx', radResult='Success'}) by (namespace,dnn)"

label dnn

exit

 Total number of successful Accounting Requests from MVNO subscribers during session release.

Radius_EP_Acct_Release_Failure

bulk-stats query Radius_EP_Acct_Release_Failure expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDU Session Release.*|PDN Disconnect', radPacketType='Rx', radResult!='Success'}) by (namespace,dnn)"

label dnn

exit

 Total number of failed Accounting Requests from MVNO subscribers during session release.

Radius_EP_Acct_Release_Type _Attempt

bulk-stats query Radius_EP_Acct_Release_Type_Attempt expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDU Session Release.*|PDN Disconnect', radPacketType='Tx'}) by (namespace, procType)"

label procType

exit

 Total number of Accounting Requests attempted by MVNO subscribers for a particular release type.

Radius_EP_Acct_Release_Type _Success

bulk-stats query Radius_EP_Acct_Release_Type_Success expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDU Session Release.*|PDN Disconnect', radPacketType='Rx', radResult!='Success'}) by (namespace, procType)"

label procType

exit

 Total number of successful Accounting Requests from MVNO subscribers for a particular release type.

Radius_EP_Acct_Release_Type _Failure

bulk-stats query Radius_EP_Acct_Release_Type_Failure expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDU Session Release.*|PDN Disconnect', radPacketType='Rx', radResult!='Success'}) by (namespace, procType)"

label procType

exit

 Total number of failed Accounting Requests from MVNO subscribers for a particular release type.

Radius_EP_Acct_Establish _Retry_Attempt

bulk-stats query Radius_EP_Acct_Establish _Retry_Attempt expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDN Connect|PDU Session Establishment', radPacketType='Retry_Tx'}) by (namespace,dnn)"

label dnn

exit

 Total number of Accounting Retry Requests attempted by MVNO subscribers during session establishment.

Radius_EP_Acct_Release_Retry _Attempt

bulk-stats query Radius_EP_Acct_Release _Retry_Attempt expression "sum(Radius_Requests_Statistics {radMsgCode='RadiusAcctReq', procType=~'PDU Session Release.*|PDN Disconnect', radPacketType='Retry_Tx'}) by (namespace,dnn)"

label dnn

exit

 Total number of Accounting Retry Requests attempted by MVNO subscribers during session release.
Table 18. RADIUS Accounting Message (Per SMF service)
Bulk Statistics Name

Query

 Description

Radius_Acct_Initial_Attempt_Svc

bulk-stats query Radius_Acct_Initial_Attempt _Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_initial', status='attempted'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Initial/Start Attempt from service endpoint.

Radius_Acct_Initial_Success_Svc

bulk-stats query Radius_Acct_Initial_Success_Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_initial', status='success'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Initial/Start Success from service endpoint.

Radius_Acct_Initial_Failure_Svc

bulk-stats query Radius_Acct_Initial_Failure_Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_initial', status!~'success|attempted'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Initial/Start Failure from service endpoint.

Radius_Acct_Terminate _Attempt_Svc

bulk-stats query Radius_Acct_Terminate_Attempt _Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_terminate', status='attempted'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Terminate/Stop Attempt from service endpoint.

Radius_Acct_Terminate _Success_Svc

bulk-stats query Radius_Acct_Terminate _Success _Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_terminate', status='success'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Terminate/Stop Success from service endpoint.

Radius_Acct_Terminate _Failure_Svc

bulk-stats query Radius_Acct_Terminate_Failure _Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_terminate', status!~'success|attempted'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Terminate/Stop Failure from service endpoint.

Radius_Acct_Update _Attempt_Svc

bulk-stats query Radius_Acct_Update_Attempt _Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_update', status='attempted'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Update/Interim Attempt from service endpoint.

Radius_Acct_Update _Success_Svc

bulk-stats query Radius_Acct_Update_Success _Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_update', status='success'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Update/Interim Success from service endpoint.

Radius_Acct_Update _Failure_Svc

bulk-stats query Radius_Acct_Update_Failure _Svc expression "sum (radius_accounting_message_stats {procedure_type='radius_update', status!~'success|attempted'}) by (namespace)"

exit

 Total number of subscribers with RADIUS Accounting Update/Interim Failure from service endpoint.
Table 19. Disconnect Message
Bulk Statistics Name

Query

 Description
Radius_EP_DM_Attempt

bulk-stats query Radius_EP_DM_Attempt expression "sum (Radius_CoaDM_Requests_Statistics {radMsgCode='DisconnectRequest', radPacketType='Rx'}) by (namespace,radSvrIP)"

label radSvrIP

exit

 Total number of RADIUS Disconnect Message Attempts
Radius_EP_DM_Success

bulk-stats query Radius_EP_DM_Attempt expression "sum (Radius_CoaDM_Requests_Statistics {radMsgCode='DisconnectRequest', radPacketType='Rx'}) by (namespace,radSvrIP)"

label radSvrIP

exit

 Total number of successful RADIUS Disconnect Messages
Radius_EP_DM_Failure

bulk-stats query Radius_EP_DM_Failure expression "sum (Radius_CoaDM_Requests_Statistics {radMsgCode='DisconnectNAK', radPacketType='Tx', radResult='Success'}) by (namespace,radSvrIP)"

label radSvrIP

exit

 Total number of failed RADIUS Disconnect Messages
Table 20. RADIUS Access Request (Secondary Authentication)
Bulk Statistics Name

Query

 Description

Radius_EP_AccessReq_Attempt

bulk-stats query Radius_EP_AccessReq_Attempt expression "sum(Radius_Requests_Statistics {radMsgCode='SecondaryAuthenReq', procType!='', radPacketType='Tx', radResult='Success'}) by (namespace,dnn)"

label dnn

exit

 Total number of MVNO subscribers with Access Request (secondary authentication) Attempts

Radius_EP_AccessReq_Success

bulk-stats query Radius_EP_AccessReq_Success expression "sum(Radius_Requests_Statistics {radMsgCode='SecondaryAuthenReq', procType!='', radPacketType='Rx', radResult='Success'}) by (namespace,dnn)"

label dnn

exit

 Total number of MVNO subscribers with successful Access Requests (secondary authentication)

Radius_EP_AccessReq_Failure

bulk-stats query Radius_EP_AccessReq_Failure expression "sum(Radius_Requests_Statistics {radMsgCode='SecondaryAuthenReq', radPacketType='Rx', radResult!='Success'}) by (namespace,dnn)"

label dnn

exit

 Total number of MVNO subscribers with failed Access Requests (secondary authentication)
Roamer UE Bulk Statistics

Use the following SMF service bulk statistics to monitor the failures or issues associated with Roamer UEs.

Table 21. Roamer UE
Bulk Statistics Name

Query

 Description

4G_In-Roamers_Attempted

bulk-stats query 4G_In-Roamers_Attempted expression "sum(smf_service_stats {roaming_status='visitor-lbo', status='attempted',rat_type='EUTRA'}) by (namespace)" exit

4G_In-Roamers_Success

bulk-stats query 4G_In-Roamers_Success expression "sum(smf_service_stats {roaming_status='visitor-lbo', status='success',rat_type='EUTRA'}) by (namespace)" exit

4G_Out-Roamers_Attempted

bulk-stats query 4G_Out-Roamers_Attempted expression "sum(smf_service_stats {roaming_status='roamer', status='attempted',rat_type='EUTRA'}) by (namespace)" exit

4G_Out-Roamers_Success

bulk-stats query 4G_Out-Roamers_Success expression "sum(smf_service_stats {roaming_status='roamer', status='success',rat_type='EUTRA'}) by (namespace)" exit

5G_In-Roamers_Attempted

bulk-stats query 5G_In-Roamers_Attempted expression "sum(smf_service_stats {roaming_status='visitor-lbo', status='attempted',rat_type='NR'}) by (namespace)" exit

5G_In-Roamers_Success

bulk-stats query 5G_In-Roamers_Success expression "sum(smf_service_stats {roaming_status='visitor-lbo', status='success',rat_type='NR'}) by (namespace)" exit

5G_Out-Roamers_Attempted

bulk-stats query 5G_Out-Roamers_Attempted expression "sum(smf_service_stats {roaming_status='roamer', status='attempted',rat_type='NR'}) by (namespace)" exit

5G_Out-Roamers_Success

bulk-stats query 5G_Out-Roamers_Success expression "sum(smf_service_stats {roaming_status='roamer', status='success',rat_type='NR'}) by (namespace)" exit

WiFi_In-Roamers_Attempted

bulk-stats query WiFi_In-Roamers_Attempted expression "sum(smf_service_stats {roaming_status='visitor-lbo', status='attempted',rat_type='WLAN'}) by (namespace)" exit

WiFi_In-Roamers_Success

bulk-stats query WiFi_In-Roamers_Success expression "sum(smf_service_stats {roaming_status='visitor-lbo', status='success',rat_type='WLAN'}) by (namespace)" exit

WiFi_Out-Roamers_Attempted

bulk-stats query WiFi_Out-Roamers_Attempted expression "sum(smf_service_stats {roaming_status='roamer', status='attempted',rat_type='WLAN'}) by (namespace)" exit

WiFi_Out-Roamers_Success

bulk-stats query WiFi_Out-Roamers_Success expression "sum(smf_service_stats {roaming_status='roamer', status='success',rat_type='WLAN'}) by (namespace)" exit

Show Bulk Statistics

Use the show bulk-stats command to view the list of all configured bulk statistics.

RADIUS Bulk Statistics

The following is a sample output of the bulk stats related to the RADIUS feature.


[unknown] cee# show bulk-stats | include Radius
smf        Radius_EP_Acct_Establish_Attempt             System  19.0   
smf        Radius_EP_Acct_Establish_Failure             System  9.0    
smf        Radius_EP_Acct_Establish_Retry_Attempt       System  11.0   
smf        Radius_EP_Acct_Establish_Success             System  10.0   
smf        Radius_EP_Acct_Release_Attempt               System  19.0   
smf        Radius_EP_Acct_Release_Failure               System  18.0   
smf        Radius_EP_Acct_Release_Retry_Attempt         System  37.0   
smf        Radius_EP_Acct_Release_Success               System  1.0    
smf        Radius_EP_Acct_Release_Type_Attempt          System  19.0   
smf        Radius_EP_Acct_Release_Type_Failure          System  18.0   
smf        Radius_EP_Acct_Release_Type_Success          System  1.0    
smf        Radius_EP_DM_Attempt                         System  14.0   
smf        Radius_EP_DM_Failure                         System  9.0    
smf        Radius_EP_DM_Success                         System  5.0    
smf        Radius_EP_SecondaryAuth_Attempt              System  16.0   
smf        Radius_EP_SecondaryAuth_Success              System  16.0   

[unknown] cee# show bulk-stats current | include Radius | exclude smf-data | exclude smf-ims 
smf        Radius_Acct_Initial_Attempt_Svc              System  3.0    
smf        Radius_Acct_Initial_Success_Svc              System  3.0    
smf        Radius_Acct_Terminate_Attempt_Svc            System  3.0    
smf        Radius_Acct_Terminate_Success_Svc            System  3.0    
smf        Radius_EP_Acct_Establish_Attempt             System  3.0    
smf        Radius_EP_Acct_Establish_Failure             System  1.0    
smf        Radius_EP_Acct_Establish_Retry_Attempt       System  3.0    
smf        Radius_EP_Acct_Establish_Success             System  2.0    
smf        Radius_EP_Acct_Release_Attempt               System  3.0    
smf        Radius_EP_Acct_Release_Failure               System  2.0    
smf        Radius_EP_Acct_Release_Retry_Attempt         System  6.0    
smf        Radius_EP_Acct_Release_Success               System  1.0    
smf        Radius_EP_Acct_Release_Type_Attempt          PDU Session Release - AMF initiated 1.0    
smf        Radius_EP_Acct_Release_Type_Attempt          PDU Session Release - RADIUS initiated 1.0    
smf        Radius_EP_Acct_Release_Type_Attempt          PDU Session Release - Admin initiated 1.0    
smf        Radius_EP_Acct_Release_Type_Failure          PDU Session Release - AMF initiated 1.0    
smf        Radius_EP_Acct_Release_Type_Failure          PDU Session Release - Admin initiated 1.0    
smf        Radius_EP_Acct_Release_Type_Success          PDU Session Release - AMF initiated 1.0    
smf        Radius_EP_Acct_Release_Type_Success          PDU Session Release - Admin initiated 1.0    
smf        Radius_EP_DM_Attempt                         System  2.0    
smf        Radius_EP_DM_Failure                         System  1.0    
smf        Radius_EP_DM_Success                         System  1.0    
smf        Radius_EP_SecondaryAuth_Attempt              System  2.0    
smf        Radius_EP_SecondaryAuth_Success              System  2.0
Roaming Bulk Statistics

The following is a sample output for the config based bulkstats related to the Roaming feature.

[unknown] cee# show bulk-stats | include Roam | include cnsgw
cnsgw      4G_In-Roamers_Attempted     System                                0.0    
cnsgw      4G_In-Roamers_Success       System                                0.0    
cnsgw      4G_Out-Roamers_Attempted    System                                0.0    
cnsgw      5G_In-Roamers_Attempted     System                                0.0    
cnsgw      5G_In-Roamers_Success       System                                0.0    
cnsgw      5G_Out-Roamers_Attempted    System                                0.0    
cnsgw      5G_Out-Roamers_Success      System                                0.0    
cnsgw      WiFi_In-Roamers_Attempted   System                                0.0    
cnsgw      WiFi_In-Roamers_Success     System                                0.0    
cnsgw      WiFi_Out-Roamers_Attempted  System                                3.0    
cnsgw      WiFi_Out-Roamers_Success    System                                4.0    
[unknown] cee#
Option-3x Bulk Statistics

The following is a sample output for the config based bulkstats related to the Option-3x feature.


[unknown] cee# show bulk-stats | include DCNR
smf        4G_DCNRUE_Attempted                 System  147.0  
smf        4G_DCNRUE_Success                   System  130.0  
[unknown] cee# 
[unknown] cee# show bulk-stats | include PRA               
smf        4G_PRA_ENABLE_Attempted             System  107.0  
smf        4G_PRA_ENABLE_Success               System  107.0  
[unknown] cee#

Grafana Charts

The Grafana charts can be used for monitoring based on the RADIUS endpoint or Service endpoint.

  • RADIUS endpoint for call flows involving RADIUS Authentication, Accounting, and Disconnect Message.

  • Service endpoint for accounting flows specific to Accounting Initial/Interim/Terminate packets.

Error Logs

This section explains the basic error conditions and the related logs to debug the failures.

Option-3x Feature

This section provides the basic error conditions and logs that are captured to debug the failures for the roaming feature.

DCNR Disabled UE or 4G capable UE only

The following example displays the error log for DCNR disabled UE or 4G capable UE only. 

2021/01/24 10:11:22.648 smf-service [ERROR] [GenericGtpMsg.go:1811] [smf-service.smf-app.sgw] secRatUsageRpt recvd wrongly for DNCR disbled UE, ignoring report

2020/12/19 13:17:17.834 smf-service [ERROR] [GenericGtpMsg.go:1834] [smf-service.smf-app.sgw] secRatUsageRpt recvd wrongly for 4gOnly UE, ignoring report
Secondary RAT Usage with invalid EBI

The following example displays the error log for secondary RAT usage with invalid EBI. 

2021/01/24 11:38:18.917 smf-service [DEBUG] [GenericGtpMsg.go:1824] [smf-service.smf-app.sgw] Secondary Rat Data Usage Report Recvd
2021/01/24 11:38:18.917 smf-service [WARN] [IntMethods.go:137] [smf-service.Policy.gen] Qos Flow not found with EBI [8]
2021/01/24 11:38:18.917 smf-service [ERROR] [GenericGtpMsg.go:1853] [smf-service.smf-app.sgw]  Qfi invalid in secRatUsageRpt
Secondary RAT Usage invalid RAT Type

The following example displays the error log for secondary RAT usage with invalid RAT type. 

2021/01/24 11:42:21.474 smf-service [DEBUG] [GenericGtpMsg.go:1824] [smf-service.smf-app.sgw] Secondary Rat Data Usage Report Recvd
2021/01/24 11:42:21.474 smf-service [ERROR] [GenericGtpMsg.go:1861] [smf-service.smf-app.sgw]  Rat type invalid in secRatUsageRpt
Secondary RAT Usage with intended PGW set to zero

The following example displays the error log for secondary RAT usage with intended P-GW set to zero (IRPGW:0). 

2021/01/24 11:33:10.390 smf-service [DEBUG] [GenericGtpMsg.go:1824] [smf-service.smf-app.sgw] Secondary Rat Data Usage Report Recvd
2021/01/24 11:33:10.390 smf-service [ERROR] [GenericGtpMsg.go:1865] [smf-service.smf-app.sgw]  secRatUsageRpt.IRPGW is false
PRA ID received greater than four

The following example displays the error log when PRA ID received is greater than four.

2021/01/24 14:48:26.085 smf-service [DEBUG] [policy_types.go:659] [smf-service.Policy.gen] praConfig:true for dnn:fast.t-mobile.com
2021/01/24 14:48:26.085 smf-service [DEBUG] [policy_pcf.go:1939] [smf-service.Policy.gen] Added PRA ID: 9388618
2021/01/24 14:48:26.085 smf-service [DEBUG] [policy_pcf.go:1939] [smf-service.Policy.gen] Added PRA ID: 9388608
2021/01/24 14:48:26.085 smf-service [DEBUG] [policy_pcf.go:1939] [smf-service.Policy.gen] Added PRA ID: 8388618
2021/01/24 14:48:26.085 smf-service [DEBUG] [policy_pcf.go:1939] [smf-service.Policy.gen] Added PRA ID: 9388619
2021/01/24 14:48:26.085 smf-service [WARN] [policy_pcf.go:1934] [smf-service.Policy.gen] Max 4 PRAs allowed, ignoring the PRA-ID (8388608) from PCF
2021/01/24 14:48:26.085 smf-service [WARN] [policy_pcf.go:1934] [smf-service.Policy.gen] Max 4 PRAs allowed, ignoring the PRA-ID (8388619) from PCF
RADIUS Authentication
Authentication Request Not Responded by Server

The following is an error log for RADIUS Authentication Request not responded by the RADIUS server.

[smf-service-n0-0] 2020/09/17 07:14:52.921 smf-service [ERROR] [GenericAAA.go:786] [smf-service0.smf-app.aaa] [imsi-123456789012345:5] [imsi-123456789012345:5] [16] Secondary Authentication Failed: TIMEOUT 
[smf-service-n0-0] *errors.errorString Secondary Authentication Failed: TIMEOUT 
[smf-service-n0-0] /opt/workspace/smf-service/src/smf-service/vendor/wwwin-github.cisco.com/
mobile-cnat-golang-lib/app-infra.git/src/app-infra/infra/Transaction.go:621 (0xd89cae) 
[smf-service-n0-0] /opt/workspace/smf-service/src/smf-service/procedures/generic/GenericAAA.go:786 (0x144fa52)
Call Failure at Authentication Stage

The following is a sample error log for call failure at the RADIUS authentication stage.

[smf-service-n0-0]  
[smf-service-n0-0] 2020/09/17 07:14:52.921 smf-service [ERROR] [idlestate.go:504] [smf-service0.smf-app.aaa] [imsi-123456789012345:5] [imsi-123456789012345:5] [16] USER_AUTHENTICATION_OR_AUTHORIZATION_FAILED 
[smf-service-n0-0] *errors.errorString USER_AUTHENTICATION_OR_AUTHORIZATION_FAILED 
[smf-service-n0-0] /opt/workspace/smf-service/src/smf-service/vendor/wwwin-github.cisco.com/
mobile-cnat-golang-lib/app-infra.git/src/app-infra/infra/Transaction.go:621 (0xd89cae) 
[smf-service-n0-0] /opt/workspace/smf-service/src/smf-service/vendor/wwwin-github.cisco.com/
mobile-cnat-golang-lib/app-infra.git/src/app-infra/infra/Transaction.go:580 (0x15d7ddc) 
[smf-service-n0-0] /opt/workspace/smf-service/src/smf-service/procedures/4g/pdnsetup/idlestate.go:537 (0x15bc4f5)
Authentication Request Rejected by RADIUS Server

The following is an error log for RADIUS Authentication Request rejected by RADIUS server.

[smf-service-n0-0] 2020/12/09 09:20:14.047 smf-service [INFO] [idlestate.go:649] [smf-service.smf-app.aaa] [imsi-123456789012345:5] [imsi-123456789012345:5] [1] Processing Secondary Authentication Response 
[smf-service-n0-0] 2020/12/09 09:20:14.047 smf-service [ERROR] [GenericAAA.go:1173] [smf-service.smf-app.aaa] [imsi-123456789012345:5] [imsi-123456789012345:5] [1] Secondary Authentication Failed: REJECT 
[smf-service-n0-0] 2020/12/09 09:20:14.047 smf-service [DEBUG] [Genericutil.go:681] [smf-service.smf-app.gen] Internal Transaction Submit with BP for MessageType: 118, SLA: 0 
[smf-service-n0-0] 2020/12/09 09:20:14.047 smf-service [DEBUG] [idlestate.go:169] [smf-service.smf-app.gen] inCallStatus:9 

***********************************************
Transaction Log received from Instance: smf.radius-ep.ajay-smf1.smf.0
************* TRANSACTION: 00004 *************
TRANSACTION SUCCESS:
    Txn Type             : SecondaryAuthenReq(2004)
    Priority             : 1
    Session State        : No_Session
LOG MESSAGES:
    2020/12/09 09:20:13.756 [TRACE] [infra.message_log.core] >>>>>>>>

    2020/12/09 09:20:13.757 [DEBUG] [Radius.smf.AAA] Starting smf AccessRequest
    2020/12/09 09:20:13.757 [DEBUG] [Radius.smf.AAA] Starting smf AccessRequest for User [msisdn-9884886688]
    2020/12/09 09:20:13.757 [DEBUG] [Radius.smf.AAA] Created new Radius Message for smf AccessRequest
    2020/12/09 09:20:13.757 [DEBUG] [Radius.smf.AAA] Selected server: 198.18.1.20:1812 , nasIP: 198.18.1.4 PID: 4194304
    2020/12/09 09:20:13.757 [DEBUG] [Radius.smf.AAA] Sending an IPC Message to UDP proxy [198.18.1.4]
    2020/12/09 09:20:13.763 [DEBUG] [Radius.smf.AAA] PID: 4194304 - Response received on channel
    2020/12/09 09:20:13.763 [DEBUG] [Radius.smf.AAA] Authentication Result for user [8899776655] = [REJECT]
    2020/12/09 09:20:13.764 [TRACE] [infra.message_log.core] <<<<<<<<

***********************************************
Authentication Response with Incorrect Authenticator

The following is an error log for RADIUS Authentication Response with incorrect authenticator.

[radius-ep-n0-0] ************* TRANSACTION: 00044 ************* 
[radius-ep-n0-0] TRANSACTION SUCCESS: 
[radius-ep-n0-0]     Txn Type             : RadiusUdpProxyMsg(2002) 
[radius-ep-n0-0]     Priority             : 1 
[radius-ep-n0-0]     Session State        : No_Session 
[radius-ep-n0-0] LOG MESSAGES: 
[radius-ep-n0-0]     2020/12/09 13:20:38.874 [TRACE] [infra.message_log.core] >>>>>>>> 
[radius-ep-n0-0]  
[radius-ep-n0-0]     2020/12/09 13:20:38.874 [DEBUG] [Radius.smf.AAA] Response received from udp proxy 
[radius-ep-n0-0]     2020/12/09 13:20:38.874 [DEBUG] [Radius.smf.AAA] SrcIp: 198.18.1.20 SrcPort: 1812 DestIp: 198.18.1.4 DestPort: 16384 
[radius-ep-n0-0]     2020/12/09 13:20:38.874 [ERROR] [Radius.smf.AAA] PID: 4194310 - Packet dropped due to invalid authenticator 
[radius-ep-n0-0]     2020/12/09 13:20:38.874 [TRACE] [infra.message_log.core] <<<<<<<< 
[radius-ep-n0-0]  
[radius-ep-n0-0] ***********************************************
RADIUS Accounting
Accounting Request Timeout

The following is an error log for RADIUS Accounting Request timeout.

[radius-ep-n0-0] ************* TRANSACTION: 00027 ************* 
[radius-ep-n0-0] TRANSACTION SUCCESS: 
[radius-ep-n0-0]     Txn Type             : IntSmfAcctReqMsg(3) 
[radius-ep-n0-0]     Priority             : 1 
[radius-ep-n0-0]     Session State        : No_Session 
[radius-ep-n0-0] LOG MESSAGES: 
[radius-ep-n0-0]     2020/12/09 13:09:10.247 [TRACE] [infra.message_log.core] >>>>>>>> 
[radius-ep-n0-0]  
[radius-ep-n0-0]     2020/12/09 13:09:10.247 [DEBUG] [Radius.smf.AAA] Starting smf AccountingRequest 
[radius-ep-n0-0]     2020/12/09 13:09:10.247 [DEBUG] [Radius.smf.AAA] Starting smf AccountingRequest for User [msisdn-9884886688] 
[radius-ep-n0-0]     2020/12/09 13:09:10.247 [DEBUG] [Radius.smf.AAA] Created new Radius Message for smf AccountingRequest 
[radius-ep-n0-0]     2020/12/09 13:09:10.247 [DEBUG] [Radius.smf.AAA] Selected server: 198.18.1.20:1813 , nasIP: 198.18.1.4 PID: 4194304 
[radius-ep-n0-0]     2020/12/09 13:09:10.247 [DEBUG] [Radius.smf.AAA] Sending an IPC Message to UDP proxy [198.18.1.4] 
[radius-ep-n0-0]     2020/12/09 13:09:15.091 [DEBUG] [Radius.smf.AAA] PID: 4194304 - Response received on channel 
[radius-ep-n0-0]     2020/12/09 13:09:15.091 [ERROR] [Radius.smf.AAA] Retried MaxNumber of times without success 
[radius-ep-n0-0]     2020/12/09 13:09:15.092 [DEBUG] [Radius.smf.AAA] Int-txn Accounting Result for user [9884886688] = [TIMEOUT] 
[radius-ep-n0-0]     2020/12/09 13:09:15.092 [TRACE] [infra.message_log.core] <<<<<<<< 
[radius-ep-n0-0]  
[radius-ep-n0-0] ***********************************************
Roaming Feature

This section provides the basic error conditions and logs that are captured to debug the failures for the roaming feature.

PLMN Validation Failure

The following example displays the error log for PLMN validation failure resulting into setting the roaming status as "none". 

2021/01/06 15:25:18.630 smf-service [DEBUG] [genericinfo.go:1597] [smf-service.smf-app.subscriber] Set roaming status to 0
2021/01/06 15:25:18.630 smf-service [DEBUG] [genericinfo.go:2317] [smf-service.smf-app.subscriber]  Subscriber is %!s(uint32=0)
2021/01/06 15:25:18.630 smf-service [ERROR] [genericinfo.go:1082] [smf-service.smf-app.subscriber] PLMN validation failed
2021/01/06 15:25:18.630 smf-service [DEBUG] [subscriber_policy_config.go:187] [misc-lib.config.subscriber-policy] LookupParameters - {imsi-123456789012345 msisdn-223310101010101 imei-123456786666660  0 123 456 intershat}
Homer UE Status (Homer)

The following is an example of the generic logs for UE Roaming Status. 

2021/01/06 15:04:39.146 smf-service [DEBUG] [genericinfo.go:1597] [smf-service.smf-app.subscriber] Set roaming status to 1
2021/01/06 15:04:39.146 smf-service [DEBUG] [genericinfo.go:2317] [smf-service.smf-app.subscriber]  Subscriber is %!s(uint32=1)
2021/01/06 15:04:39.146 smf-service [DEBUG] [subscriber_policy_config.go:187] [misc-lib.config.subscriber-policy] LookupParameters - {imsi-123456789012345 msisdn-9999988888 imei-352099001761480 Abf123 2 310 310 intershat}
Out-roamer UE Status (Roamer)

The following is an example of the generic logs for out-roamer UE status. 

2021/01/06 16:11:02.710 smf-service [DEBUG] [genericinfo.go:1597] [smf-service.smf-app.subscriber] Set roaming status to 4
2021/01/06 16:11:02.710 smf-service [DEBUG] [genericinfo.go:2317] [smf-service.smf-app.subscriber]  Subscriber is %!s(uint32=4)
2021/01/06 16:11:02.710 smf-service [DEBUG] [subscriber_policy_config.go:187] [misc-lib.config.subscriber-policy] LookupParameters - {imsi-123456789012345
In-roamer UE Status (Visitor LBO)

The following is an example of the generic logs for in-roamer UE status. 

2021/01/06 15:54:32.323 smf-service [DEBUG] [genericinfo.go:1597] [smf-service.smf-app.subscriber] Set roaming status to 2
2021/01/06 15:54:32.323 smf-service [DEBUG] [genericinfo.go:2317] [smf-service.smf-app.subscriber]  Subscriber is %!s(uint32=2)
2021/01/06 15:54:32.323 smf-service [DEBUG] [subscriber_policy_config.go:187] [misc-lib.config.subscriber-policy] LookupParameters - {imsi-123456789012345 msisdn-223310101010101 imei-123456786666660  0 310 310 intershat}
Generic MVNO Call Flow Error Logs
Incorrect Static IP to Pool Mapping

The following is a sample error log for incorrect static IP to pool mapping or if static IP received from RADIUS is not found with any UPF. 

[smf-service-n0-0] 2020/09/23 07:42:25.969 smf-service [DEBUG] [rmgrutil.go:501] [smf-service.smf-app.resource] [imsi-123456789012345:5] [imsi-123456789012345:5] [16] response received for message NmgrRersourceMgmtResponse 
[smf-service-n0-0] 2020/09/23 07:42:25.969 smf-service [INFO] [upmgrCacheApi.go:450] [misc-lib.upmgrcache.gen] Cache doesnot have entry for UpfEpKey:  
[smf-service-n0-0] 2020/09/23 07:42:25.969 smf-service [ERROR] [rmgrutil.go:73] [smf-service.smf-app.resource] [imsi-123456789012345:5] [imsi-123456789012345:5] [16] Both the associated nodemgr instances for upfEpKey:  is down 
[smf-service-n0-0] *errors.errorString Both the associated nodemgr instances for upfEpKey:  is down 
[smf-service-n0-0] /opt/workspace/smf-service/src/smf-service/vendor/wwwin-github.cisco.com/
mobile-cnat-golang-lib/app-infra.git/src/app-infra/infra/Transaction.go:621 (0xd8b29e) 
[smf-service-n0-0] /opt/workspace/smf-service/src/smf-service/procedures/generic/rmgrutil.go:73 (0x14dbd61)
Idle Timeout-based Release
Idle Timeout Received from RADIUS

The following is a sample error log for idle timeout received from RADIUS.

[smf-service-n0-0] 2020/09/23 16:10:11.965 smf-service [DEBUG] 
[Genericutil.go:7158] [smf-service.smf-app.gen] Idle timeout value received from Radius: 10 
[smf-service-n0-0] 2020/09/23 16:10:11.965 smf-service [DEBUG] 
[Genericutil.go:7168] [smf-service.smf-app.gen] Starting cp idle timer with timeout value: 10
Absolute Session Timeout Received from RADIUS

The following is a sample error log for absolute session timeout received from RADIUS. 

[smf-service-n0-0] 2020/09/23 16:10:11.964 smf-service [DEBUG] 
[Genericutil.go:7200] [smf-service.smf-app.gen] Session absolute timeout value 
received from Radius: 200
Session Cleanup

The following is a sample error log for session cleanup.

[smf-service-n0-0] 2020/09/23 16:10:21.966 smf-service [WARN] [stateHandler.go:187]
[smf-service.smf-app.gen] [imsi-123456789012345:5] [imsi-123456789012345:5] [21] 
TIMEOUT -- Cp Idle Session Timer Expired, Triggering release
Disconnect Message
Disconnect Message Received from Unknown Client

The following is a sample error log when disconnect message is received from an unknown client. 

[radius-ep-n0-0]  
[radius-ep-n0-0] 2020/11/25 10:30:02.960 radius-ep [INFO] [processor.go:157] [Radius.smf.Ipc] Process continue - 2003 
[radius-ep-n0-0] 2020/11/25 10:30:02.960 radius-ep [DEBUG] [coa.go:23] [Radius.smf.AAA] [] [] [11] Coa/Disconnect Req received from udp proxy 
[radius-ep-n0-0] 2020/11/25 10:30:02.960 radius-ep [DEBUG] [coa.go:43] [Radius.smf.AAA] [] [] [11] SrcIp: 198.18.1.20 SrcPort: 3799 DestIp: 198.18.1.4 DestPort: 3799 
[radius-ep-n0-0] 2020/11/25 10:30:02.960 radius-ep [ERROR] [coa.go:253] [Radius.smf.Ipc] Bng Coa/Disconnect req failed - Invalid Coa Client 198.18.1.20 
[radius-ep-n0-0] 2020/11/25 10:30:02.960 radius-ep [ERROR] [coa.go:48] [Radius.smf.AAA] [] [] [11] Unable to process Coa/Disconnect request - Error during init of Radius Message Invalid Coa Client 198.18.1.20 
[radius-ep-n0-0] 2020/11/25 10:30:02.960 radius-ep [DEBUG] [MasterBlueprint.go:345] [infra.transaction.core] Last stage ( init_done ) -> Next stage ( finished ) 
[radius-ep-n0-0] ************* TRANSACTION: 00011 ************* 
[radius-ep-n0-0] TRANSACTION SUCCESS: 
[radius-ep-n0-0]     Txn Type             : RadiusUdpProxyCoaMsg(2003) 
[radius-ep-n0-0]     Priority             : 1 
[radius-ep-n0-0]     Session State        : No_Session 
[radius-ep-n0-0] LOG MESSAGES: 
[radius-ep-n0-0]     2020/11/25 10:30:02.960 [TRACE] [infra.message_log.core] >>>>>>>> 
[radius-ep-n0-0]  
[radius-ep-n0-0]     2020/11/25 10:30:02.960 [DEBUG] [Radius.smf.AAA] Coa/Disconnect Req received from udp proxy 
[radius-ep-n0-0]     2020/11/25 10:30:02.960 [DEBUG] [Radius.smf.AAA] SrcIp: 198.18.1.20 SrcPort: 3799 DestIp: 198.18.1.4 DestPort: 3799 
[radius-ep-n0-0]     2020/11/25 10:30:02.960 [ERROR] [Radius.smf.AAA] Unable to process Coa/Disconnect request - Error during init of Radius Message Invalid Coa Client 198.18.1.20 
[radius-ep-n0-0]     2020/11/25 10:30:02.960 [TRACE] [infra.message_log.core] <<<<<<<< 
[radius-ep-n0-0]  
[radius-ep-n0-0] *********************************************** 
[radius-ep-n0-0]
Disconnect Message Received with Invalid Session ID Key

The following is a sample error log when disconnect message is received with invalid session ID key. 

[radius-ep-n0-0] ************* TRANSACTION: 00009 ************* 
[radius-ep-n0-0] TRANSACTION SUCCESS: 
[radius-ep-n0-0]     Txn Type             : RadiusUdpProxyCoaMsg(2003) 
[radius-ep-n0-0]     Priority             : 1 
[radius-ep-n0-0]     Session State        : No_Session 
[radius-ep-n0-0] LOG MESSAGES: 
[radius-ep-n0-0]     2020/11/25 10:49:43.942 [TRACE] [infra.message_log.core] >>>>>>>> 
[radius-ep-n0-0]  
[radius-ep-n0-0]     2020/11/25 10:49:43.942 [DEBUG] [Radius.smf.AAA] Coa/Disconnect Req received from udp proxy 
[radius-ep-n0-0]     2020/11/25 10:49:43.942 [DEBUG] [Radius.smf.AAA] SrcIp: 198.18.1.20 SrcPort: 3799 DestIp: 198.18.1.4 DestPort: 3799 
[radius-ep-n0-0]     2020/11/25 10:49:43.942 [DEBUG] [Radius.smf.AAA] Decoded coa message type is DisconnectRequest 
[radius-ep-n0-0]     2020/11/25 10:49:43.942 [ERROR] [Radius.smf.AAA] Unable to process DisconnectRequest - Error during construct Invalid DNN/IPv4Addr/IPv6Pfx value 
[radius-ep-n0-0]     2020/11/25 10:49:43.942 [TRACE] [infra.message_log.core] <<<<<<<< 
[radius-ep-n0-0]  
[radius-ep-n0-0] ***********************************************