목차

소개

문제

분석

모든 피어가 Dead임을 기록합니다.

SMF 확인

Grafana체크

Nexus 검사

솔루션

소개

이 문서에서는 와 관련된 SMF(Session Management Function) 로그 경고를 트러블슈팅하는 방법에 대해 설명합니다. All Peers are Dead, Setting status code to 0.

문제

세션 충격이 SMF에 보고되었습니다.

분석

모든 피어가 Dead임을 기록합니다.

로그는 SelectedProfileName:CHF-OFF 내부의 모든 피어가 종료되었음을 나타냅니다.

로그는 SMF에 구성된 모든 엔드포인트를 대상으로 하며, 모든 피어가 프로파일 내부에서 데드 상태인 것을 볼 때마다 항상 세션에 영향을 미칩니다.

master-1 b26897bce81a[2516]:

master-1 c77834f772f7[2516]: ************* TRANSACTION: 2475167152 *************

master-1 c77834f772f7[2516]: ************* TRANSACTION: 2475167152 *************

master-1 c77834f772f7[2516]: TRANSACTION SUCCESS:

master-1 c77834f772f7[2516]: TRANSACTION SUCCESS:

master-1 c77834f772f7[2516]: GR Instance ID : 1

master-1 c77834f772f7[2516]: GR Instance ID : 1

master-1 c77834f772f7[2516]: Txn Type : N40ChargingDataReq(3585)

master-1 c77834f772f7[2516]: Txn Type : N40ChargingDataReq(3585)

master-1 c77834f772f7[2516]: Priority : 1

master-1 c77834f772f7[2516]: Priority : 1

master-1 c77834f772f7[2516]: Session Namespace : smf(1)

master-1 c77834f772f7[2516]: Session Namespace : smf(1)

master-1 c77834f772f7[2516]: CDL Slice Name : smf

master-1 c77834f772f7[2516]: CDL Slice Name : smf

master-1 c77834f772f7[2516]: LOG MESSAGES:

master-1 c77834f772f7[2516]: LOG MESSAGES:

master-1 c77834f772f7[2516]: 2023/09/10 15:00:00.007 [ERROR] [nrfClient.Discovery.nrf] All Peers are Dead, Setting status code to 0 (timeout)

master-1 c77834f772f7[2516]: 2023/09/10 15:00:00.007 [ERROR] [nrfClient.Discovery.nrf] All Peers are Dead, Setting status code to 0 (timeout)

master-1 c77834f772f7[2516]: 2023/09/10 15:00:00.007 [ERROR] [nrfClient.Discovery.nrf] Message send failed, response [Type:CHF ServiceName:nchf-convergedcharging SelectedProfileName:"CHF-OFF" FailureProfile:"Fail-H-CHF-OFF" GroupID:"CHF-*" ]

master-1 c77834f772f7[2516]: 2023/09/10 15:00:00.007 [ERROR] [nrfClient.Discovery.nrf] Message send failed, response [Type:CHF ServiceName:nchf-convergedcharging SelectedProfileName:"CHF-OFF" FailureProfile:"Fail-H-CHF-OFF" GroupID:"CHF-*" ]

master-1 c77834f772f7[2516]: ***********************************************

master-1 c77834f772f7[2516]: ***********************************************

컨피그레이션에 따라 SMF는 HTTP 코드 504(시간 제한) 시스템이 있는 경우 우선순위가 더 높은 기본 서버에 연결을 시도한 다음 SMF는 보조 서버에 연결을 시도합니다. 실패할 경우 시스템은 세션을 계속 모드로 설정합니다.

예에서, 오프라인에 대한 2차 충전 기능(CHF)은 10.10.10.2이다. SMF에서 504 오류를 받았으며 작업은 FailureContinueAction입니다.

master-2 42013075464a[2621]: 2023/09/10 15:00:00.063 rest-ep [ERROR] [RestClient.go:175] [infra.rest_client.core] Error in rest call err Post "http://10.10.10.2:1090/OFFLINE/nchf-convergedcharging/v2/chargingdata": context deadline exceeded

master-2 42013075464a[2621]: 2023/09/10 15:00:00.063 rest-ep [ERROR] [Config.go:1721] [nrfClient.Discovery.nrf] Send to NF rpcName[CHF], method:[DataRequest] EndPoint[http://10.10.10.2:1090/OFFLINE/nchf-convergedcharging/v2] failed

master-2 42013075464a[2621]: ************* TRANSACTION: 2252879781 *************

master-2 42013075464a[2621]: TRANSACTION SUCCESS:

master-2 42013075464a[2621]: GR Instance ID : 1

master-2 42013075464a[2621]: Txn Type : N40ChargingDataReq(3521)

master-2 42013075464a[2621]: Priority : 1

master-2 42013075464a[2621]: Session Namespace : smf(1)

master-2 42013075464a[2621]: CDL Slice Name : smf

master-2 42013075464a[2621]: LOG MESSAGES:

master-2 42013075464a[2621]: 2023/09/10 15:00:00.063 [ERROR] [rest_ep.app.ChargingIntf] {imsi-1234567891011121:21} Received Charging Data Response error with timediff 10001557123 - Request message {{"invocationSequenceNumber":1,"invocationTimeStamp":"2025-11-10T14:29:29Z","nfConsumerIdentification":{"nFIPv4Address":"10.10.10.12","nFName":"dce0c1d7-aa37-4f2c-870b-6f7c1be10af1","nFPLMNID":{"mcc":"123","mnc":"456"},"nodeFunctionality":"SMF"},"notifyUri":"http://10.10.10.12:8195/callbacks/v2/notifyUri/1909959397/chargingNotification","pDUSessionChargingInformation":{"chargingId":1909959397,"pduSessionInformation":{"authorizedQoSInformation":{"5qi":1,"arp":{"preemptCap":"NOT_PREEMPT","preemptVuln":"PREEMPTABLE","priorityLevel":1}},"authorizedSessionAMBR":{"downlink":"2048000 bps","uplink":"2048000 bps"},"chargingCharacteristicsSelectionMode":"VISITING_DEFAULT","dnnId":"data","hPlmnId":{"mcc":"123","mnc":"456"},"networkSlicingInfo":{"sNSSAI":{"sst":1}},"pduAddress":{"iPv6dynamicPrefixFlag":true,"pduIPv6AddresswithPrefix":"x:x:x:x::"},"pduSessionID":21,"pduType":"IPV6","ratType":"WLAN","servingCNPlmnId":{"mcc":"123","mnc":"456"},"sscMode":"SSC_MODE_1","startTime":"2025-11-10T14:29:29Z"},"userInformation":{"roamerInOut":"IN_BOUND","servedGPSI":"msisdn-12345678901"},"userLocationinfo":{"n3gaLocation":{"portNumber":4505,"ueIpv4Addr":"x.x.x.x"}}},"subscriberIdentifier":{"subscriberIdentityType":"SUPI","supi":"imsi-1234567891011121"}}}

master-2 42013075464a[2621]: 2023/09/10 15:00:00.063 [ERROR] [nrfClient.SendMesg.NRF] FHI status 504 timediff 1000332537, Uri: http://10.10.10.2:1090/OFFLINE/nchf-convergedcharging/v2, retryCount = 0 loopMaxRetry = 0, maxRetry = 0

master-2 42013075464a[2621]: 2023/09/10 15:00:00.063 [ERROR] [nrfClient.Discovery.nrf] Message send failed, response [Type:CHF Http2_Status:504 FailAction:FailureContinueAction MsgType:3587 ServiceName:nchf-convergedcharging SelectedProfileName:"CHF-OFF" FailureProfile:"Fail-H-CHF-OFF" GroupID:"CHF-*" ]

master-2 42013075464a[2621]: ***********************************************

SMF 확인

SMF에서 문제를 보고한 엔드포인트와 관련된 피어 및 연결된 시간을 확인합니다.

smf# show peers

GR                                                                                POD              CONNECTED              ADDITIONAL

INSTANCE  ENDPOINT        LOCAL ADDRESS         PEER ADDRESS           DIRECTION  INSTANCE   TYPE  TIME           RPC     DETAILS     INTERFACE NAME     VRF

--------------------------------------------------------------------------------------------------------------------------------------------------------------

1         <none>          192.168.1.1        10.10.10.2:1090      Outbound   rest-ep-0  Rest  4 hours        CHF     <none>      n40                NA

1         <none>          192.168.1.2        10.10.10.2:1090      Outbound   rest-ep-1  Rest  4 hours        CHF     <none>      n40                NA

1         <none>          192.168.1.3        10.10.10.1:1090     Outbound   rest-ep-2  Rest  4 hours        CHF     <none>      n40                NA

1         <none>          192.168.1.3        10.10.10.2:1090      Outbound   rest-ep-2  Rest  4 hours        CHF     <none>      n40                NA

1         <none>          192.168.1.4        10.10.10.1:1090     Outbound   rest-ep-3  Rest  4 hours        CHF     <none>      n40                NA

1         <none>          192.168.1.2        10.10.10.1:1090     Outbound   rest-ep-1  Rest  4 hours        CHF     <none>      n40                NA

1         <none>          192.168.1.4        10.10.10.2:1090      Outbound   rest-ep-3  Rest  2 hours        CHF     <none>      n40                NA

1         <none>          192.168.1.1        10.10.10.1:1090     Outbound   rest-ep-0  Rest  4 hours        CHF     <none>      n40                NA

// CHF related profiles

profile network-element chf CHF-OFFLINE

nf-client-profile        CHF-OFF

failure-handling-profile Fail-H-CHF-OFF

discovery local

exit

// Here is configuration for CHF profile where all peers are dead

profile nf-client nf-type chf

chf-profile CHF-OFF

locality LOC1

priority 1

service name type nchf-convergedcharging

responsetimeout 1000

endpoint-profile epprof

capacity   10

api-root   OFFLINE

uri-scheme http

version

uri-version v2

exit

exit

endpoint-name ep1

priority 1

capacity 10

primary ip-address ipv4 10.10.10.1

primary ip-address port 1090

exit

endpoint-name ep2

priority 2

capacity 10

primary ip-address ipv4 10.10.10.2

primary ip-address port 1090

exit

exit

exit

exit

exit

// Failure handling that in case of timeout (HTTP code 504) then try secondary server one time and then proceed with continuing the session

profile nf-client-failure nf-type chf

profile failure-handling Fail-H-CHF-OFF

service name type nchf-convergedcharging

responsetimeout 1000

message type ChfConvergedchargingCreate

status-code httpv2 504

retry  1

action continue

exit

exit

message type ChfConvergedchargingUpdate

status-code httpv2 504

retry  1

action continue

exit

exit

message type ChfConvergedchargingDelete

status-code httpv2 504

retry  1

action continue

exit

exit

exit

그라파나 수표

HTTP 504 시간 초과와 문제 시간 간의 직접적인 상관관계가 관찰되었습니다.

query: sum(increase(smf_restep_http_msg_total{nf_type="chf", namespace=~"$namespace"}[15m])) by (api_name, response_cause, response_status)

Nexus 검사

플랩이 있었는지 확인하세요.

Nexus# show logging last 500 | include BFD

솔루션

이 문제의 해결책은 SMF가 클라이언트이고 CHF가 서버이기 때문에 이 경우에 다릅니다.

SMF로 인한 연결 손실이 아닙니다.