The SMF supports High Availability (HA) of UDP proxy. The HA model of UDP proxy is based on the keepalived virtual IP concepts.

For more information on UDP proxy redundancy, see the High Availability for the UDP Proxy section in the Pods and Services Reference chapter.

The SMF supports IPAM redundancy and load balancing for each UPF. The IPAM running in the Node Manager microservice has two IPAM instances that are associated to each UPF. When one IPAM instance is inactive, the other IPAM instance manages the address allocation requests for the UPF.

For more information on node manager redundancy, see the IPAM Redundancy Support Per UPF section in the IP Address Management chapter.

High availability inside rack is an arrangement that handles server level redundancies. In this solution, in case of a single server failure, a few pods according to the design might get distributed in another available server. Meanwhile, other pods remain in pending state until the server comes back.

In this scenario, the critical pods that handle the external traffic run on Active-Stand-by pair on a different server. The redundancy support in the routing layer is provided by leaf and spine layers.

This section describes the deployment of SMF pods and its microservices.

The following figure shows the deployment model of six VMs in SMF.

In this model, the pods are deployed on VM pairs. Two replicas are available for each protocol pod (for example, rest-ep, protocol-ep, and gtp-ep). One instance is deployed on each protocol VM.

Similarly, service pods and session pods are distributed on both the service and session VMs equally. Such a distribution is controlled by labelling the VMs as well as implementing the K8 affinity and anti-affinity rules during pod scheduling.

This model ensures that, during VM reboot scenarios, at least 50% of the replicas of each pod type are available to handle user signaling.

Graceful pod restart allows pod to complete ongoing processing within 30 seconds. Abrupt pod restart will affect ongoing transactions without impact to PDU sessions.

The following is an example of VM labelling and replica configuration.

k8 label protocol-layer key smi.cisco.com/node-type value smf-proto
exit
k8 label service-layer key vm-type value smf-svc
exit
k8 label cdl-layer key smi.cisco.com/node-type value smf-cdl
exit
k8 label oam-layer key smi.cisco.com/node-type value oam
exit

endpoint pfcp
 replicas 1
 nodes    2
exit
endpoint service
 replicas 1
 nodes    2
exit
endpoint protocol
 replicas 1
 nodes    2
 vip-ip 209.165.201.28
exit
endpoint sbi
 replicas 1
 nodes    2

To verify the configuration, use the following show command:

show running-config instance instance-id instance_id endpoint 

The following is an example output of this show command.

[unknown] smf# show running-config instance instance-id 1 endpoint 
instance instance-id 1
 endpoint nodemgr
  replicas 1
  nodes    2
 exit
 endpoint gtp
  replicas 1
  vip-ip 209.165.202.149
 exit
 endpoint pfcp
  replicas                2
  enable-cpu-optimization true
  interface n4
   heartbeat
    interval               0
    retransmission-timeout 3
    max-retransmissions    5
   exit
  exit
 exit
 endpoint service
  replicas 2
 exit
 endpoint protocol
  replicas 1
  vip-ip 209.165.202.149
 exit
exit 

This command output displays the configurations related to multiple endpoints, such as endpoint names, pod replicas, nodes, and so on.

SMF identifies active L2 VIPs on the protocol node that do not have any traffic. If case of an active L2 VIPs, there may not be any traffic due to various reasons, including network connectivity issues.

Upon detecting the no traffic scenario on active L2 VIPs, the SMF attempts to restart the pod or triggers the GR based on the CLI configuration action. It facilitates the restoration of traffic resulting in minimal loss of sessions.

This feature can be enabled or disabled using a CLI.

• The traffic monitoring on IPC L2 VIP happens periodically and is matched with configurable no-traffic-duration time.

• If there is no traffic on the active IPC L2 VIP for the configured time duration, depending on the action configured either the GR trigger happens, so that other rack can handle the traffic, or the pod restarts, so that VIP switches to the other pod restoring traffic.

• If a pod restart action is configured and even after the VIP switchover if traffic is not restored, the IPC connection monitoring feature will trigger the GR scenario if configured to minimize the impact.

• Following checks are monitored periodically. If the checks fulfill the condition, the traffic is monitored:

  • CLI config no-traffic-duration should be greater than 10 seconds.

  • CLI config session-threshold should be greater than or equal to 1000. The default value is 1000.

  • Whether any of the GR Instance ID roles is PRIMARY.

  • Whether internal VIP is present on the protocol node.

  • If the session count is greater than or equal to the session threshold configured using the CLI, traffic monitoring is triggered. However, if the session count is less than 1000, traffic monitoring is stopped.

• Single IPC endpoint handles traffic for both the GR Instance IDs in GR mode. Traffic monitoring on IPC endpoint will not be GR Instance ID specific. However, checks are done for both the GR Instance IDs.

Use the following configuration to trigger HA (pod-restart) scenario or GR (trigger-switch-over) scenario:

config 
   monitor 
      active-instance-traffic {  no-traffic-duration no_traffic_duration session-threshold session_threshold action { pod-restart | trigger-switch-over  } } 

NOTES:

• active-instance-traffic —Specify this keyword to enable or disable active traffic monitoring.

• no-traffic-duration no_traffic_duration —Maximum allowed time in seconds of no traffic on active instance. The value range is 10-300. There is no default value set for this CLI. The value of no-traffic-duration should be less than the value of max-conn-downtime if set.

• session-threshold session_threshold —Minimum session count required to start the active instance traffic monitoring. The value range is 10-10000000. The default value is 1000.

• action { pod-restart | trigger-switch-over } —Action to take on condition fulfillment. The possible values for this command are pod-restart and trigger-switch-over . The default value is pod-restart .

This section defines various statistics supported in this feature.

In active-active mode,

• The inter-rack deployment is transparent to the adjacent NFs.

• The inter-rack deployment contains two instances of the CCG function, each instance manifest itself with a set of interface IPs.

• Each instance support sets of sessions and continue to use the same IP for session consistency.

• At a specific time period, one CCG instance can be primary only on one rack and standby on the other rack.

• The set of interface IPs that are associated with the CCG instance, dynamically route to the primary rack of the instance.

SMF supports primary/standby redundancy in which data is replicated from the primary to standby instance. The primary instance provides services in normal operation. If the primary instance fails, the standby instance becomes the primary and takes over the operation. To achieve inter-rack redundancy, two primary/standby pairs can be set up where each rack is actively processing traffic and standby is acting as backup for the remote rack.

In an Active-Active inter-rack redundancy deployment, consider there are two racks: Rack-1 and Rack-2 located in the same data center. All the NFs are trying to reach instance-1 and instance-2.

For NFs, both the instances are active. But in real, instance-1 and instance-2 are divided across racks.

Rack-1 has instance-1 and instance-2. In a pre-trigger scenario, instance-1 is local and acts as Primary and instance-2 is in Standby mode.

Rack-2 also has instance-1 and instance-2. In a pre-trigger scenario, instance-2 is local and acts as Primary and instance-1 is in Standby mode.

In case, if Rack-1 goes down, the traffic moves to Rack-2. On Rack-2 both the instances, instance-1 and instance-2 acts as Primary.

This configuration is needed to provide a inter-rack redundancy configuration for multiple rack. With instance ID, endpoint configurations should be configured for each rack.

Sample Configuration 1

The following is a sample configuration for endpoint VIP configuration under one instance:

config 
  instance instance-id gr_instanceId 
    endpoint endpoint_name 
        vip-ip vip_ip_address 
  exit 
exit 

Example:

config
instance instance-id 1
 endpoint sbi
  vip-ip 209.165.201.21
 exit
exit

Sample Configuration 2

The following is a sample configuration to provide information on system-id, cluster-id and slice-name under an instance:

config 
  instances instance instance_id 
   system-id system_id 
   cluster-id cluster_id 
   slice-name cdl_slice_name 
  exit 
exit 

Example:

config
 instances instance 1
  system-id smf
  cluster-id smf
  slice-name 1
 exit
exit

Note	It is recommended to have the same values for system-id, cluster-id in the instance, and app-name, cluster-name in deployment.

Only two instances can be configured on each local and remote rack, and corresponding endpoints can be instantiated.

A local instance-id is the identity of the local rack irrespective of if the rack is redundant or not.

Local Instance ID Configuration

The local instance is configured using the local-instance command.

local-instance instance 1

Endpoint configuration must be under instance specified by each unique instance ID.

Endpoint Configuration Example

Following are a few configuration examples.

Note	In the following example, instance-id "1" is a local instance-id, and endpoints configured under it belong to the local rack. Optionally, remote rack instance-id "2" can be configured for endpoints belonging to the inter-rack.

instance instance-id 1
 endpoint li
  replicas 1
  nodes    2
  vip-ip 209.165.201.6
  vip-ip 209.165.201.13
 exit
 endpoint gtp
  replicas 1
  nodes    2
  retransmission timeout 5 max-retry 4
  vip-ip 209.165.201.6
  vip-ip 209.165.201.4
  interface s5
   echo interval          60
   echo retransmission-timeout 5
   echo max-retransmissions 4
  exit
  interface s2b
   echo interval 60
   echo retransmission-timeout 5
   echo max-retransmissions 4
  exit
 exit
exit
instance instance-id 2
 endpoint li
  replicas 1
  nodes    2
  vip-ip 209.165.201.6
  vip-ip 209.165.201.13
 exit
exit
endpoint gtp
  replicas 1
  nodes    2
  retransmission timeout 5 max-retry 4
  vip-ip 209.165.201.6
  vip-ip 209.165.201.5
  interface s5
   echo interval 60
   echo retransmission-timeout 5
   echo max-retransmissions 4
  exit
  interface s2b
   echo interval 60
   echo retransmission-timeout 5
   echo max-retransmissions 4
  exit
 exit
exit

Add instance for PGW FQDN corresponding to local and remote instances.

Example

Following is a configuration example.

Note	In the following example, instance-id "1" is a local instance-id, and the SMF profile configured under it belongs to the local rack. Optionally, remote rack instance-id "2" can be configured for FQDN belonging to the inter-rack.

profile smf smf1
locality LOC1
allowed-nssai [ slice1 ]
instances 1 fqdn cisco.com.apn.epc.mnc456.mcc123
instances 2 fqdn cisco.com.apn.epc.mnc567.mcc123

This section describes how to configure the dynamic routing using BGP.

Configuring AS and BGP Router IP Address

To configure the AS and IP address for the BGP router, use the following commands:

config 
   router bgp local_as_number 
   exit 
exit 

NOTES:

• router bgp local_as_number —Specify the identification number for the AS for the BGP router.

  In a inter-rack redundancy deployment, you need to configure two Autonomous Systems (AS).

  • One AS for leaf and spine.

  • Second AS for both racks: Rack-1 and Rack-2.

Configuring BGP Service Listening IP Address

To configure the BGP service listening IP address, use the following commands:

config 
   router bgp local_as_number 
      interface interface_name 
   exit 
exit 

NOTES:

• router bgp local_as_number —Specify the identification number for the AS for the BGP router.

• interface interface_name —Specify the name of the interface.

Configuring BGP Neighbors

To configure the BGP neighbors, use the following commands:

config 
   router bgp local_as_number 
      interface interface_name 
      neighbor neighbor_ip_address remote-as as_number 
   exit 
exit 

NOTES:

• router bgp local_as_number —Specify the identification number for the AS for the BGP router.

• interface interface_name —Specify the name of the interface.

• neighbor neighbor_ip_address —Specify the IP address of the neighbor BGP router.

• remote-as as_number —Specify the identification number for the AS.

Configuring Bonding Interface

To configure the bonding interface related to the interfaces, use the following commands:

config 
   router bgp local_as_number 
      interface interface_name 
      bondingInterface interface_name 
   exit 
exit 

NOTES:

• router bgp local_as_number —Specify the identification number for the AS for the BGP router.

• interface interface_name —Specify the name of the interface.

• bondingInterface interface_name —Specify the related bonding interface for an interface. If the bonding interface is active, then the BGP gives a higher preference to the interface-service by providing a lower MED value.

Configuring Learn Default Route

If the user configures specific routes on their system and they need to support all routes, then they must set the learnDefaultRoute as true.

Note	This configuration is optional.

To configure the Learn Default Route, use the following commands:

config 
   router bgp local_as_number 
      learnDefaultRoute true/false 
   exit 
exit 

NOTES:

• router bgp local_as_number —Specify the identification number for the AS for the BGP router.

• learnDefaultRoute true/false —Specify the option to enable or disable the learnDefaultRoute parameter. When set to true, BGP learns default route and adds it in the kernel space. By default, it is false.

Configuring BGP Port

To configure the Port number for a BGP service, use the following commands:

config 
   router bgp local_as_number 
      loopbackPort port_number 
   exit 
exit 

NOTES:

• router bgp local_as_number —Specify the identification number for the AS for the BGP router.

• loopbackPort port_number —Specify the port number for the BGP service. The default value is 179.

Policy Addition

The BGP speaker pods learns many route information from its neighbors. However, only a few of them are used for supporting the outgoing traffic. This is required for egress traffic handling only, when SMF is sending information outside to AMF/PCF. Routes are filtered by configuring import policies on the BGP speakers and is used to send learned routes to the protocol pods.

A sample CLI code for policy addition and the corresponding descriptions for the parameters are shown below.

$bgp policy <policy_Name> ip-prefix 209.165.200.225 subnet 16 masklength-range 21..24 as-path-set “^65100”

This configuration controls the number of BGP speaker pods in deployment. BGP speaker advertises service IP information for incoming traffic from both the racks.

Note	Use non-bonded interface in BGP speaker pods for BGP peering. BGP peering per Proto node is supported with only two BGP routers/leafs. Considering two Proto nodes, there can be maximum of four BGP neighborships.

instance instance-id  instance_id endpoint bgpspeaker interface { bgp | bfd } internal base-port start base_port_number 

config 
instance instance-id instance_id 
 endpoint bgpspeaker 
   replicas replica_id 
   nodes node_id 
   interface bgp 
     internal base-port start base_port_number  
   exit 
   interface bfd 
     internal base-port start base_port_number 
  exit 
exit 

NOTES:

• instance instance-id instance_id —Specify the GR instance ID.

• base_port_number —Specify the port range only if logical NF is configured. This range depends on your deployment.

Example

The following is a configuration example:

instance instance-id 1
endpoint bgpspeaker
   replicas 1
   nodes    2
   interface bgp
     internal base-port start {24000}
   exit
   interface bfd
     internal base-port start {25000}
 exit

The following section provides IPAM configuraton examples.

SMF-1 Example

The following is a configuration example for SMF-1:

ipam
 instance 1
  address-pool pool-1
  vrf-name ISP
   tags
   dnn dnn-1
  exit
  ipv4
    address-range 209.165.201.1 209.165.201.31
 exit
instance 2
 address-pool pool-2
  vrf-name ISP
  tags
   dnn dnn-2
  exit
  ipv4
    address-range 209.165.202.129 209.165.202.159
 exit
exit

SMF-2 Example

The following is a configuration example for SMF-2:

ipam
 instance 1
 address-pool pool-1
  vrf-name ISP
  tags
   dnn dnn-1
  exit
  ipv4
    address-range 209.165.201.1 209.165.201.31
 exit
 instance 2
 address-pool pool-2
  vrf-name ISP
   tags
   dnn dnn-2
  exit
  ipv4
    address-range 209.165.202.129 209.165.202.159
 exit
exit

Endpoints must be configured under an instance. Two Geo-Redundancy pods are needed on each rack. You should also configure VIP for internal and external Geo interface for ETCD/CachePod replication.

instance instance-id  instance_id endpoint geo interface { geo-internal | geo-external } vip-ip { vip_ip_address } vip-port { vip_port_number }
 

config 
instance instance-id instance_id 
 endpoint geo 
  replicas replica_id 
  nodes node_id 
  internal base-port start base_port_number 
  interface geo-internal 
   vip-ip vip_ip_address vip-port vip_port_number 
  exit 
  interface geo-external 
   vip-ip vip_ip_address vip-port vip_port_number 
  exit 
exit 
exit 

NOTES:

• instance instance-id instance_id —Specify GR instance ID. One instance ID for local rack and other for another rack.

• vip-ip vip_ip_address —Specify VIP IP address for Internal/External Geo interface.

• vip-port vip_port_number —Specify VIP port number.

• internal base-port start base_port_number —Specify port range only if logical NF is configured.

Example

The following is a configuration example:

instance instance-id 1
endpoint geo
  replicas 1
  nodes 2
  internal base-port start 25000
  interface geo-internal
   vip-ip 209.165.201.8 vip-port 7001
  exit
  interface geo-external
   vip-ip 209.165.201.8 vip-port 7002
  exit
exit

To configure pod monitoring and failover thresholds in the inter-rack setup, use the following sample configuration. The geo pod monitors the configured pod name.

config 
 geomonitor 
  podmonitor pods pod_name 
   retryCount value 
   retryInterval interval_value 
   retryFailOverInterval failover_interval 
   failedReplicaPercent percent_value 
  exit 
 exit 

NOTES:

• pods pod_name —Specify the name of the pod to be monitored. For example, Cache-pod, rest-ep, and so on.

• retryCount value —Specify the retry counter value to retry if pod fails to ping after which pod is marked as down. It should be an integer in the range of 1-10.

• retryInterval interval_value —Specify the retry interval in milliseconds if the pod successfully pings. It should be an integer in the range of 200-10000.

• retryFailOverInterval failover_interval —Specify the retry interval in milliseconds if the pod fails to ping. It should be an integer in the range of 200-10000.

• failedReplicaPercent percent_value —Specify the percent value of failed replica after which the inter-rack redundancy failover is triggered. It should be an integer in the range of 10-100.

Configuration Example

The following is an example configuration.

geomonitor podmonitor pods cache-pod
 retryCount 3
 retryInterval 5
 retryFailOverInterval 1
 failedReplicaPercent 40
exit

Remote cluster monitoring auto corrects roles (it becomes self-primary, when the remote rack is in STANDBY_ERROR state) for uninterrupted traffic flow of traffic. However, this auto role correction gets done only for specific roles.

To configure this feature, use the following sample configuration:

config 
    geomonitor 
        remoteclustermonitor 
            retryCount value 
            retryInterval interval_value 
            end 

NOTES:

• retryCount value —Specify the retry count before making the current rack PRIMARY. It should be an integer in the range of 1-10. The default value is 3.

• retryInterval interval_value —Specify the retry interval in the count of milliseconds, after which the remote rack status gets fetched. It should be an integer in the range of 200-50000. The default value is 3000.

Configuration Example

The following is an example configuration

geomonitor remoteclustermonitor
retryCount 3
retryInterval 3000

The following command is used to monitor the traffic.

config 
 geomonitor 
  trafficMonitor 
   thresholdCount value 
   thresholdInterval interval_value 
  exit 
 exit 

NOTES:

• thresholdCount value —It specifies the number of calls received for standby instance. It should be an integer in the range of 0-10000. Default value is 0. Both UDP-proxy and REST-EP must be considered for the counter value.

• thresholdInterval interval_value —It specifies the maximum duration to hit the threshold count value in ms. It should be an integer in the range of 100-10000. Default value is 3000.

Configuration Example

The following is an example configuration

geomonitor trafficmonitor
thresholdCount 3
thresholdInterval 3000

The following areas of the enhanced geo replication pod are part of the geo redundancy strengthening feature:

• The system makes GRPC replication and admin stream available when it becomes healthy.

• The system ensures that both instances have the correct role in all scenarios.

• The system maintains data replication between the geo replication pod of rack-1 and rack-2 in sync.

• The system verifies that checksum and checkpoint data is always correct on both racks to ensure data integrity.

• The system provides a mechanism for users to sync replication data, when it becomes out of sync, and requires manual intervention.

  To know more about the command, see the Geo Replication Pull Data section.

• The system ensures that Geo VIP movement between geo replication pods does not impact functionality.

• The system takes measures to prevent failures of dependent processes from affecting its functionality for users.

This section describes operations, administration, and maintenance support for this feature.

Before deploying the CDL GR, user must configure the following:

• CDL Session Database and define the base configuration.

• Kafka for CDL.

• Zookeeper for CDL.

In CDL, along with existing GR related parameters, GR instance awareness must be enabled using a feature flag on all the racks. Also, the mapping of system-id to slice names should also be provided for this feature to work on all the racks.

The CDL is also equipped with Geo Replication (GR) failover notifications, which can notify the timer expiry of session data and bulk notifications to the currently active rack. The CDL uses Border Gateway Protocol (BGP) through App-Infra for the GR failover notifications.

The CDL subscribes to the key value on both the GR racks. The App-Infra sends notifications to the CDL when there is any change in these key values. A key value indicates the state of the CDL System ID or the GR instance. The GR instance is mapped to the CDL slices using the CDL system ID or the GR instance ID in the key.

The system ID is mandatory on both the racks. The GR instance ID in the NF configuration must match the CDL system ID.

CDL has instance-specific data slices. It also allows users to configure instance-specific slice information at the time of bringing up.

• CDL notifies the data on expiry or upon bulk notification request from the active slices.

• CDL determines the active instance based on the notification from app-infra memory-cache.

• CDL slice is a partition within a CDL instance to store a different kind of data. In this case, NF stores a different instance of data.

Note	CDL slice name should match with the slice-name configured in GR.

The following sample setup configures the Geo Replication Pull Data. These commands help and perform the replication data sync activities, during the event of any malfunction scenario. To transition to the GR role and minimize any replication data mismatch on the system, you can initiate the following commands to synchronize the data across all accessible racks:

geo replication-pull instance-id gr_instanceId 

NOTES:

• geo replication-pull —It pulls the replication data from the peer rack and syncs it with the local rack.

• instance-id gr_instanceId —Specify the GR Instance ID.

To reset the GR instance role (for example, roles from STANDBY_ERROR to STANDBY to PRIMARY), use the following sample commands:

geo reset-role role role instance-id gr_instanceId 

NOTES:

• role role —Specify the new role for the given rack.

  The role can be PRIMARY or STANDBY.

• instance-id gr_instanceId —Specify the GR Instance ID.

Important

The command geo reset-role triggers change in the role for the given instance on the local rack. The remote rack does not receive any message for the same command. It is only possible to change the role for the given instance ID from STANDBY_ERROR to STANDBY and STANDBY to PRIMARY. Another role change is not possible.

To switch the GR role, initiate the command on the primary rack (for example, role PRIMARY to STANDBY only), and use the following command.

geo switch-role { role primary | standby  instance-id gr_instanceId [ failback-interval  failback_interval ] } 

NOTES:

• role role —Specify the new role for the given rack.

  The roles can be primary or standby . It’s mandatory to trigger manual switchover from primary role for a specific GR instance ID.

• instance-id gr_instanceId —Specify the GR Instance ID

• failback-interval is an optional command to provide backward compatibility of upgrades between releases. The recommended value of failback-interval is 0.

Important

geo switch-role command triggers manual failover from one rack to another rack for specific instance ID. The rack which triggers the failover changes from the PRIMARY role to the STANDBY_ERROR role. In between, the rack which triggers the failover, sends a failover (Trigger GR) message to another rack. The other rack which receives the failover message changes from the STANDBY role to the PRIMARY role.

Note	failback-interval is deprecated from current release and will be discontinued from the subsequent releases.

This section describes show/clear commands that help in debugging issues.

To capture messages for subscriber (gr-instance aware), use the following command:

monitor subscriber [ capture-duration duration  | gr-instance gr_instance_id  | imei imei_id | imsi imsi_value | internal-messages [ yes ] | namespace [ sgw | smf ] | nf-service [ sgw | smf ] | supi supi_id | transaction-logs [ yes ] ] 

Note	In 2021.02 and later releases, the namespace keyword is deprecated and replaced with the nf-service keyword.

NOTES:

• capture-duration duration : Specify the duration in seconds during which monitor subscriber is enabled. The default value is 300 seconds (5 minutes). This is an optional parameter.

• gr-instance gr_instance_id : Specify the GR instance ID. The instance ID 1 denotes the local instance ID.

• imei imei_id : Specify the subscriber IMEI. For example: 123456789012345, *

• imsi imsi_value : Specify the subscriber IMSI. For example: 123456789, *

• internal-messages [ yes ] : Enable internal messages when set to yes . By default, it is disabled. This is an optional parameter.

• namespace [ sgw | smf ] : Enable the specified namespace. By default, namespace is set to none. This is an optional parameter.

  Important

  This keyword is deprecated in release 2021.02.0 and replaced with nf-service keyword.

• nf-service [ sgw | smf ] : Enable the specified NF service. By default, nf-service is set to none. This is a mandatory parameter.

  Important

  The nf-service keyword replaces the namespace keyword in release 2021.02 and beyond.

• supi supi_id : Specify the subscriber identifier. For example: imsi-123456789, imsi-123*

• transaction-logs [ yes ] : Enable transaction logs when set to yes . By default, it is disabled. This is an optional parameter.

  To view the transaction history logs, use the dump transactionhistory command.

  Note	The most recent transaction logs are stored in a circular queue of size 1024 transaction logs.

Example

The following is a configuration example.

monitor subscriber imsi 123456789 gr-instance 1
supi: imsi-123456789
captureDuration: 300
enableInternalMsg: false
enableTxnLog: false
namespace(deprecated. Use nf-service instead.): none
nf-service: none
gr-instance: 1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   295  100    98  100   197  10888  21888 --:--:-- --:--:-- --:--:-- 29500
Command: --header Content-type:application/json --request POST --data 
{"commandname":"mon_sub","parameters":{"supi":"imsi-123456789","duration":300,
"enableTxnLog":false,"enableInternalMsg":false,"action":"start","namespace":"none",
"nf-service":"none","grInstance":1}} http://oam-pod:8879/commands
Result start mon_sub, fileName ->logs/monsublogs/none.imsi-123456789_TS_2021-04-09T09:59:59.964148895.txt
Starting to tail the monsub messages from file: logs/monsublogs/none.imsi-123456789_TS_2021-04-09T09:59:59.964148895.txt
Defaulting container name to oam-pod.
Use 'kubectl describe pod/oam-pod-0 -n smf' to see all the containers in this pod.

To capture packets on different interfaces (gr-instance aware), use the following command:

monitor protocol { interface interface_name [ capture-duration duration  | gr-instancegr_instance | pcacp yes | | ] | list [ | ] }  

NOTES:

• interface interface_name —Specify the interface name on which PCAP is captured. This CLI allows the configuration of multiple interface names in a single CLI command.

• capture-duration duration —Specify the duration in seconds during which pcap is captured. The default is 300 seconds (5 minutes).

• The configured interface names can be retrieved using the show endpoint CLI command.

• gr-instance gr_instance_id —Specify the GR instance ID. The instance ID 1 denotes the local instance ID.

• pcap yes —Configure this option to enable PCAP file generation. By default, this option is disabled.

• list —Monitor protocol list files.

Example

The following is a configuration example.

monitor protocol interface sbi gr-instance 1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   220  100    95  100   125   8636  11363 --:--:-- --:--:-- --:--:-- 20000
Command: --header Content-type:application/json --request POST --data 
{"commandname":"mon_pro","parameters":{"interface":"sbi","duration":300,"action":
"start","enable_pcap":false,"grInstance":1}} http://oam-pod:8879/commands
Result start mon_pro, fileName ->logs/monprologs/sessintfname_sbi_at_2021-04-30T05:26:22.712229347.txt
Starting to tail the monpro messages from file: logs/monprologs/sessintfname_sbi_at_2021-04-30T05:26:22.712229347.txt
Defaulting container name to oam-pod.
Use 'kubectl describe pod/oam-pod-0 -n cn' to see all of the containers in this pod.

The following section provides information on GR setup health check.

• All critical pods are in good condition to serve user traffic.

  Use the following command to check whether GR and CDL related pods are in Running state.

  kubectl get pods -n cn-cn1 -o wide | grep georeplication-pod
  kubectl get pods -n cn-cn1 -o wide | grep cdl
  kubectl get pods -n cn-cn1 -o wide | grep mirror-maker

• Keepalived pods are in healthy state to monitor all VIPs which are configured for check-interface/check-port.

  Use the following command to check whether keepalived pods in “smi-vips” namespace are in “Running” state.

  kubectl get pods -n smi-vips

• Health-check of pods related to CDL: Check the status of CDL db-endpoint, slot and indexes. All should be in STARTED or ONLINE state for both System IDs 1 and 2.

  cdl show status 
  message params: {cmd:status mode:cli dbName:session sessionIn:{mapId:0 limit:500 key: purgeOnEval:0 filters:[] nextEvalTsStart:0 nextEvalTsEnd:0 allReplicas:false maxDataSize:4096} sliceName:}
  db-endpoint {
      endpoint-site {
          system-id 1
          state STARTED
          total-sessions 4
          site-session-count 2
          total-reconciliation 0
          remote-connection-time 66h37m31.36054781s
          remote-connection-last-failure-time 2021-07-13 11:24:10.233825924 +0000 UTC
          slot-geo-replication-delay 2.025396ms
      }
      endpoint-site {
          system-id 2
          state STARTED
          total-sessions 4
          site-session-count 2
          total-reconciliation 0
          remote-connection-time 66h58m49.83449066s
          remote-connection-last-failure-time 2021-07-13 11:02:51.759971655 +0000 UTC
          slot-geo-replication-delay 1.561816ms
      }
  }
  slot {
      map {
          map-id 1
          instance {
              system-id 1
              instance-id 1
              records 4
              capacity 2500000
              state ONLINE
              avg-record-size-bytes 1
              up-time 89h38m37.335813523s
              sync-duration 9.298061ms
          }
          instance {
              system-id 1
              instance-id 2
              records 4
              capacity 2500000
              state ONLINE
              avg-record-size-bytes 1
              up-time 89h39m11.1268024s
              sync-duration 8.852556ms
          }
          instance {
              system-id 2
              instance-id 1
              records 4
              capacity 2500000
              state ONLINE
              avg-record-size-bytes 1
              up-time 89h28m38.274713022s
              sync-duration 8.37766ms
          }
          instance {
              system-id 2
              instance-id 2
              records 4
              capacity 2500000
              state ONLINE
              avg-record-size-bytes 1
              up-time 89h29m37.934345015s
              sync-duration 8.877442ms
          }
      }
  }
  index {
      map {
          map-id 1
          instance {
              system-id 1
              instance-id 1
              records 4
              capacity 60000000
              state ONLINE
              up-time 89h38m16.119032086s
              sync-duration 2.012281769s
              leader false
              geo-replication-delay 10.529821ms
          }
          instance {
              system-id 1
              instance-id 2
              records 4
              capacity 60000000
              state ONLINE
              up-time 89h39m8.47664588s
              sync-duration 2.011171261s
              leader true
              leader-time 89h38m53.761213379s
              geo-replication-delay 10.252683ms
          }
          instance {
              system-id 2
              instance-id 1
              records 4
              capacity 60000000
              state ONLINE
              up-time 89h28m29.5479133s
              sync-duration 2.012101957s
              leader false
              geo-replication-delay 15.974538ms
          }
          instance {
              system-id 2
              instance-id 2
              records 4
              capacity 60000000
              state ONLINE
              up-time 89h29m11.633496562s
              sync-duration 2.011566639s
              leader true
              leader-time 89h28m51.29928233s
              geo-replication-delay 16.213323ms
          }
      }
  }

• CDL replication status

  Check whether four gRPC connections are established between the CDL EP session pods (of each namespace) across the racks in GRPC_Connections_to_RemoteSite panel of CDL Replication Stats Grafana dashboard. Check Grafana on both racks.

  

• Admin port status between the racks for geo-replication.

  Check heartbeat messages between geo-replication pods across the racks in Periodic_Heartbeat_to_Remote_Site panel of GR Statistics Grafana dashboard.

  

• BGP/BFD link status on rack

  Check whether neighborship with BGP peers is established in BGP Peers panel of BGP, BFD Statistics Grafan dashboard.

  

  Check whether BFD link is in connected state in BFD Link Status panel of BGP, BFD Statistics Grafana dashboard.

  

• Roles of each instances are in healthy state

  Check that in each rack the roles are not in STANDBY_ERROR state at any point of time.

  • Active/Standby Model: Roles should be in the following states on each rack

    Rack-1:

    show role instance-id 1
    result "PRIMARY"
    show role instance-id 2
    result "PRIMARY"

    Rack-2:

    show role instance-id 1
    result "STANDBY"
    show role instance-id 2
    result "STANDBY"

  • Active/Active Model: Roles should be in the following states on each rack.

    Rack-1:

    show role instance-id 1
    result "PRIMARY"
    show role instance-id 2
    result "STANDBY"

    Rack-2:

    show role instance-id 1
    result "STANDBY"
    show role instance-id 2
    result "PRIMARY"

On Rack-1

1. Verify that roles of both instances on Rack-1 are in STANDBY_ERROR.

   show role instance-id 1
   result "STANDBY_ERROR"

   show role instance-id 2
   result "STANDBY_ERROR"

2. Initiate reset role for both instances on Rack-1 to STANDBY. This step transitions the roles from STANDBY_ERROR/STANDBY_ERROR to STANDBY/STANDBY.

   geo reset-role instance-id 1 role standby
   geo reset-role instance-id 2 role standby

3. Verify that roles of both instances have moved to STANDBY on Rack-1.

   show role instance-id 1
   result "STANDBY"

   show role instance-id 2
   result "STANDBY"

4. Initiate switch role for instance-id 1 on Rack-2 to STANDBY with failback-interval of 30 seconds. This step transitions the roles of Rack-2 from PRIMARY/PRIMARY to STANDBY_ERROR/PRIMARY and Rack-1 from STANDBY/STANDBY to PRIMARY/STANDBY.

   geo switch-role instance-id 1 role standby [failback-interval 0]

5. Verify that roles of both instances on Rack-2 are in STANDBY_ERROR/PRIMARY.

   show role instance-id 1
   result "STANDBY_ERROR"

   show role instance-id 2
   result "PRIMARY"

6. Verify that roles of both instances on Rack-1 are in PRIMARY/STANDBY.

   show role instance-id 1
   result "PRIMARY"

   show role instance-id 2
   result "STANDBY"

7. Initiate reset role for instance-id 1 on Rack-2 to STANDBY. This step transitions the roles of Rack-2 from STANDBY_ERROR/PRIMARY to STANDBY/PRIMARY.

   geo reset-role instance-id 1 role standby

8. Verify that the roles of Rack-2 are in STANDBY/PRIMARY.

   show role instance-id 1
   result "STANDBY"

   show role instance-id 2
   result "PRIMARY"

On Rack-2

1. Verify that roles of both the instances on Rack-2 are in STANDBY_ERROR.

   show role instance-id 1
   result "STANDBY_ERROR"

   show role instance-id 2
   result "STANDBY_ERROR"

2. Initiate reset role for both instances on Rack-2 to STANDBY. This step transitions the roles from STANDBY_ERROR/STANDBY_ERROR to STANDBY/STANDBY.

   geo reset-role instance-id 1 role standby
   geo reset-role instance-id 2 role standby

3. Verify that the roles of both the instances move to STANDBY on Rack-2.

   show role instance-id 1
   result "STANDBY"

   show role instance-id 2
   result "STANDBY"

4. Initiate switch role for instance-id 2 on Rack-1 to STANDBY. This step transitions roles of Rack-1 from PRIMARY/PRIMARY to PRIMARY/STANDBY_ERROR and Rack-2 from STANDBY/STANDBY to STANDBY/PRIMARY.

   geo switch-role instance-id 2 role standby [failback-interval 0]

5. Verify that roles of instances on Rack-1 are in PRIMARY/STANDBY_ERROR mode.

   show role instance-id 1
   result "PRIMARY"

   show role instance-id 2
   result "STANDBY_ERROR"

6. Verify that roles of instances on Rack-2 are in STANDBY/PRIMARY mode.

   show role instance-id 1
   result "STANDBY"

   show role instance-id 2
   result "PRIMARY"

7. Initiate reset role for instance-id 2 on Rack-1 to STANDBY. This step transitions the roles of Rack-1 from PRIMARY/STANDBY_ERROR to PRIMARY/STANDBY.

   geo reset-role instance-id 2 role standby

8. Verify that roles of instances on Rack-1 are in PRIMARY/STANDBY.

   show role instance-id 1
   result "PRIMARY"

   show role instance-id 2
   result "STANDBY"

The following section describes KPIs.

ETCD/Cachepod Replication KPIs

The following table lists ETCD/Cachepod Replication KPIs.

Geo Rejected Role Change KPIs

The following table lists Geo Rejected Role Change KPIs.

Monitoring KPIs

The following table lists monitoring KPIs.

BFD KPIs

The following table lists BFD KPIs.

GR Instance Information

Geo Maintenance Mode

The following section provides details on GR-specific bulkstats.

bulk-stats query GR-BGP-Incoming-Failed-Routes
 expression "sum(bgp_incoming_failedrouterequest_total) by (namespace, interface, service_IP, next_hop, instance_id)"
 labels     [ instance_id interface next_hop service_IP ]
 alias      gr-bgp-routes-in
exit
bulk-stats query GR-Geo-Monitoring-Failure
 expression "sum(geo_monitoring_total{ControlActionNameType=~'MonitorPod|RemoteMsgHeartbeat|
RemoteMsgGetSiteStatus|RemoteSiteRoleMonitoring|RemoteClusterPodFailure|RemoteMsgNotifyFailover|
RemoteMsgNotifyPrepareFailover|MonitorVip',status!~'success|monitoring.*'}) by (namespace, 
AdminNode, ControlActionType, ControlActionNameType,  pod, status, status_code)"
 labels     [ pod AdminNode ControlActionNameType status status_code ]
 alias      gr-geo-monitoring-failure
exit
bulk-stats query GR-Geo-Monitoring-Success
 expression "sum(geo_monitoring_total{ControlActionNameType=~'MonitorPod|RemoteMsgHeartbeat|
RemoteMsgGetSiteStatus|RemoteSiteRoleMonitoring|RemoteClusterPodFailure|RemoteMsgNotifyFailover|
RemoteMsgNotifyPrepareFailover',status=~'success|monitoring.*'}) by (namespace, AdminNode, 
ControlActionType, ControlActionNameType,  pod, status)"
 labels     [ pod AdminNode ControlActionNameType status ]
 alias      gr-geo-monitoring
exit
bulk-stats query GR-Geo-Monitoring-Total
 expression "sum(geo_monitoring_total{ControlActionNameType=~'MonitorPod|RemoteMsgHeartbeat|
RemoteMsgGetSiteStatus|RemoteSiteRoleMonitoring|RemoteClusterPodFailure|RemoteMsgNotifyFailover
|RemoteMsgNotifyPrepareFailover|MonitorVip'}) 
by (namespace, AdminNode, ControlActionType, ControlActionNameType,  pod, status)"
 labels     [ pod AdminNode ControlActionNameType status ]
 alias      gr-geo-monitoring
exit
bulk-stats query GR-Geo-Replication-Failure
 expression "sum(geo_replication_total{ReplicationNode=~'CACHE_POD|ETCD|PEER',status!='success',
ReplicationRequestType='Response'}) by (namespace, ReplicationNode, ReplicationSyncType,
ReplicationReceiver,ReplicationRequestType,status,status_code)"
 labels     [ pod ReplicationNode ReplicationReceiver ReplicationRequestType ReplicationSyncType status status_code ]
 alias      gr-geo-replication-failure
exit
bulk-stats query GR-Geo-Replication-Success
 expression "sum(geo_replication_total{ReplicationNode=~'CACHE_POD|ETCD|PEER',
status='success',ReplicationRequestType='Response'}) by (namespace, ReplicationNode, 
ReplicationSyncType,ReplicationReceiver,ReplicationRequestType,status)"
 labels     [ pod ReplicationNode ReplicationReceiver ReplicationRequestType ReplicationSyncType status ]
 alias      gr-geo-replication-success
exit
bulk-stats query GR-Geo-Replication-Total
 expression "sum(geo_replication_total{ReplicationNode=~'CACHE_POD|ETCD|PEER'}) 
by (namespace, ReplicationNode, ReplicationSyncType,ReplicationReceiver,
ReplicationRequestType,pod)"
 labels     [ pod ReplicationNode ReplicationReceiver ReplicationRequestType ReplicationSyncType ]
 alias      gr-geo-replication-total
exit
bulk-stats query GR-Trigger-ResetRole-Api
 expression "sum(geo_monitoring_total{ControlActionNameType=~'TriggerGRApi|ResetRoleApi'}) 
by (namespace, AdminNode, ControlActionType, ControlActionNameType,  pod, status, status_code)"
 labels     [ pod AdminNode ControlActionNameType status status_code ]
 alias      gr-api
exit
bulk-stats query GR-CDL-Index-Replication
 expression "sum(consumer_kafka_records_total) by (pod, origin_instance_id)"
 labels     [ origin_instance_id pod ]
 alias      gr-cdl-index-replication
exit
bulk-stats query GR-CDL-Inter-Rack-Replications-Failures
 expression "sum(datastore_requests_total{local_request='0',errorCode!='0'}) by (operation,sliceName,errorCode)"
 labels     [ sliceName operation errorCode ]
 alias      gr-cdl-inter-rack-replications
exit
bulk-stats query GR-CDL-Inter-Rack-Replications-Success
 expression "sum(datastore_requests_total{local_request='0',errorCode='0'}) by (operation,sliceName,errorCode)"
 labels     [ sliceName operation errorCode ]
 alias      gr-cdl-inter-rack-replications
exit
bulk-stats query GR-CDL-Inter-Rack-Replications-Total
 expression "sum(datastore_requests_total{local_request='0'}) by (operation,sliceName,errorCode)"
 labels     [ sliceName operation errorCode ]
 alias      gr-cdl-inter-rack-replications
exit
bulk-stats query GR-CDL-Intra-Rack-Operations-Failures
 expression "sum(datastore_requests_total{local_request='1',errorCode!='0'}) by (operation,sliceName,errorCode)"
 labels     [ sliceName operation errorCode ]
 alias      gr-cdl-intra-rack-operations
exit
bulk-stats query GR-CDL-Intra-Rack-Operations-Success
 expression "sum(datastore_requests_total{local_request='1',errorCode='0'}) by (operation,sliceName,errorCode)"
 labels     [ sliceName operation errorCode ]
 alias      gr-cdl-intra-rack-operations
exit
bulk-stats query GR-CDL-Intra-Rack-Operations-Total
 expression "sum(datastore_requests_total{local_request='1'}) by (operation,sliceName,errorCode)"
 labels     [ errorCode operation sliceName ]
 alias      gr-cdl-intra-rack-operations
exit
bulk-stats query GR-CDL-Session-Count-Per-Slice
 expression sum(avg(db_records_total{namespace=~'$namespace',session_type='total'})by(systemId,sliceName))by(sliceName)
 labels     [ sliceName ]
 alias      gr-cdl-session-count-per-slice
exit
bulk-stats query GR-CDL-Session-Count-Per-System-ID
 expression sum(avg(db_records_total{namespace=~'$namespace',session_type='total'})
by(systemId,sliceName))by(systemId)
 labels     [ systemId ]
 alias      gr-cdl-session-count-per-system-id
exit
bulk-stats query GR-CDL-Slot-Records-Per-Slice
 expression "sum(slot_records_total{pod=~'.*',systemId!=''}) by (pod, sliceName)"
 labels     [ pod sliceName ]
 alias      gr-cdl-slot-records-per-slice
exit
bulk-stats query GR-CDL-Slot-Records-Per-System-ID
 expression "sum(slot_records_total{pod=~'.*',systemId!=''}) by (pod, systemId)"
 labels     [ pod systemId ]
 alias      gr-cdl-slot-records-per-system-id
exit
bulk-stats query GR-CDL-Total-Session-Count
 expression "sum(db_records_total{namespace=~'$namespace',session_type='total'}) by (systemId,sliceName)"
 labels     [ sliceName systemId ]
 alias      gr-cdl-total-session-count
exit

For more information on GR-related statistics, see the following:

• In RADIUS statistics, you can filter GR-specific statistics using grInstId label.

  For more information, see the UCC 5G Session Management Function - Metrics Reference.

• In GTP Endpoint statistics, you can filter GR-specific statistics using gr_instance_id label.

  For more information, see the UCC 5G Session Management Function - Metrics Reference.

• In SMF statistics, you can filter GR-specific statistics using gr_instance_id label.

  For more information, see the UCC 5G Session Management Function - Metrics Reference.

• In REST Endpoint statistics, you can filter GR-specific statistics using gr_instance_id label.

  For more information, see the UCC 5G Session Management Function - Metrics Reference.

• In IPAM-related statistics, you can filter GR-specific statistics using grInstId label.

  For more information, see the UCC 5G Session Management Function - Metrics Reference.

The following section provides details on GR alerts.

BFD Alerts

The following table list alerts for rule group BFD with interval-seconds as 60.

GR Alerts

The following table list alerts for rule group GR with interval-seconds as 60.

CDL Alerts

The following table list alerts for rule group CDL with interval-seconds as 60.

In the inter-site redundancy deployment, the Multi-exit Discriminator (MED) attribute, which is a non-transitive BGP attribute, is used to inform the neighboring routers of the best path to reach the active service VIP. However, MED values are not shared beyond neighboring routers to their eBGP peers.

In the inter-rack deployment model, AS-path prepending technique is used in BGP to influence the path selection of routes advertised to other BGP routers. AS-path prepending technique makes a certain route more favorable over another by manipulating the AS-path in the BGP route advertisement. The difference in the number of AS-paths getting prepended by software logic helps to provide guidance to upstream routers to choose the best routable path.

Following table captures the number of AS-path prepend required for various combination of service VIP and interface bonding status.

To enable the inter-site deployment, you must configure the AS-path prepend flag in the BGP configuration.

Following is the sample configuration:

config 
  router local_as_number 
    bfd bfd_configuration 
    interface bgp_local_interface 
    prepend as-path { false | true } 
    exit 
   exit 

NOTES:

• interface bgp_local_interface : Specify BGP local interface.

• prepend as-path { false | true } : Select true to enable the Inter-site deployment. By default this option is disabled.

  If this option is set to false , the logical SMF deployment follows the inter-rack model.

In the inter-rack solution, leaf and spine servers were incorporated into the same AS-path to enable the utilization of VxLAn techniques. This deployment model allows for the support, monitoring, and exchange of redundancy data and events among various pods, while remaining reachable across racks. Specifically, the external IP address of the Geo-pod on both racks were integrated into a common subnet, enabling data exchange without the need for route discovery as they shared the same VLAN. Additionally, communication between racks remained within the boundaries of the leaf and spine servers, without the need for packets to traverse beyond them.

The following PODs require external connectivity between the sites:

• Geo-pod

• CDL-ep (s)

• CDL-Kafka-server(s)

Unlike service VIPs, these IP addresses do not require floating across sites. A routing decision needs to be made only once during production deployment. Therefore, static routing was used instead of advertising these external IP addresses through BGP.

Static routes are added to the local K8S cluster protocol nodes for reaching remote CDL/Kafka/Geo Networks. The leafs automatically advertise all connected local CDL/Kafka/Geo Networks to the spines. The spines redistribute those routes to the Arg router. The Arg routers know how to reach both Sites' CDL/Kafka/Geo Networks. If a packet is destined for a remote CDL/Kafka/Geo network, the local K8S protocol node has a static route that points to the local leaf. The packet then goes to the spine. The spines have a default route that points to the Arg router. The Arg routers have proper routes to send packets to the remote site.

In addition, the Arg routers are configured with the "default-originate" keyword, which designates the ARG routers as the default gateway for the spines. Consequently, the spines redirect all traffic for which a routing path is not available in the spines to the Arg routers.

This section describes that how the inter-site redundancy deployment works in the different scenarios:

In a scenario with two sites and both operational, each site have one primary/active GR (Gateway Router) instance. The endpoint configured on each site carries traffic corresponding to its active instance. It is possible that the Aggregation Router (ARG) may not be directly connected to the sites, and there could be multiple autonomous systems between the ARG and sites. The ARG router is responsible for the convergence of the routes from both sites.

Following are the various scenarios for the site redundancy:

• During sunny weather conditions, Site A advertises the best route for GR1 and a less favorable route for the endpoint associated with GR2.

  On the ARG, following are the routes:

  • Routes from Site A

    • 209.165.200.225 AS-Path 65000 65000 65000 65000

    • 209.165.200.235 AS-Path 65000 65000 65000 65000 65000 65000 65000

  • Routes from Site B

    • 209.165.200.225 AS-Path 64000 64000 64000 64000 64000 64000

    • 209.165.200.235 AS-Path 64000 64000 64000 64000

  • Best Path:

    • Site-A 209.165.200.225 AS-path 65000 65000 65000 65000

    • Site -B 209.165.200.235 AS-path 64000 64000 64000 64000

Now if the Site A goes down:

The BGP session on the ARG routers goes down after the BGP convergence time, resulting in the withdrawal of routes learned from Site A. The current state of routes at ARG is as follows.

• Routes from Site B:

  • 209.165.200.225 AS-Path 64000 64000 64000 64000 64000 64000 64000

  • 209.165.200.235 AS-Path 64000 64000 64000 64000

  Now the traffic for both the endpoints 209.165.200.225 and 209.165.200.225 is being sent towards site B.

• The traffic monitoring feature at site B has identified traffic for the IP address 209.165.200.225. Once the packet threshold for this IP address is exceeded, the GeoReplication pod triggers the transition of the GR-1 instance from standby mode to active mode.

• After the Geo-pod notifies the BGPSpeaker-pod of the transition of GR1 to the primary instance, the BGPSpeaker-pod readvertises the routes for the GR1 endpoint using the most favorable routing path.

• Now the routes in the ARG are updated with following information:

  • Routes from site B:

    • 209.165.200.225 AS-Path 64000 64000 64000 64000

    • 209.165.200.235 AS-Path 64000 64000 64000 64000

As part of this feature, the following Key Performance Indicators (KPIs) are associated with split brain support.

Geo Split Brain Detection Total KPI

Geo Split Brain Resolution Total

Geo Role Reason Change Total