This chapter describes cnBNG installation and configuration using the Ultra Cloud Core Subscriber Microservices Infrastructure (SMI) Cluster Manager and the BNG Operations (Ops) Center. The BNG Ops Center is based on the ConfD command line interface (CLI).

To install the SMI Cluster Manager, refer to the "Deploying the SMI Cluster Manager on VMware vCenter" section in the Ultra Cloud Core Subscriber Microservices Infrastructure - Deployment Guide.

The SMI Ops Center is the platform to install the cnBNG cluster with the offline or online repository. It is mandatory to install the SMI Ops Center to set up and access the BNG Ops Center.

Note	To access the offline or online repository, contact your Cisco Account Manager or representative to get access to the offline or online repository.

This section describes how to install cnBNG and access the BNG Ops Center.

The Ultra Cloud Core SMI platform is responsible for setting up and managing the Cloud Native Broadband Network Gateway application.

Prerequisites

• Ensure that the node labels are according to the pod deployment layout.

• Ensure that the external VIPs are according to the requirement of NF.

• Enable Istio for pod to pod traffic load balancing.

Node Labels are key and value pairs that are attached to nodes at cluster synchronization. Each node can have a set of key and value labels defined. Each key must be unique for a node. With labels, users can map their NF pods onto nodes in a loosely coupled manner.

Important

The pod-level labeling configuration is applicable only when the cnBNG CP is deployed on a bare metal server. Ensure to configure the node label on the SMI cluster deployer before mapping the pods. Following is the sample command for master-1 labeling: [cndp-clpnc-cm-cm-primary] SMI Cluster Deployer (config-nodes-master-1)# k8s node-labels smi.cisco.com/svc-type bng-node

To map the pods with node labels, use the following sample configuration:

config 
   k8 label protocol-layer key label_key value label_value 
   k8 label service-layer key label_key value label_value 
   k8 label cdl-layer key label_key value label_value 
   k8 label oam-layer key label_key value label_value 
   end 

Following is an example configuration of pod to node-label mapping:

k8 label protocol-layer key smi.cisco.com/node-type value bng-proto
exit
k8 label service-layer key vm-type value bng-svc
exit
k8 label cdl-layer key smi.cisco.com/node-type value bng-cdl
exit
k8 label oam-layer key smi.cisco.com/node-type value oam
exit

High Availablity on cnBNG CP is validated on BareMetal server deployment. For more informaton about High Availablity, see High Availability and CP Reconciliation.

The cnBNG CNF Deployment on AIO BareMetal Server explains the process of onboarding a cnBNG Cloud Native Function (CNF) on the Cloud Native Deployment Platform (CNDP) on the BareMetal all-in-one (AIO) Kubernetes (K8s) cluster.

In the AIO deployment, all the management VMs are hosted on a different UCS server, however, this depends on the deployment strategy.

The cnBNG CNF is hosted on another UCS server referred as AIO server. During installation, the Cluster Manager (CM) accesses the AIO via the Cisco Integrated Management Controller (IMC) interface and adds the respective image and SMI packages to complete the installation.

Note	The management VMs are the Inception, Cluster Manager, and RADIUS servers.

The CNDP is a 'SMI Bare-Metal'. The Cluster Manager uses REST APIs (instead of VIM APIs) of the management cards, which are on the servers, to create a set of Linux servers and then loads the K8s software.

Before installing network functions (NF) on the Cluster Manager, the common containerized software from SMI is installed. For example, monitoring and logging. The SMI NFs include their own common containerized software.

You can now deploy the cnBNG cluster using the Inception server alone. You do not require the SMI Cluster Manager, which was previously used along with the Inception server to deploy the cnBNG cluster. This enhancement can help you save on hardware resources (servers).

Starting with Release 2025.01, the cnBNG control-plane can also be deployed on a Red Hat OpenShift cluster.

Supported Functionalities:

The following functionalities are supported:

• OpenShift Container Platform (OCP) versions 4.16 and 4.17 are supported.

• Single Node OpenShift (SNO) is supported.

• Both cnBNG Standalone and CP-GR are supported.

• Only IPOE is supported.

• Compatible with IPv4-only and dual-stack configurations.

Requirements:

The following software versions are required:

• CEE version: 2025.01 or above

• cnBNG-CP version: 2025.01.0 or above

Monitoring and Management:

The following monitoring and management functionalities are available:

• Monitoring dashboards are available via CEE.

• Local storage support is provided through CEE.

• Cisco TAC debug support is available through CEE.

• Log and metric forwarding capabilities are provided via CEE.

ISTIO:

ISTIO is not supported.

Note	This guide does not cover the installation of the OpenShift cluster or the setup of Helm and the Docker registry.

Set Up the cnBNG Application on an OCP Cluster

After installing the OpenShift Container Platform (OCP) cluster, follow these steps to set up the cnBNG application:

1. Add the hostname to the /etc/hosts file.

2. Assign the necessary node labels.

3. Execute the following command to enable support for external IP addresses:

   oc patch network cluster --type merge -p '{ "spec": { "externalIP": { "policy": {"allowedCIDRs": ["0.0.0.0/0", "::/0"]}}}}'

4. Add Helm and Docker repositories for the BNG and CEE charts.

5. Install the cee-ops-center and bng-ops-center charts using the specified inputs.

   Ensure that your configuration includes the following global settings:

   global:
     registry: <docker-registry>
     istio: false
     singleNode: true
     useVolumeClaims: true
     caasModel: "openshift"
     pathBasedIngress: false
     smiNfMonitoring: true
     smiPlatformMonitoring: true
     ingressHostname: <ingress>
     imagePullPolicy: IfNotPresent
     imagePullSecrets:
       - name: <secret-name>
     networkPolicy:
       enabled: false

   The CLI commands for bng-ops-center and cee-ops-center are consistent with CNDP, allowing you to enable additional functionalities using the same setup.

The current solution for a cnBNG Control Plane consists of two racks working together to provide high availability. Each rack contains four Kubernetes (K8s) clusters and is in the process of scaling up to a fifth Kubernetes cluster.

This MOP outlines the steps to scale a cnBNG Control Plane deployment from Half-Rack to Full-Rack.

The procedure includes

• adding new servers

• scaling up Kubernetes (K8s) clusters

• scaling up CDL, and

• scaling up services.

Before you begin

Perform these steps prior to the actual maintenance window.

• Generate configuration files for K8s node scale up.

  • Prepare and review the config change payload for scaling up each cluster.

  • Verify that label configuration is in line with the current pod.

  • Load and merge change payload files into the Deployer configuration and perform a commit dry-run.

  • Verify that all changes are as per the expectation.

  • Exit Deployer config mode without issuing a commit.

    Note	Actual commits are done during the maintenance window.

Estimated Time for Procedure

Software Used

Adding and Wiring New Servers

1. Update the firmware.

   Ensure that new servers have the same firmware version as existing nodes.

   Note	This step should be done prior to the actual maintenance window before racking the servers in the pod.

2. Install and wire servers.

   • Install new servers in the same rack as existing servers.

   • Wire servers using available ports in current switches.

3. Power up servers.

4. Verify that there are no existing boot or virtual disks on these new servers.

5. Ensure that disks appear as Unconfigured-good.

6. Check the CIMC IP addresses of these new servers for connectivity.

7. Configure switch ports.

   • Configure leaf switch ports for new servers.

     Use the same configuration as other leaf ports connected to existing worker nodes.

   • Verify from the ClusterManager Linux shell that you can ping CIMC IP addresses of new nodes.

Scale Up Kubernetes Cluster

You must scale up the Kubernetes cluster in each rack of the pod.

Follow these steps to scale up the Kubernetes cluster:

1. Perform a planned GR switchover to move all cnBNG instances to the RackInService as primary.

   Note	RackInService refers to the rack hosting all primary GR instances of all cnBNGs in the pod.

2. Reset states of cnBNG GR instances in STANDBY_ERROR state to STANDBY state in RackBeingScaled.

   Note	RackBeingScaled refers to the rack undergoing K8s node scaling.

3. Verify that all GR instances in RackInService are in PRIMARY state and those in RackBeingScaled are in STANDBY state.

4. Set geo-maintenance mode in all cnBNGs in both RackBeingScaled and RackInService.

5. Shut down all cnBNGs in RackBeingScaled.

6. Add the configuration in Deployer via CLI or NSO load merge config change payload files to scale up RackBeingScaled.

   See the section "K8s Cluster scale-up config example" for example configuration.

7. Manually add networks in the netplan on the new server.

   • Command: sudo vi /etc/netplan/50-cloud-init.yaml

   • Apply: sudo netplan apply

   Note	If the server is new, you don't have to add netplan manually. If you are reusing a server, write a netplan manually.

8. [Optional] If you are reusing an existing server, reset it using the CLI command:

   • kubeadm reset --force

   • Clean up (delete) the virtual drive by logging into the CIMC of that server.

9. Issue a cluster sync to scale up RackBeingScaled.

   • Command: clusters svi-cnbng-gr-tb1 actions sync run debug true

10. Monitor the cluster sync progress by logging into the Cluster Manager.

11. Verify if all nodes in the K8s cluster in RackBeingScaled are in ready state.

12. Verify if all pods in the K8s cluster in RackBeingScaled are in good running state.

13. Start up all cnBNGs in RackBeingScaled.

14. Verify if the added nodes appear in the cnBNG Ops Center configurations.

15. Verify if both GR instances of all cnBNGs in RackBeingScaled are in STANDBY state.

16. Unset geo-maintenance mode in all cnBNGs in both RackBeingScaled and RackInService

Post-Requirements:

Repeat this procedure to scale up K8s cluster in each rack of the Pod.

Scale Up CDL

Follow these steps to scale up the CDL:

1. Verify that all cnBNGs in the pod are in good state.

2. Update CDL Grafana dashboard to monitor the total number of index keys that have been rebalanced.

   • Query: sum(index_rebalanced_keys_total{namespace="$namespace"}) by (cdl_slice, shardId, pod)

   • Legend: irkt-{{cdl_slice}}-{{shardId}}-{{pod}}

   Note	CDL rebalancing happens once the CDL rebalance query is issued.

3. Make rack-1 as ACTIVE and rack-2 as STANDBY.

   1. Unset geo maintenance mode in all cnBNGs in both racks (if previously set).

   2. Make both GR instances of all cnBNGs in Rack1 PRIMARY.

   3. Make both GR instances of all cnBNGs in Rack2 STANDBY.

   4. Set geo maintenance mode in all cnBNGs in both racks.

4. Add configuration changes to Rack2 (STANDBY)

   1. Shut down all cnBNGs in Rack2.

   2. Add configuration changes to all cnBNGs in Rack2 (see "CDL Configuration changes" section).

   3. Start up all cnBNGs in Rack2.

5. Verify the cnBNG State

   • Ensure that all cnBNGs are in good state.

   • Both GR instances in all cnBNGs in Rack2 should be in STANDBY state.

   • Both GR instances in all cnBNGs in Rack1 should be in PRIMARY state.

6. Verify CDL indices and slots sync

   1. Verify if CDL indices and slots in Rack2 can sync with remote peers (may take ~15 mins).

7. Switch rack states

   • Unset geo maintenance mode in all cnBNGs in both racks.

   • Make both GR instances of all cnBNGs in Rack2 PRIMARY.

   • Make both GR instances of all cnBNGs in Rack1 STANDBY.

   • Set geo maintenance mode in all cnBNGs in both racks.

8. Add configuration changes to Rack1 (STANDBY)

   1. Shut down all cnBNGs in Rack1.

   2. Add configuration changes to all cnBNGs in Rack1 (see "CDL Configuration changes" section).

   3. Start up all cnBNGs in Rack1.

9. Verify the state of all cnBNGs.

   • Ensure that all cnBNGs are in good state.

   • Both GR instances in all cnBNGs in Rack1 should be in STANDBY state.

   • Both GR instances in all cnBNGs in Rack2 should be in PRIMARY state.

10. Verify if CDL indices and slots in Rack1 can sync with remote peers (may take ~15 mins).

11. Trigger CDL index rebalancing.

    • Issue the following command in all cnBNGs in the current STANDBY rack:

      cdl rebalance-index run

12. Verify CDL index rebalancing completion.

    1. Monitor progress with the following command in the STANDBY rack cnBNG:

       cdl rebalance-index status

    2. Validate rebalancing with the following command in the STANDBY rack cnBNG:

       cdl rebalance-index validate

13. Remove CDL scale up mode configuration:

    config
      cdl datastore session
      no mode
      commit
      end
    

14. Unset Geo maintenance mode in all cnBNGs in both racks.

15. Initiate GR switchovers

    Switch all cnBNGs in the pod to ACTIVE-ACTIVE mode.

16. Verify that all cnBNGs are operating in sunny day mode.

Scale Up Services

You can now scale up cnBNG Services.

Note	There is no restriction to always scale up Cluster1 before Cluster2. You can scale up any cluster first, depending on the existing role set on the cluster.

Follow these steps to scale up cnBNG services:

1. Make Rack-1 Active and Rack-2 Standby

   • Unset geo maintenance mode in all cnBNGs in both racks (if previously set).

   • Make both GR instances of all cnBNGs in Rack1 PRIMARY.

   • Make both GR instances of all cnBNGs in Rack2 STANDBY.

   • Set geo maintenance mode in all cnBNGs in both racks.

2. Add configuration changes in Rack-2 (STANDBY).

   1. Shut down all cnBNGs in Rack2.

   2. Add the configuration changes to all cnBNGs in Rack2 (see the "Service scaleup config changes" section).

   3. Start up all cnBNGs in Rack2.

3. Verify the cnBNG state.

   • Ensure that all cnBNGs are in good state.

   • Both GR instances in all cnBNGs in Rack2 should be in STANDBY state.

   • Both GR instances in all cnBNGs in Rack1 should be in PRIMARY state.

4. Verify that the scale-up configuration changes are applied (may take approximately 15 mins).

5. Make Rack-2 active and Rack-1 standby

   1. Unset geo maintenance mode in all cnBNGs in both racks.

   2. Make both GR instances of all cnBNGs in Rack2 PRIMARY.

   3. Make both GR instances of all cnBNGs in Rack1 STANDBY.

   4. Set Geo maintenance mode in all cnBNGs in both racks.

6. Add configuration changes in Rack-1 (STANDBY)

   1. Shutdown all cnBNGs in Rack1.

   2. Add the configuration changes to all cnBNGs in Rack1 (see "Service Scale up Configuration" section).

   3. Start up all cnBNGs in Rack1.

7. Verify the cnBNG State.

   • Ensure that all cnBNGs are in good state.

   • Both GR instances in all cnBNGs in Rack1 should be in STANDBY state.

   • Both GR instances in all cnBNGs in Rack2 should be in PRIMARY state.

8. Verify that the scale-up configuration changes are applied (may take ~15 mins).

9. Remove the geo maintenance mode in both clusters.

10. Move both CP-GR clusters to their original state.

Configuration Examples

K8s Cluster Scale Up:

Scale up configurations for both the servers are available in the following combinations:

• Node Scaling for Service only

• Node Scaling for CDL only

• Node Scaling for both CDL and Service

Node Scaling for Service Only

clusters svi-cnbng-gr-tb1
 nodes server-5
  host-profile bng-ht-sysctl-enable
  k8s node-type worker
  k8s ssh-ip 1.1.111.15
  k8s ssh-ipv6 2002:4888:1:1::111:15
  k8s node-ip 1.1.111.15
  k8s node-ipv6 2002:4888:1:1::111:15
  k8s node-labels smi.cisco.com/svc-type service
  exit
  ucs-server cimc ip-address 10.81.103.78
  initial-boot netplan vlans bd0.mgmt.3103
   addresses [ 10.81.103.119/24 ]
   gateway4  10.81.103.1
  exit
  initial-boot netplan vlans bd1.k8s.111
   addresses [ 1.1.111.15/24 2002:4888:1:1::111:15/112 ]
   routes 203.203.203.50/32 1.1.111.1
   exit
   routes 2002:4888:203:203::203:50/128 2002:4888:1:1::111:1
   exit
  exit
  initial-boot netplan vlans bd1.inttcp.104
   dhcp4     false
   dhcp6     false
   addresses [ 1.1.104.15/24 2002:4888:1:1::104:15/112 ]
   id        104
   link      bd1
   routes 2.2.104.0/24 1.1.104.1
   exit
   routes 2002:4888:2:2::104:0/112 2002:4888:1:1::104:1
   exit
  exit
  os tuned enabled
 exit
exit

clusters svi-cnbng-gr-tb2
 nodes server-5
  host-profile bng-ht-sysctl-enable
  k8s node-type worker
  k8s ssh-ip 2.2.112.15
  k8s ssh-ipv6 2002:4888:2:2::112:15
  k8s node-ip 2.2.112.15
  k8s node-ipv6 2002:4888:2:2::112:15
  k8s node-labels smi.cisco.com/svc-type service
  exit
  ucs-server cimc ip-address 10.81.103.58
  initial-boot netplan vlans bd0.mgmt.3103
   addresses [ 10.81.103.68/24 ]
   gateway4  10.81.103.1
  exit
  initial-boot netplan vlans bd1.inttcp.104
   dhcp4     false
   dhcp6     false
   addresses [ 2.2.104.15/24 2002:4888:2:2::104:15/112 ]
   id        104
   link      bd1
   routes 1.1.104.0/24 2.2.104.1
   exit
   routes 2002:4888:1:1::104:0/112 2002:4888:2:2::104:1
   exit
  exit
  initial-boot netplan vlans bd1.k8s.112
   addresses [ 2.2.112.15/24 2002:4888:2:2::112:15/112 ]
   routes 203.203.203.50/32 2.2.112.1
   exit
   routes 2002:4888:203:203::203:50/112 2002:4888:2:2::112:1
   exit
  exit
  os tuned enabled
 exit
exit

Node Scaling for CDL Only

clusters svi-cnbng-gr-tb1
 nodes server-5
  host-profile bng-ht-sysctl-enable
  k8s node-type worker
  k8s ssh-ip 1.1.111.15
  k8s ssh-ipv6 2002:4888:1:1::111:15
  k8s node-ip 1.1.111.15
  k8s node-ipv6 2002:4888:1:1::111:15
  k8s node-labels smi.cisco.com/sess-type cdl-node
  exit
  ucs-server cimc ip-address 10.81.103.78
  initial-boot netplan vlans bd0.mgmt.3103
   addresses [ 10.81.103.119/24 ]
   gateway4  10.81.103.1
  exit
  initial-boot netplan vlans bd1.cdl.103
   dhcp4     false
   dhcp6     false
   addresses [ 1.1.103.15/24 2002:4888:1:1::103:15/112 ]
   id        103
   link      bd1
   routes 2.2.103.0/24 1.1.103.1
   exit
   routes 2002:4888:2:2::103:0/112 2002:4888:1:1::103:1
   exit
  exit
  initial-boot netplan vlans bd1.k8s.111
   addresses [ 1.1.111.15/24 2002:4888:1:1::111:15/112 ]
   routes 203.203.203.50/32 1.1.111.1
   exit
   routes 2002:4888:203:203::203:50/128 2002:4888:1:1::111:1
   exit
  exit
  os tuned enabled
 exit
exit

clusters svi-cnbng-gr-tb2
 nodes server-5
  host-profile bng-ht-sysctl-enable
  k8s node-type worker
  k8s ssh-ip 2.2.112.15
  k8s ssh-ipv6 2002:4888:2:2::112:15
  k8s node-ip 2.2.112.15
  k8s node-ipv6 2002:4888:2:2::112:15
  k8s node-labels smi.cisco.com/sess-type cdl-node
  exit
  ucs-server cimc ip-address 10.81.103.58
  initial-boot netplan vlans bd0.mgmt.3103
   addresses [ 10.81.103.85/24 ]
   gateway4  10.81.103.1
  exit
  initial-boot netplan vlans bd1.cdl.103
   dhcp4     false
   dhcp6     false
   addresses [ 2.2.103.15/24 2002:4888:2:2::103:15/112 ]
   id        103
   link      bd1
   routes 1.1.103.0/24 2.2.103.1
   exit
   routes 2002:4888:1:1::103:0/112 2002:4888:2:2::103:1
   exit
  exit
  initial-boot netplan vlans bd1.k8s.112
   addresses [ 2.2.112.15/24 2002:4888:2:2::112:15/112 ]
   routes 203.203.203.50/32 2.2.112.1
   exit
   routes 2002:4888:203:203::203:50/112 2002:4888:2:2::112:1
   exit
  exit
  os tuned enabled
 exit
exit

Node Scaling for both CDL and Services

clusters svi-cnbng-gr-tb1
 nodes server-5
  host-profile bng-ht-enable
  k8s node-type worker
  k8s ssh-ip 1.1.111.15
  k8s ssh-ipv6 2002:4888:1:1::111:15
  k8s node-ip 1.1.111.15
  k8s node-ipv6 2002:4888:1:1::111:15
  k8s node-labels smi.cisco.com/sess-type cdl-node
  exit
  k8s node-labels smi.cisco.com/svc-type service
  exit
  ucs-server cimc ip-address 10.81.103.78
  initial-boot netplan vlans bd0.mgmt.3103
   addresses [ 10.81.103.119/24 ]
   gateway4  10.81.103.1
  exit
  initial-boot netplan vlans bd1.inttcp.104
   dhcp4     false
   dhcp6     false
   addresses [ 1.1.104.15/24 2002:4888:1:1::104:15/112 ]
   id        104
   link      bd1
   routes 2.2.104.0/24 1.1.104.1
   exit
   routes 2002:4888:2:2::104:0/112 2002:4888:1:1::104:1
   exit
  exit
  initial-boot netplan vlans bd1.cdl.103
   dhcp4     false
   dhcp6     false
   addresses [ 1.1.103.15/24 2002:4888:1:1::103:15/112 ]
   id        103
   link      bd1
   routes 2.2.103.0/24 1.1.103.1
   exit
   routes 2002:4888:2:2::103:0/112 2002:4888:1:1::103:1
   exit
  exit
  initial-boot netplan vlans bd1.k8s.111
   addresses [ 1.1.111.15/24 2002:4888:1:1::111:15/112 ]
   routes 203.203.203.50/32 1.1.111.1
   exit
   routes 2002:4888:203:203::203:50/128 2002:4888:1:1::111:1
   exit
  exit
  os tuned enabled
 exit
exit

clusters svi-cnbng-gr-tb2
 nodes server-5
  host-profile bng-ht-enable
  k8s node-type worker
  k8s ssh-ip 2.2.112.15
  k8s ssh-ipv6 2002:4888:2:2::112:15
  k8s node-ip 2.2.112.15
  k8s node-ipv6 2002:4888:2:2::112:15
  k8s node-labels smi.cisco.com/sess-type cdl-node
  exit
  k8s node-labels smi.cisco.com/svc-type service
  exit
  ucs-server cimc ip-address 10.81.103.58
  initial-boot netplan vlans bd0.mgmt.3103
   addresses [ 10.81.103.85/24 ]
   gateway4  10.81.103.1
  exit
  initial-boot netplan vlans bd1.inttcp.104
   dhcp4     false
   dhcp6     false
   addresses [ 2.2.104.15/24 2002:4888:2:2::104:15/112 ]
   id        104
   link      bd1
   routes 1.1.104.0/24 2.2.104.1
   exit
   routes 2002:4888:1:1::104:0/112 2002:4888:2:2::104:1
   exit
  exit
  initial-boot netplan vlans bd1.cdl.103
   dhcp4     false
   dhcp6     false
   addresses [ 2.2.103.15/24 2002:4888:2:2::103:15/112 ]
   id        103
   link      bd1
   routes 1.1.103.0/24 2.2.103.1
   exit
   routes 2002:4888:1:1::103:0/112 2002:4888:2:2::103:1
   exit
  exit
  initial-boot netplan vlans bd1.k8s.112
   addresses [ 2.2.112.15/24 2002:4888:2:2::112:15/112 ]
   routes 203.203.203.50/32 2.2.112.1
   exit
   routes 2002:4888:203:203::203:50/112 2002:4888:2:2::112:1
   exit
  exit
  os tuned enabled
 exit
exit

CDL Configuration Changes

cdl datastore session
 label-config    session
 geo-remote-site [ 2 ]
 mode scale-up
 slice-names     [ 1 2 ]
 overload-protection disable true
 endpoint go-max-procs 16
 endpoint replica 3
 endpoint copies-per-node 2
 endpoint settings slot-timeout-ms 750
 endpoint external-ip 1.1.103.100
 endpoint external-ipv6 2002:4888:1:1::103:100
 endpoint external-port 8882
 index go-max-procs 8
 index replica 2
 index prev-map-count 2
 index map    3
 features instance-aware-notification enable true
 features instance-aware-notification system-id 1
  slice-names [ 1 ]
 exit
 features instance-aware-notification system-id 2
  slice-names [ 2 ]
 exit
 slot go-max-procs 8
 slot replica 2
 slot map     6
 slot notification limit           1500
 slot notification max-concurrent-bulk-notifications 20
exit


cdl label-config session
 endpoint key smi.cisco.com/sess-type
 endpoint value cdl-node
 slot map 1
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 2
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 3
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 4
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 5
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 6
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 index map 1
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 index map 2
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 index map 3
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
exit

cdl datastore session
 label-config    session
 geo-remote-site [ 1 ]
 mode scale-up
 slice-names     [ 1 2 ]
 overload-protection disable true
 endpoint go-max-procs 16
 endpoint replica 3
 endpoint copies-per-node 2
 endpoint settings slot-timeout-ms 750
 endpoint external-ip 2.2.103.100
 endpoint external-ipv6 2002:4888:2:2::103:100
 endpoint external-port 8882
 index go-max-procs 8
 index replica 2
 index prev-map-count 2
 index map    3
features instance-aware-notification enable true
 features instance-aware-notification system-id 1
  slice-names [ 1 ]
 exit
 features instance-aware-notification system-id 2
  slice-names [ 2 ]
 exit
 slot go-max-procs 8
 slot replica 2
 slot map     6
 slot notification limit           1500
 slot notification max-concurrent-bulk-notifications 20
exit


cdl label-config session
 endpoint key smi.cisco.com/sess-type
 endpoint value cdl-node
 slot map 1
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 2
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 3
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 4
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 5
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 slot map 6
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 index map 1
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 index map 2
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
 index map 3
  key   smi.cisco.com/sess-type
  value cdl-node
 exit
exit

The commands that are required for node scaling are highlighted.

Service Scale up Configuration

Scaling up services by increasing replica counts and nodes is supported. Reducing replicas and nodes is not supported and it will impact the system.

instance instance-id 1
 endpoint dhcp
  replicas 4
  nodes    2
 exit
exit
 endpoint sm
  replicas 6
  nodes    2
 exit
exit

instance instance-id 2
 endpoint dhcp
  replicas 4
  nodes    2
 exit
exit
 endpoint sm
  replicas 6
  nodes    2
 exit
exit

instance instance-id 1
 endpoint dhcp
  replicas 4
  nodes    3
 exit
exit
 endpoint sm
  replicas 6
  nodes    3
 exit
exit

instance instance-id 2
 endpoint dhcp
  replicas 4
  nodes    3
 exit
exit
 endpoint sm
  replicas 6
  nodes    3
 exit
exit