Cisco Optical Network Controller Installation Guide, Releases 26.x.x

Set up the supercluster

Updated: May 7, 2026

Want to summarize with AI?

Use this procedure to set up the supercluster, including SSH access, static eastbound routes, cluster joining, connectivity checks, BGP routing, and service validation.

Set up the supercluster by completing the required configuration and validation tasks.

Complete the subtasks in the order listed to prepare networking, routing, and cluster membership before starting the supercluster.

Before you begin

Before you begin, you must have created three VMs for geo-redundant deployment of Cisco Optical Network Controller. For more details, see Install and deploy geo-redundant Cisco Optical Network Controller

Procedure

	1.	Complete Connect to the supercluster virtual machines.
	2.	Complete Configure eastbound routes between supercluster nodes.
	3.	Complete Join clusters into a supercluster.
	4.	Complete Verify connectivity and start the supercluster.
	5.	Complete Configure Border Gateway Protocol for supercluster routing.
	6.	Complete Validate supercluster services and version.

The supercluster setup tasks are complete.

Connect to supercluster VM using SSH keys

Connect to each supercluster VM using SSH keys for secure access.

Use the PEM key generated during SSH key setup to access each node.

Before you begin

Confirm that three VMs have been created for geo-redundant deployment of Cisco Optical Network Controller. For more details, see Install and deploy geo-redundant Cisco Optical Network Controller.
Verify that each VM is powered on. Wait for the IP addresses for the VMs on vSphere to appear.

Follow these steps to connect to the supercluster virtual machines.

Procedure

	1.	Connect to each VM using the PEM key generated during SSH Key Generation.
	2.	Log in to each VM using the private key. `# ssh -i <private-key_file> nxf@<node_ip>` Note If you are prompted for a password, there might be a problem with the key. If your SSH key has a passphrase, the system prompts you for the passphrase. If you are prompted for a password even after entering your SSH key passphrase, your PEM key might be wrong or corrupted. If the command times out, check your network settings and make sure the node is reachable. After the nodes are deployed, check the OVA deployment progress in the Tasks console of vSphere Client. Upon successful deployment, Cisco Optical Network Controller can take about 20 minutes to boot. The default user ID is admin. Set the password using the `sedo security user set admin --password` command.

You are connected to each supercluster VM.

What to do next

Configure static eastbound routes between supercluster nodes

Configure static routes to allow eastbound traffic to pass between supercluster nodes. If peer node eastbound IPs are in different subnets, create static routes for eastbound traffic between the nodes.

Before you begin

Connect to supercluster VM using SSH keys

Follow these steps to configure eastbound routes.

Procedure

	1.	Navigate to the configuration directory. `cd /etc/systemd/network/`
	2.	Identify the network configuration file for the eastbound interface ens256. For example, it may be named 10-cloud-init-ens256.network.
	3.	Open the configuration file with administrative privileges. Update the [Route] section by adding the static routes using this template. Note Replace all placeholders with the actual IP addresses and gateway information. `[Match] Name=ens256 [Network] DHCP=no DNS=<dns-server-ip> [Address] Address=<cluster1-eastbound-ip>/<subnet-mask> [Route] Destination=<eastbound-subnet-of-cluster2>/<subnet-mask> Gateway=<gateway-ip> [Route] Destination=<eastbound-subnet-of-cluster3>/<subnet-mask> Gateway=<gateway-ip>`
	4.	Save the file. Exit the editor. Example: `# Example [Match] Name=ens256 [Network] DHCP=no DNS=10.10.128.236 [Address] Address=172.10.10.11/24 [Route] Destination=172.10.20.0/24 Gateway=172.30.10.2 [Route] Destination=172.10.30.0/24 Gateway=172.30.10.2` Note Verify that the `Name` in the `[Match]` section mateches the correct network interface. Verify that the DNS and gateway IPs are correctly assigned for your network.
	5.	Use the ping command to verify connectivity between the nodes.
	6.	Restart the `systemd-networkd` service to apply the changes. Example: `sudo systemctl restart systemd-networkd`
	7.	Verify that the routes are created. `ip route`
	8.	If you change the eastbound IP address or eastbound network configuration, run the sudo sedo system set-eastbound command on each node to update the eastbound interface before you join the clusters into a supercluster. Example: `sudo sedo system set-eastbound ens256`

The eastbound routes are configured. Connectivity has been verified.

Join clusters into a supercluster

Join three clusters into a single supercluster.

Before you begin

Complete the Border Gateway Protocol (BGP) and eastbound routing configuration before you join clusters.
Ensure you can run commands on each cluster node.

Follow these steps to join clusters into a supercluster.

Procedure

	1.	Use the sedo supercluster status command on each node to retrieve the cluster ID. Example: sedo supercluster status # Sample output ┌────────────────────────────────────────────────────────────┐ │ Supercluster Status │ ├──────────────┬─────────────────────────────────────────────┤ │ Cluster ID │ vk0uFBSwM1vX4_mC1BAabDxAKXYUTv1KH5dcCDawZw4 │ │ Cluster Name │ cluster1 │ │ Cluster Role │ worker │ │ Peers │ <No Peers> │ │ Initialized │ No │ └──────────────┴─────────────────────────────────────────────┘ Note You need the cluster ID for each node for the next steps.
	2.	Connect cluster1 to cluster2. Initiate the connection on cluster1. Example: `# Sample output sudo sedo supercluster wait-for -b 172.20.2.89:10443 uUD21AaV4cQ8CzZQf0E0YrGmALi0vHASpZI07YzcsQ Listening for join requests on 172.20.2.89:10443... Please run the following on peer node: $ sudo /usr/bin/sedo supercluster join Lh9Gv3FwSUsx7Gu_7EJoIMe4r5YE6ApyHqOEt83fko https://172.20.2.89:10443/join/g4jKVulJo74ptz82lMvngQ` Run the join command generated by cluster1 on cluster2. Example: `sudo /usr/bin/sedo supercluster join Lh9Gv3FwSUsx7Gu_7EJoIMe4r5YE6ApyHqOEt83fko https://172.20.2.89:10443/join/g4jKVulJo74ptz82lMvngQ`
	3.	Connect cluster1 to cluster3. Initiate the connection on cluster1. `sudo sedo supercluster wait-for -b <cluster1_node_eastbound_ip>:10443 <cluster3_node_cluster_id>` Run the join command generated by cluster1 on cluster3.
	4.	Connect cluster2 to cluster3. Initiate the connection on cluster2. `sudo sedo supercluster wait-for -b <cluster2_node_eastbound_ip>:10443 <cluster3_node_cluster_id>` Run the join command generated by cluster2 on cluster3.

Clusters are joined and ready for connectivity validation.

Verify cluster connectivity and start the supercluster

Verify connectivity, start the supercluster, and confirm its operational status.

Before you begin

Ensure that clusters are joined and reachable.

Follow these steps to verify connectivity and start the supercluster.

Procedure

Use the sedo supercluster connectivity command to verify connectivity between clusters.

Note

Wait until all connections are successful. Clusters typically establish connectivity within 5 minutes.

Example:

sudo sedo supercluster connectivity

┌────────────────────────────────────────────────────────────────┐
│ Supercluster Connectivity          │
├───────────────────────┬───────────────────────┬──────┬─────────┤
│ FROM                  │ TO                    │ RTT  │ RESULT  │
├───────────────────────┼───────────────────────┼──────┼─────────┤
│ cluster2/controller-0 │ cluster1/controller-0 │ 14ms │ Success │
│ cluster2/controller-0 │ cluster3/controller-0 │ 15ms │ Success │
│ cluster1/controller-0 │ cluster3/controller-0 │ 12ms │ Success │
│ cluster1/controller-0 │ cluster2/controller-0 │ 12ms │ Success │
│ cluster3/controller-0 │ cluster2/controller-0 │ 13ms │ Success │
│ cluster3/controller-0 │ cluster1/controller-0 │ 13ms │ Success │
└───────────────────────┴───────────────────────┴──────┴─────────┘

Use the sedo supercluster start command to start the supercluster.

Note

The node where you execute this command becomes the active node. The other worker node becomes the standby node.

Example:

sudo sedo supercluster start

Checking Supercluster connectivity...Passed
Initiating Supercluster...Done

Use the sedo supercluster status to verify the supercluster status.

Example:

This sample output shows the result of the status command on the standby node. When DB replication is streaming and DB Lag is 0 bytes, the geo-redundant deployment is running.

sedo supercluster status
┌──────────────────────────────────────────────────────────────────────────────────────┐
│ Supercluster Status          │
├──────────────────┬───────────────────────────────────────────────────────────────────┤
│ Cluster ID       │ QgQV2uXgP1udqshlIssyTwf3LZzEyRh6I3z5MH8almA                       │
│ Cluster Name     │ cluster1  │
│ Cluster Role     │ worker    │
│ Peers            │ cluster2 (worker, jaWeN9BdXUUTxvofwt6Hukt6OQXIUaqo4NxN6zHYDc)     │
│                  │ cluster3 (arbitrator, SUCrwqQjXToG5GKBwckcg_CtzgHstQigaEM1X0988E) │
│ Mode             │ Running   │
│ Current Active   │ cluster1  │
│ Previous Active  │           │
│ Standby Clusters │ cluster2  │
│ Last Switchover  │           │
│ Last Failover    │           │
│ Last Seen        │ controller-0.cluster2: 2025-03-19 11:16:57.051 +0000 UTC          │
│                  │ controller-0.cluster3: 2025-03-19 11:16:57.047 +0000 UTC          │
│                  │ controller-0.cluster1: 2025-03-19 11:16:57.051 +0000 UTC          │
│ Last Peer Error  │           │
│ Server Error     │           │
│ DB Replication   │ streaming │
│ DB Lag           │ 0 bytes   │
└──────────────────┴───────────────────────────────────────────────────────────────────┘

The supercluster is running and connectivity is confirmed.

Configure BGP for supercluster routing

Configure BGP so the supercluster can advertise the virtual IP route.

Before you begin

Obtain the router IP address, autonomous system number, and password from your network administrator before you begin.
Confirm that each node can reach the northbound network.

Follow these steps to configure BGP for supercluster routing.

Procedure

	1.	Initialize BGP on cluster 1 and 2 nodes. `sedo ha bgp init <current_node_name> <current_node_northbound_ip> <current_node_as> --nexthop <current_node_northbound_ip>`
	2.	Add a BGP router to each node. sedo ha bgp router add <current_node_name> <bgp_router_ip> <bgp_router_as> <bgp_password> --ttl-min 255 Note Collect the BGP router IP address, router autonomous system number, and BGP password from your network administrator. The BGP password must match the neighbor configuration on the router. Example: `sedo ha bgp router add conc2512-2 192.168.125.1 65534 password --ttl-min 255`

BGP is initialized, and the router is added to each node.

Validate supercluster service status and installed version

Validate the service status and confirm the installed version.

Run these checks after the supercluster has started.

Before you begin

Follow these steps to validate services and version.

Procedure

Check the status of all pods.

sedo system status
┌───────────────────────────────────────────────────────────────────────────────────┐
│ System Status (Fri, 20 Sep 2024 08:21:27 UTC)         │
├────────┬──────────────────────────────┬───────┬─────────┬──────────┬──────────────┤
│ OWNER  │ NAME                         │ NODE  │ STATUS  │ RESTARTS │ STARTED      │
├────────┼──────────────────────────────┼───────┼─────────┼──────────┼──────────────┤
│ onc    │ monitoring                   │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-alarm-service            │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-apps-ui-service          │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-circuit-service          │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-collector-service        │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-config-service           │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-devicemanager-service    │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-inventory-service        │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-nbi-service              │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-netconfcollector-service │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-osapi-gw-service         │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-pce-service              │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-pm-service               │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-pmcollector-service      │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-topology-service         │ node1 │ Running │ 0        │ 3 hours ago  │
│ onc    │ onc-torch-service            │ node1 │ Running │ 0        │ 3 hours ago  │
│ system │ authenticator                │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ controller                   │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ flannel                      │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ ingress-proxy                │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ kafka                        │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ loki                         │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ metrics                      │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ minio                        │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ postgres                     │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ promtail-cltmk               │ node1 │ Running │ 0        │ 12 hours ago │
│ system │ vip-add                      │ node1 │ Running │ 0        │ 12 hours ago │
└────────┴──────────────────────────────┴───────┴─────────┴──────────┴──────────────┘

Note

The pod statuses appear in separate terminal sessions for each node.

The status of all services must be Running.

Check the current version.

sedo version
┌──────────────────────────────────────────────────────────────────────────────────────────┐
│Installer: 24.3.2                │
├──────────────┬──────────────────────────────────────────────────────────┬────────────────┤
│ NODE NAME    │ OS VERSION                   │ KERNEL VERSION │
├──────────────┼──────────────────────────────────────────────────────────┼────────────────┤
│ node1-c1-sa1 │ NxFOS 3.2-555 (93358ad257a6cf1e3da439144e3d2e8343b53008) │ 6.1.0-31-amd64 │
└──────────────┴──────────────────────────────────────────────────────────┴────────────────┘
┌────────────────────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────┬──────────────┐
│ IMAGE NAME         │ VERSION                        │ NODES        │
├────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┼──────────────┤
...
└────────────────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────┴──────────────┘

The service status and version information are confirmed.

Need help?

Open a support case

(Requires a Cisco Service Contract)

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.