Troubleshooting Information

This section describes troubleshooting information for this feature.

Problem

In the scenario of multiple AMF deployments, the secondary pods can’t be brought up, when the Master node has used the default pod limit. The secondary pods fail to get started, due to one of the following reasons:

  • Pods are in the pending state.

  • The nodes don't match the pod and node affinity or the antiaffinity rules.

Resolution

In circumstances when all the pods are used, you can increase the number of pods in the Kubernetes cluster.

To configure the Kubernetes maximum pod count, use the following steps:

On an Existing AMF Deployment:

  1. In the SMI Deployer CLI, use the following configuration:

    clusters node-defaults k8s max-pods maximum_pods 

    NOTES:

    1. max-pods maximum_pods —Specify the maximum number of pods per node. Default is 256. Must be an integer in the range of 10-2000.

  2. Assign labels to the nodes within the cluster and sync the changes using the following command:

    clusters nodes actions sync run 
  3. Delete the Istio directory from the Master node using the following command:

    rm -rf /var/lib/smi/istio/ 
  4. Synchronize the clusters to reflect the configuration using the following commands:

    clusters cluster_name actions sync run reset-k8s-nodes true debug true 
    clusters cluster_name actions sync run sync-phase opscenter debug true 
    Important

    The synchronization procedure erases the AMF's day-1 or N configuration.

On a New AMF Deployment:

  1. In the SMI Deployer CLI, use the following configuration:

    clusters node-defaults k8s max-pods maximum_pods 

    NOTES:

    1. max-pods maximum_pods —Specify the maximum number of pods per node. Default is 256. Must be an integer in the range of 10-2000.

  2. Synchronize the clusters using the following commands:

    clusters cluster_name actions sync run force-vm-redeploy true debug true 
        

For more information on the SMI configuration, see Ultra Cloud Core Subscriber Microservices Infrastructure - Operations Guide.