The Master VM is the main VM for the whole system and requires a cloud-init configuration file.

Some of the configuration details in a cloud-init configuration are:

• Users: non-root administrative user details for the ‘cps’ user that is created and assigned to the ‘docker’ group. The user also has an SSH key injected. The SSH key is user-supplied and should be unique for every installation.

• Password: password for the ‘cps’ user. The password may be removed for all production installations.

• Default path for cps.pem is /etc/puppet/modules/qps/templates/certs

• Configuration details in JSON format (swarm.json) for the system to start successfully.

• Informational file: details of the product and VM within that product they are working with when they log in via SSH.

For an example of a cloud-init configuration file for a Master VM, see Master Cloud Init Example.

nova boot --config-drive true --user-data=node-master-0.cfg \
  --flavor=cps.medium --image=docker-host-1.13.1-2 \
  --nic net-id="$Internal,v4-fixed-ip=172.16.2.11" \
  --nic net-id="$Management,v4-fixed-ip=172.18.11.121" \
  --block-device id=${iso},source=volume,dest=volume,device=/dev/vdb \
  --block-device id=${volume},source=volume,dest=volume,device=/dev/vdc \
  --availability-zone nova \
  --security-groups esc-security-group \
  docker-pcrf-master-0

The launching of a Docker engine VM (aka: Worker or Engine) is identical, though with less information injected into the ‘swarm.json’ file. Worker VMs might or might not have fixed IP addresses, and there is an assumption that if there isn’t a need for a fixed address that the interface will get it’s address via DHCP. This is primarily true for interfaces on the “internal” network, but there is no prohibition against doing it on other networks.

An example of the cloud-init file for an example Worker VM can be seen in Worker Cloud Init Example. The values included in the worker ‘swarm.json’ file are:

• role

• identifier

• master

• network

In the simplest case a VM is launched exclusively on the “internal” network, assumed to DHCP, and has no Cinder volumes. The Nova boot command for that situation looks like:

nova boot --config-drive true --user-data=node-pcrf-0.cfg \
  --flavor=cps.medium --image=docker-host-1.13.1-2 \
  --nic net-id="$Internal" \
  --availability-zone nova \

This script is specific to only the Master VM, and performs tasks that put the Master in play. The tasks will be outlined below as a sequence of bullet points.

• Condition the target system

  • If there is a /dev/vdc device first determine if it is formatted, and if not format the volume to be an EXT4 filesystem.

  • Remove all data from the mounted file system (/mnt/install) because the only time this runs on a Master VM is when it is booting, and that always calls for a fresh installation.

  • Copy all ISO data to the writable filesystem (/mnt/install).

  • Create the ‘cps-app’ system user.

• Load the configuration from the ‘/root/swarm.json’ file.

  • During loading combine the values from ‘/mnt/iso/release.json’ file

• Write the updated configuration to ‘/mnt/install/swarm.json’

• Stop the timesync daemon.

• Configure the Docker Engine daemon appropriately for operation on the Master.

• Restart the Docker service.

• Set the VM’s hostname to match the VM’s system-internal identifier.

• Install a locally-provided image from the saved image file ‘cps-registry-latest.tgz’.

• Launch the registry container.

• Launch the registry upgrade container.

• Launch Weave.

• Launch the Engine Proxy container.

• Launch the Personality service.

• Launch the product-specific initialization container identified by the ‘init’ value in ‘swarm.json’.

At this point control for the behavior of the system passes out of the startup processes and into the hands of the initialization container. For now, this ultimately leads to the invocation of the product specific Orchestrator. It is the Orchestrator that is responsible for launching all the remaining containers which make up the deployed application.

This script is run for each of the Worker VMs that are launched. Operations on the Worker are very different from the Master, and that will be reflected in the steps detailed below.

• Load configuration information from the ‘/root/swarm.json’ file for the VM.

• Read the release information from the Master VM, and incorporate it into an updated ‘/root/swrm.json’ file.

• Determine the interface and IP for the Internal network

• If an ‘identifier’ value is not set in the provided ‘/root/swarm.json’ file, create a unique engine identifier value.

• Create the ‘cps-app’ system user.

• If the role is ‘control’ then :

  • If there is a /dev/vdc device first determine if it is formatted, and if not format the volume to be an EXT4 filesystem.

  • Mount /dev/vdc on /data.

  • If ‘reinitialize_data’ is set, clear out the /data filesystem.

• If ‘zing’ is set and the role is ‘policy’, enable Zing operation.

• Set the hostname to match the ‘identifier’.

• Stop the timesync daemon.

• Configure the Docker engine to include information about this Worker instance (it’s identifier and it’s network interface) so that the Orchestrator can know how to use the Worker in the system.

• Launch Weave

At this point the Worker VM is ready to accept containers. This will conclude the low-level startup process. From this point forward additional startup is handled by the Orchestrator and can vary from product to product.