Controller Gateway is the component responsible for all the interaction between a Crosswork Data Gateway instance and its Controller. It manages the session creation with Controller and makes sure all the payloads and responses are signed and verified for integrity. Components such as Image Manager, Vitals Monitor, Route manager and all collectors interact via Controller Gateway with Controller to exchange the details those components need.

Note	When the Controller Gateway stops, any alerts are not updated in cdg-alerts.log. However, when it starts, it sends an alert that it has started. This is because all the alerts go through the Controller Gateway and if it is down, Controller won't receive the alerts.

The Image Manager comes into play upon Crosswork Data Gateway VM boot up. It establishes a session with the Controller using information from the Enrollment Package generated during VM bootstrap. It then downloads the functional images from the repository as instructed by the Controller and brings up the containers.

It has the following responsibilities:

• Periodically pull boot-config file from the Controller via Controller Gateway.

• Based on the boot-config and local images metadata cache, determine if the functional images and docker-compose file need to be downloaded.

• Send appropriate alerts to Controller, if there are issues while processing the boot-config.

• Stop and remove any containers that are no longer called for in the latest boot-config.

• Cleanup the local images metadata cache to keep it synchronized with the latest boot-config received from Controller.

The Vitals Monitor monitors the health and vitals of the Crosswork Data Gateway VM. It samples the CPU, memory, disk usage, docker containers details, etc. and aggregates this information in a file on the host filesystem.

In Cloud solution, the host is mounted on downloaded functional image container app. The app may then collect relevant metrics and send to Controller for further processing.

However, in a customer-hosted solution, the vitals are periodically archived in a local persistent volume that is mounted within each container. This information is accessible from both VM and the Docker container. This data can be exported to the Controller periodically.

For more information, see Monitor Crosswork Data Gateway Health.

Route Manager periodically syncs the configured static routes from the Controller. It then directs the traffic to devices on different Southbound destinations based on the routes.

Route manager adds/deletes the static routes by comparing the existing routes configured on the VM with the routes configuration pulled from the Controller.

Appropriate alerts are sent to the Controller if there is any failure in processing routes configuration.

The Docker Engine manages all containers.

Once the session with the Controller is established, the Image Manager requests the Controller for a boot-config response. The Controller prepares a boot-config response containing a list of functional images, docker-compose file, their SHA digests, and the location from where the image tar files can be downloaded.

The following figure shows a sample boot-config response:

{
  "pollingPeriodSecs": 30,
  "files": [
    {
      "fileType": "CONTAINER_FILE",
      "url": "<url_of_container_file>",
      "hash": "<hash>",
      "container": {
        "name": "mdt-collector",
        "imageRepoName": "<image_repo_name>",
        "imageRepoTag": "<image_repo_tag>"
      }
    },
    {
      "fileType": "CONTAINER_FILE",
      "url": "<url_of_container_file>",
      "hash": "<hash>",
      "container": {
        "name": "cli-collector",
        "imageRepoName": "<image_repo_name>",
        "imageRepoTag": "<image_repo_tag>"
      }
    },
    {
      "fileType": "CONTAINER_FILE",
      "url": "<url_of_container_file>",
      "hash": "<hash>",
      "container": {
        "name": "snmp-collector",
        "imageRepoName": "<image_repo_name>",
        "imageRepoTag": "<image_repo_tag>"
      }
    },
    {
      "fileType": "COMPOSE_FILE",
      "url": "url_of_compose_file",
      "hash": "<hash_of_compose_file>"
    }
  ]
}

The following table explains the attributes of the boot-config response:

Crosswork Data Gateway's Image Manager constantly polls the Controller for the latest version of the boot-config. It looks for changes in following three attributes to determine if an image should be upgraded:

• URL of the image location

• Hash key of the image

• Image repository tag

The Controller indicates the polling interval in the boot-config. After the first poll, the Image Manager adjusts its polling interval accordingly.

Whenever Image Manager polls and finds changes in the functional images version, it downloads the new set of images, stop the old functional images container, and brings up the new functional images containers.

Note

boot-config mentions the downloadable functional images. The service containers specified in the docker-compose file should be from those functional images i.e. they should match. For example, if three functional images are specified in boot-config.json, then the service containers specified in the docker-compose file should match those three functional images.

The docker-compose YAML file is required to bring up the Docker containers. It tells Docker which containers to launch when the VM is powered on.

The downloadable docker-compose.yml is part of boot-config.json provided in the Controller. The Image Manager downloads and starts the container images based on the provided docker-compose.yml.

Crosswork Data Gateway Environment Variables

Crosswork Data Gateway provides the following environment variables that can be set for each service in the docker-compose file:

Environment Variable	Use
$CDG_ID	The Unique UUID of the Crosswork Data Gateway
$DG_CONTROLLER_URL	URL to reach the Controller
$DG_MGMT_IP	Self IP address on management network
$DG_DATA_IP	Self IP address on data network
$HTTP_PROXY	The URL of the management network proxy server
$HTTPS_PROXY
$NO_PROXY	Optional comma-separated list of subnets and domains that will not be sent to the proxy server
$DG_FTP_PROXY
$DG_LOG_VOLUME	Path used to mount logs from containers to the host VM

Also, you can define additional environment variables in the 'environment' section of the docker-compose file as shown in the following example:

environment;
    ENV_ONE: variable1
    ENV_TWO: variable2
    INSTANCE_UNIQUE_ID: ${CDG_ID}
    CONTROLLER_URL: ${DG_CONTROLLER_URL}

Note	Initial memory is allocated to collector containers based on docker-compose configured input.

Volumes

Filesystem paths must also be specified in the Docker Compose file so that they can be mounted onto the Crosswork Data Gateway VM. Crosswork Data Gateway provides the following environment variable to point to the Crosswork Data Gateway VM root:

• $DG_VOLUME

Following are the three supported locations for mounts:

Volume Location	Use
$DG_LOG_VOLUME	Where container logs are mounted on the VM
$DG_LOG_VOLUME/vitals	Where the vitals are mounted on the VM
/proc

Note	Volumes can also be defined using short syntax: <HOST PATH>:<CONTAINER PATH> where <HOST PATH> is one of the volume location from the table above. For example, volumes: - /proc/:/container/path/to/proc - ${DG_LOG_VOLUME}:/container/path/to/logs - ${DG_LOG_VOLUME}/vitals:/container/path/to/vitals

version: "3.4"
services:

  management-service:
    container_name: app-manager
    image: controller:latest
    environment:
      ENV_ONE: variable1
      ENV_TWO: variable2
      INSTANCE_UNIQUE_ID: ${CDG_ID}
      CONTROLLER_URL: ${DG_CONTROLLER_URL}
    ports:
    - "6060:6060/udp"
    - "6061"
    volumes:
    - /proc/:/container/path/to/proc
    - ${DG_LOG_VOLUME}:/container/path/to/logs
    - ${DG_LOG_VOLUME}/vitals:/container/path/to/vitals

  database-service:
    container_name: mysql-db
    image: MySQL:latest
    ports:
    - "5432"
    environment:
      ENV_ONE: variable1
      SAMPLE_ENV: variable2
      GATEWAY_UUID: ${CDG_ID}
    volumes:
    - /proc/:/container/path/to/proc
    - ${DG_LOG_VOLUME}:/container/path/to/logs

  web-service:
    container_name: http-server
    image: http-server:latest
    environment:
      ENV_ONE: variable1
      ENV_TWO: variable2
    ports:
    - "8443"
    - "8080:8080/tcp"
    volumes:
    - /proc/:/container/path/to/proc
    - ${DG_LOG_VOLUME}:/container/path/to/logs
    healthcheck:
      est: << Logic to check the health of the service, runa as a CMD. >>
      interval: 30s
      timeout: 10s
      retries: 1
      start_period: 10s

Every Crosswork Data Gateway instance needs to be identified by the Controller by means of an immutable identifier. For this purpose, the VM bootstrap procedure involves generating a Crosswork Data Gateway enrollment package.

The enrollment package is a JSON document created from the information obtained through the OVF template that is populated by the user during installation.

Upon successful generation, the enrollment package is stored as a file in a local filesystem. You can retrieve the enrollment package via Crosswork Data Gateway Main Menu. See Export Enrollment Package.

It includes the all necessary information about Crosswork Data Gateway, such as Certificate, UUID of the Crosswork Data Gateway instance, and metadata like Crosswork Data Gateway instance name, Creation Time, version info, etc.

A sample Crosswork Data Gateway enrollment package JSON is shown below:

{
   "dg_vm_id":"4A4ACD31-4F9B-420E-9059-D5671A957A4B",
   "dg_ca_root":"-----BEGIN CERTIFICATE-----\nMIIEMjCCAxqgAwIBAgIBATANBgkqhkiG9w0BAQUFADB7MQswCQYDVQQGEwJHQjEb\nMBkGA1UECA
    wSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHDAdTYWxmb3JkMRow\nGAYDVQQKDBFDb21vZG8gQ0EgTGltaXRlZDEhMB8GA1UEAwwYQUFBIENlcnRpZm
    lj\nYXRlIFNlcnZpY2VzMB4XDTA0MDEwMTAwMDAwMFoXDTI4MTIzMTIzNTk1OVowezEL\nMAkGA1UEBhMCR0IxGzAZBgNVBAgMEkdyZWF0ZXIgTWFuY2hl
    c3RlcjEQMA4GA1UE\nBwwHU2FsZm9yZDEaMBgGA1UECgwRQ29tb2RvIENBIExpbWl0ZWQxITAfBgNVBAMM\nGEFBQSBDZXJ0aWZpY2F0ZSBTZXJ2aWNlcz
    CCASIwDQYJKoZIhvcNAQEBBQADggEP\nADCCAQoCggEBAL5AnfRu4ep2hxxNRUSOvkbIgwadwSr+GB+O5AL686tdUIoWMQua\nBtDFcCLNSS1UY8y2bmhG
    C1Pqy0wkwLxyTurxFa70VJoSCsN6sjNg4tqJVfMiWPPe\n3M/vg4aijJRPn2jymJBGhCfHdr/jzDUsi14HZGWCwEiwqJH5YZ92IFCokcdmtet4\nYgNW8I
    oaE+oxox6gmf049vYnMlhvB/VruPsUK6+3qszWY19zjNoFmag4qMsXeDZR\nrOme9Hg6jc8P2ULimAyrL58OAd7vn5lJ8S3frHRNG5i1R8XlKdH5kBjHYp
    y+g8cm\nez6KJcfA3Z3mNWgQIJ2P2N7Sw4ScDV7oL8kCAwEAAaOBwDCBvTAdBgNVHQ4EFgQU\noBEKIz6W8Qfs4q8p74Klf9AwpLQwDgYDVR0PAQH/BAQD
    AgEGMA8GA1UdEwEB/wQF\nMAMBAf8wewYDVR0fBHQwcjA4oDagNIYyaHR0cDovL2NybC5jb21vZG9jYS5jb20v\nQUFBQ2VydGlmaWNhdGVTZXJ2aWNlcy
    5jcmwwNqA0oDKGMGh0dHA6Ly9jcmwuY29t\nb2RvLm5ldC9BQUFDZXJ0aWZpY2F0ZVNlcnZpY2VzLmNybDANBgkqhkiG9w0BAQUF\nAAOCAQEACFb8AvCb
    6P+k+tZ7xkSAzk/ExfYAWMymtrwUSWgEdujm7l3sAg9g1o1Q\nGE8mTgHj5rCl7r+8dFRBv/38ErjHT1r0iWAFf2C3BUrz9vHCv8S5dIa2LX1rzNLz\nRt
    0vxuBqw8M0Ayx9lt1awg6nCpnBBYurDC/zXDrPbDdVCYfeU0BsWO/8tqtlbgT2\nG9w84FoVxp7Z8VlIMCFlA2zs6SFz7JsDoeA3raAVGI/6ugLOpyypEB
    Ms1OUIJqsi\nl2D4kF501KKaU73yqWjgom7C12yxow+ev+to51byrvLjKzg6CYG1a4XXvi3tPxq3\nsmPi9WIsgtRqAEFQ8TmDn5XpNpaY
    bg==\n-----END CERTIFICATE-----\n",
   "dg_mac_address":"00:0a:95:9d:68:16",
   "dg_ip_address":"172.23.85.145",
   "dg_user_friendly_name":"CDG1.customer.com”,
  “dg_deploy_time”: 1556910613,
  “dg_version”: “1.0”,
  “dg_resource_profile”: {
	“profile_type”: “Large”
	“profile”: {
	 	“cpu”: 8,
  		“memory”: “50”,
   		“nics”: 3
	  }
  }
}

The Route Manager pulls the static routes configuration i.e., static-routes.json from the Controller. The static-routes config response describes the following details of each route:

• IP address of the destination network

• Next hop address

• Management and data network interface to be used

Route manager adds/deletes the static routes by comparing the existing routes configured on the VM with the contents of static-routes config pulled from the Controller.

Shown below is a sample static-routes config response:

{
"routes": [
{
"destination_network": "10.20.0.0/16",
"next_hop_address": "192.168.20.145",
"nic_name": "NORTH"
},
{
"destination_network": "20.1.0.0/16",
"next_hop_address": "192.168.20.147",
"nic_name": "SOUTH"
},
{
"destination_network": "3.3.3.0/24",
"next_hop_address": "172.23.93.23",
"nic_name": "MGMT"
},
{
"destination_network": "2001:4:4::/64",
"next_hop_address": "2001:420:10e:2015::1",
"nic_name": "MGMT"
}
]
}