Cisco Managed Services Accelerator 4.1.0 Installation and Upgrade Documentation
Bias-Free Language
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
To ensure that your MSX installation is trouble free, you must plan your deployment and perform the pre-installation tasks
listed in this chapter.
Preparing OpenStack
OpenStack must be installed on the host where you will be installing MSX. The OpenStack RC file (typically named openrc.sh)
is used by command-line clients to set environmental variables and to provide credentials that are required for using OpenStack
services. This section describes the steps for downloading the OpenStack RC file and preparing it for use. The steps given
here are to be performed on the host in which you will be installing MSX.
Editing the OpenStack RC File
Procedure
Step 1
To download the OpenStack RC file, navigate to Project > API Access in the OpenStack GUI.
Step 2
Click Download Openstack RC File, and select the OpenStack RC File (Identity API v3).
Step 3
Edit the OpenStack RC file as specified:
Comment out the lines:
# echo “Please enter your OpenStack Password:”
# read -sr OS_PASSWORD_INPUT
Source the OpenStack RC file. You will be prompted for the OpenStack password. If you would like to avoid being prompted each
time you source the file, then update the password (shown in bold below) in the OpenStack RC file.
source <path_to_openrc_file>
Sample OpenStack RC File
export OS_AUTH_URL=http://1.1.1.1:5000/v3
export OS_PROJECT_ID=c47f690aab6c4822ad72b8bdc9aa72a2
export OS_PROJECT_NAME="admin"
export OS_USER_DOMAIN_NAME="Default"
if [ -z "$OS_USER_DOMAIN_NAME" ]; then unset OS_USER_DOMAIN_NAME; fi
export OS_PROJECT_DOMAIN_ID="default"
if [ -z "$OS_PROJECT_DOMAIN_ID" ]; then unset OS_PROJECT_DOMAIN_ID; fi
# unset v2.0 items in case set
unset OS_TENANT_ID
unset OS_TENANT_NAME
# In addition to the owning entity (tenant), OpenStack stores the entity
# performing the action as the **user**.
export OS_USERNAME="admin"
# With Keystone you pass the keystone password.
echo "Please enter your OpenStack Password for project $OS_PROJECT_NAME as user $OS_USERNAME: "
read -sr OS_PASSWORD_INPUT
export OS_PASSWORD=$OS_PASSWORD_INPUT
# If your configuration has multiple regions, we set that information here.
# OS_REGION_NAME is optional and only valid in certain environments.
export OS_REGION_NAME="RegionOne"
# Don't leave a blank variable, unset it if it was empty
if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3
Setting OpenStack Quotas for MSX Installation
Ensure that you have allocated OpenStack quotas to support this MSX installation. For instructions on setting quotas, see
the OpenStack documentation.
Table 1. OpenStack Quotas
Quota
Value Needed
Instances
50
Floating IP
4
Security Groups
30
Security Group Rules
100
Volumes
50
Validating the External Network
MSX Platform requires floating IP addresses that come from an external network. You can validate the name of the external
network used on your OpenStack instance by accessing the OpenStack dashboard. Points to note:
Ensure that you name the external network as ‘external’ in the OpenStack dashboard. To see the names of your networks in the
OpenStack user interface, navigate to Project > Network > Networks.
If you are using a different network name in the OpenStack dashboard, ensure that you update the os_fip_pool and os_ext_net
parameters in the MSX configuration file (/msx-4.1.0/ansible/group_vars/all/main.yml) to include the network name. For more
information, see Configuring the main.yml Bootstrap File.
Downloading CentOS Cloud Image
To download the CentOS Cloud image on the OpenStack host where you will be installing MSX:
Procedure
Step 1
Download the CentOS-7-x86_64-GenericCloud-1804_02.qcow2 cloud image from the CentOS servers to the OpenStack host where you will be installing MSX.
Note
You should use the CentOS version specified above. Using any other version can result in MSX installation failures.
Step 2
Ensure that the downloaded copy of CentOS is not corrupted by verifying the SHA256 file checksum using the command:
certutil -hashfile <CentOS_image_filename.qcow2> SHA256
glance image-create --name CentOS-7-x86_64-GenericCloud-1804_02.qcow2 --disk-format=qcow2
--container-format=bare --visibility public --file CentOS-7-x86_64-GenericCloud-1804_02.qcow2
--progress
nova flavor-create --is-public True m1.large auto 8192 80 4
nova flavor-create --is-public True m1.medium auto 4096 40 2
nova flavor-create --is-public True m1.xlarge auto 16384 80 8
Compare the resulting hash to see that it matches the value shown in sha256sum.txt , which can be found at the same CentOS
download location as the OS image. If they match, then the file is not corrupt.
Step 3
Add the downloaded image to OpenStack using Glance. Use the command:
The name of the image must match os-image name under OpenStack details in the MSX configuration file (main.yml).
Step 4
Create the following flavors required by the MSX installer, if they do not already exist within OpenStack:
nova flavor-create --is-public True m1.large auto 8192 80 4
nova flavor-create --is-public True m1.medium auto 4096 40 2
nova flavor-create --is-public True m1.xlarge auto 16384 80 8
Preparing the Container
The MSX installer is packaged in a Docker image file that contains all the files and images required for a complete MSX installation.
The Docker image can be installed from a host or a VM that has access to the OpenStack or AWS deployment. You need to install
Docker on this host or VM. If necessary, Docker can be installed by executing the command:
yum install docker
After installing Docker, ensure that the Docker engine and the Docker daemon are up and running. Open a command-line terminal,
and run the following Docker command to verify that Docker is working and that the latest release is installed.
docker version
Note
The host needs to have at least 15 GB of free space to prepare the container for an MSX installation.
[root@CentOS localadmin] # docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> e94ad25e65a9 5 days ago 383 MB
Step 5
Create a shared directory (/MSXconfig) between the local host and the MSX installation container. Create a directory named
/tmp/repo on the host, and untar msx-binaries-4.1.0-xxx.tar.gz into the directory.
Step 6
Copy your existing OpenStack RC file to the folder that will be attached as a mount to the Installer container:
cp /root/openrc MSXconfig/openrc
If you are installing MSX in dual DC mode, download OpenStack RC file from both sites.
If you have issues accessing the shared directory from the container, add ":z" after the "PathOnContainer" in the Docker run
command. This is an example, with /MSXconfig as the local directory.
docker run -v /home/localadmin/MSXconfig:/MSXconfig:z --name msx-4.1.0 -it e94ad25e65a9 /bin/bash
Run the Docker container in a screen session. Running Docker in a screen session protects your installation from connectivity
issues that might cause the Linux shell to terminate. To install the screen utility, use yum install screen. Start your screen
session with this command:
screen -S <Screen_Session_Name>
Step 8
If installing MSX in a dual data
center mode, copy the openrc files to /msx-4.1.0/ansible/vms-backup/infra.
Source the OpenStack RC file from the Docker container:
source /MSXconfig/openrc
Creating a Key Pair
A new key pair must be created to securely connect to Inception and other MSX virtual machines. This process is the same
for both OpenStack and AWS installations. If you already have a key pair, you can use it to connect to the MSX VMs. In this
case, you will need to copy both the public and private keys to /msx-<version>/ansible/keys. For example: /msx-4.1.0/ansible/keys
If you do not have existing keys, then you can create and locate your keys using the given procedure:
You will need to configure the main.yml variables file before proceeding with the installation.
vi /msx-4.1.0/ansible/group_vars/all/main.yml
This section lists the parameters from the main.yml file that must be configured while installing MSX. For a complete list
of parameters with default values, see Sample main.yml file in this section.
OpenStack Parameters to Change in main.yml:
os_image: Same as the downloaded CentOS file (CentOS-7-x86_64-GenericCloud-1804_02.qcow2).
os_fip_pool: Change this setting to ‘external’ if your network in OpenStack is named as ‘external.’
os_ext_net: Change this setting to ‘external’ if your network in OpenStack is named as ‘external.’
DNS and NTP Settings for OpenStack: Change the DNS and NTP default setting to settings that match your environment.
proxy / no_proxy settings:
Proxy/no_proxy settings are not required to deploy MSX. In some cases, however, to enable Google Map or other functionality
that requires Internet access, you would need to set the proxy values.
Provide the proxy settings based on your requirements:
If your deployment requires a proxy server (shown below), then uncomment both #proxy: and #no_proxy: and set their values.
The no_proxy: list must contain the values shown below, but you can also append other servers that do not require proxy functionality.
use_existing_ips: Set use_existing_ips to ‘yes,’ if you want to use the same public IP addresses across the MSX deployment.
Ensure that these external IP addresses are available as Floating IP addresses or Elastic IP addresses in your OpenStack Project
and Virtual Private Cloud, respectively. Also, specify these IP addresses in the external_addresses.yml file.
FQDN Settings / Route Settings: As part of the MSX installation, route53 can be used to register the FQDN of the MSX portal to the ciscovms.com domain.
If you want to use the ciscomsx.com domain, change the settings to:
use_route53: true
vms_domain: Has to be a domain you control in route53, for example, ciscomsx.com.
vms_subdomain: Has to be unique in your domain and must not contain "." , "_”, or the uppercase letters, for example, the
subdomain can be msxservices.
Note
If the use_route53 variable is set to 'true' in FQDN Setting / Route Settings, you need to export the https_proxy with the
value used in the proxy / no_proxy settings and then unset it immediately after creating infra.
export https_proxy=<proxy_server >
If the use_route53 variable is set to false, then you can skip this step.
SMTP Settings for the Email Notifications:
smtp_server
SMTP port number
GOOGLE API KEY Settings:
- google_api_key: <your_Google_API_key>
Provider_Name, Provider_Display_Name, and Provider_Description must be same as OS_TENANT_NAME.
Elastic Search Volume Size
Use this setting to change the elastic search volume size. Default is 100 GB.
License Token for CSR VPN: Only applicable if you are deploying MSX in the dual data center mode. Provide license token from your smart account to
license the CSRs that are used to set up a tunnel across the two data centers.
iPnP SSL HARDENING variable: iPnP in MSX supports TLS version 1.0,1.1, and 1.2 for SSL handshake with devices. You can configure this value during installation.
Depending on the configuration of the devices in your deployment, set this variable to one of the given values:
0: Supports TLS 1.0, 1.1, and 1.2
1: Supports TLS 1.1 and 1.2
2: Supports TLS 1.2
Note
For enhanced security, we recommend that you use TLS 1.1 or higher.
AWS Parameters to Change in main.yml:
Ensure that you have set the _tag parameters to enable easier instance searching and management.
Make sure to choose the region that is closest to you. Remember that some newer regions may not have enough Availability Zones
within those regions. Typically, MSX is deployed across three Availability Zones, but you can deploy using two, if necessary.
Contact Cisco Technical Assistance Center (TAC) for more details.
Sample main.yml File
The main.yml file can be found at this location: /msx-4.1.0/ansible/group_vars/all/main.yml.
General Instructions
Read the included comments for guidance on filling out your deployment parameters.
Update the bolded “ChangeMe” values to conform to the required settings for your deployment, wherever it is applicable.
Parameters starting with os_ are used for OpenStack and will be ignored during an AWS install.
Parameters starting with aws_ are used for AWS and will be ignored during an OpenStack install.
---
# Target cloud. Currently supports one of aws or openstack
# For fields that differ between aws and openstack, openstack will be set as
# default
cloud: openstack
#cloud: aws
#cloud: azure
# If deploying on a micropod the root volumes need to be on a CEPH Volume
#micropod_deployment: True
# If deployer container needs proxy to access internet, you need to set
# https_prox, http_proxy, and no_proxy in shell environment so it can access
# external resource like Route53, etc.
# If control plane needs proxy to access internet, you need to set proxy and
# no_proxy variable here which will be used while running comands on VMs in
# control plane.
# Shell environment has no correlation with variables here, it's possible that
# you deploy from a container with direct internet access while control plane
# needs proxy and vice versa.
# proxy:
# Use this no_proxy below if you have a proxy server setup as an example
# no_proxy: localhost,registry.service.consul,vault.service.consul,consul.service.consul,kubernetes,*.service.consul
# no_proxy:
# delete kubernetes persistent volume when destroy-infra
delete_k8s_pv: true
# Are you installing in a network without internet access?
isolated: true
# For isolated, the following registry and download_url MUST be used instead
# of the above
registry: registry.service.consul:5000
download_url: "{{ cloud_env[cloud]['dl_url'] }}"
# If the registry is protected by authentication, credentials are defined in
# ansible vault
secure_registry: true
# For AWS, use the following registry and download_url
# It's almost always true that AWS needs secure_registry set to yes
#registry: ChangeMe
#download_url: ChangeMe
# Feature flag to enable / disable CIS Benchmark OS hardening. Should be
# enabled for Production
harden_os: true
# SSH Keys. These can be located on a volume that is presented to the
# installation container using the -v parameter
ssh_public_key_file: /msx-{{ RELEASE }}/ansible/keys/id_rsa.pub
ssh_private_key_file: /msx-{{ RELEASE }}/ansible/keys/id_rsa
keypair_name: vmskeypair
ssh_user: centos
centos_user: cloud-user
# In order to use the same public IPs across installs, set use_existing_ips
# to yes, pace external IP addresses to be used in ./external_addresses.yml
# Ensure the external IP addreses you specify are FIPs/EIPs which are
# available in your project/VPC.
#
use_existing_ips: no
# Openstack configuration details
os_image: CentOS-7-x86_64-GenericCloud-1804_02.qcow2
centos_image: CentOS_Image_2.3
os_fip_pool: external
os_ext_net: external
os_dmz_pool: external
os_dmz_net: external
avail_zone: nova
# AWS configuration details
# control plane CIDR should avoid these subnets as CSR does not work with them:
# - 10.2.x.x/16
# - 10.3.x.x/16
# - 10.0.3.0/24
# - 10.0.2.0/24
# - 10.0.1.0/24
# - 10.1.3.0/24
# - 10.1.2.0/24
# we create /24 subnets by default to reduce manual changes during deployment,
# you need to hardcode them if this does not fit target environment.
aws_owner_tag: ChangeMe
aws_environment_tag: ChangeMe
aws_team_tag: ChangeMe
aws_region: us-east-2
aws_bucket_name: "{{ vms_subdomain }}-msx-bucket{% if dual_dc %}-{{ dc }}{% endif %}.{{ vms_domain }}"
aws_efs_name: "{{ vms_subdomain }}-msx-efs{% if dual_dc %}-{{ dc }}{% endif %}.{{ vms_domain }}"
aws_cidr: "{{ dc_env[dc]['aws']['aws_cidr'] }}"
avail_subnets:
# Make sure AZ for private/public subnets match each other
private:
# Private subnets, index 0-2
- cidr: "{{ aws_cidr|ipaddr('net')|ipaddr(256*1)|ipaddr('address') }}/24"
az: "{{aws_region}}a"
resource_tags: {"Name": "VMS_CP_{{ vms_subdomain }}-private-0"}
- cidr: "{{ aws_cidr|ipaddr('net')|ipaddr(256*2)|ipaddr('address') }}/24"
az: "{{aws_region}}b"
resource_tags: {"Name": "VMS_CP_{{ vms_subdomain }}-private-1"}
- cidr: "{{ aws_cidr|ipaddr('net')|ipaddr(256*3)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "VMS_CP_{{ vms_subdomain }}-private-2"}
public:
# Public subnets, index 3-5
- cidr: "{{ aws_cidr|ipaddr('net')|ipaddr(256*4)|ipaddr('address') }}/24"
az: "{{aws_region}}a"
resource_tags: {"Name": "VMS_CP_{{ vms_subdomain }}-public-0"}
- cidr: "{{ aws_cidr|ipaddr('net')|ipaddr(256*5)|ipaddr('address') }}/24"
az: "{{aws_region}}b"
resource_tags: {"Name": "VMS_CP_{{ vms_subdomain }}-public-1"}
- cidr: "{{ aws_cidr|ipaddr('net')|ipaddr(256*6)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "VMS_CP_{{ vms_subdomain }}-public-2"}
# demo mode use the first of private/public subnets, production mode uses all
aws_vpc_subnets: |
{{ avail_subnets['private'][0:deployment_mode_env[deployment_mode]['aws_subnets']]
+ avail_subnets['public'][0:deployment_mode_env[deployment_mode]['aws_subnets']] }}
aws_dp_cidr: "{{ dc_env[dc]['aws']['aws_dp_cidr'] }}"
aws_dp_vpc_subnets:
# Public subnets, index 0-14
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*1)|ipaddr('address') }}/24"
az: "{{aws_region}}a"
resource_tags: {"Name": "VMS_DP_{{ vms_subdomain }}-public-0"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*2)|ipaddr('address') }}/24"
az: "{{aws_region}}b"
resource_tags: {"Name": "VMS_DP_{{ vms_subdomain }}-public-1"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*3)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_IWAN_INET_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*4)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_B1_INET_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*5)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_B2_INET_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*6)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_H_INET_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*7)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_B1_MPLS_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*8)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_B2_MPLS_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*9)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_H_MPLS_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*10)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_B1_LAN_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*11)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_B2_LAN_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*12)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_H_LAN_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*13)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_H_PNP_{{ vms_subdomain }}"}
- cidr: "{{ aws_dp_cidr|ipaddr('net')|ipaddr(256*14)|ipaddr('address') }}/24"
az: "{{aws_region}}c"
resource_tags: {"Name": "CPE_H_DMZ_{{ vms_subdomain }}"}
# Check https://cisco.jiveon.com/docs/DOC-1748962 for how to create AMI with
# encrypted root EBS volume
aws_centos_image_name: CentOS Linux 7 x86_64 HVM EBS 1804 VMS Encrypted
aws_csrhub_image_name: cisco-CSR-.16.09.02-BYOL-HVM*
aws_csrhub_username: ec2-user
delete_aws_vol_on_termination: true
cloud_env:
aws:
dl_url: "http://{{ aws_bucket_name }}.s3-website.{{ aws_region }}.amazonaws.com/repo/binaries/"
dns_servers:
- "{{ aws_cidr|ipaddr('net')|ipaddr(2)|ipaddr('address') }}"
ntp_servers:
- 0.amazon.pool.ntp.org
- 1.amazon.pool.ntp.org
- 2.amazon.pool.ntp.org
- 3.amazon.pool.ntp.org
volume_device: /dev/xvdb
flavors:
csr_hub: c4.large
inception: c4.large
nso: m4.large
edge: m4.large
kubernetes_node: m4.2xlarge
kubernetes_master: m4.large
openstack:
# dns_servers and ntp_servers need to be list
dl_url: https://registry.service.consul:9000/repo/binaries/
dns_servers:
- ChangeMe
ntp_servers:
- ChangeMe
volume_device: /dev/vdb
flavors:
csr_hub: small.csr1000v
inception: m1.medium
nso: m1.large
edge: m1.large
kubernetes_node: m1.xlarge
kubernetes_master: m1.large
azure:
region: eastus
vms_network: vms-network
vms_app_subnet: vms-app-subnet
vms_orch_subnet: vms-orch-subnet
dns_servers:
- 168.63.129.16
ntp_servers:
- time1.google.com
- time2.google.com
volume_device: /dev/sdc
# Azure premium disk is slower than standard if size is smaller than 128GB
# ref: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/disk-scalability-targets
managed_disk_type: Standard_LRS
flavors:
nat: Standard_DS1_v2
csr_hub: Standard_DS2_v2
inception: Standard_DS2_v2
nso: Standard_DS2_v2
edge: Standard_DS2_v2
kubernetes_node: Standard_DS4_v2
kubernetes_master: Standard_DS2_v2
centos_image:
offer: CentOS
publisher: OpenLogic
sku: '7.4'
version: latest
csrhub_image:
offer: cisco-csr-1000v
publisher: cisco
sku: "16_7"
version: latest
# so far either aws or aws-us-gov for {{ aws_s3_region }}
aws_s3_region: aws
# DNS Configuration
# VMS Network details
# As part of the VMS installation, route53 can be used to register the FQDN of
# the VMS portal to the ciscovms.com domain
use_route53: true
# Must be a domain you control in route53 if you have selected yes above
vms_domain: ciscovms.com
# Must be unique in your domain and can't contain "." or "_". Must not contain
# uppercase letters.
# Please change me, someone always forgots this! :)
vms_subdomain: vmsservices
mode: FreshInstall
# For living object dashboard - external_assurance_url
# The value when set to a valid URL allows VMS to redirect to that external
# assurance link instead of showing service assurance metrics gather by VMS.
# The metics for Performance tab “View Full Performance” link and Site Map:
# Branch “Detail View” link currently present in VMS will be redirected to the
# URL value provided in this variable.
# If the default value of 'ChangeMe' is not updated during deployment time VMS
# would continue showing the metrics gathered from its own service assurance
# engine and displaying the metrics with the VMS UI without any redirection to
# external URL. Sample: http://www.amazon.com
EXTERNAL_ASSURANCE_URL: "ChangeMe"
NODE_PROXY_ENABLED: false
# livesp config
livesp_reverse_proxy_uplink: "ChangeMe"
livesp_deployed: no
# Google API key need to be modified here from customer side
google_api_key: "AIzaSyCKCGans8q5vrZNtngc2D5vOIrp3pBMdNE"
# Used for SFTP connection from Billing Micro service
sftp_host: "sftp://partnersftp.webex.com"
sftp_port: "22"
sftp_user: "med-vms-iwan"
sftp_rsa_private_key_path: "ChangeMe"
sftp_destination_directory: "/Input/"
sftp_source_directory: "/data/vms/monthlyReports"
sftp_max_retries: "-1"
sftp_wait_time: "300000"
sftp_timeout: "60000"
sftp_monthly_report_filename_prefix: "ChangeMe"
sftp_use_root_user_directory: "true"
sftp_connect_with_password_or_key: "PASSWORD"
external_reference_id: "ChangeMe"
dns_servers: "{{ cloud_env[cloud]['dns_servers'] }}"
ntp_servers: "{{ cloud_env[cloud]['ntp_servers'] }}"
dual_dc: false
dc: active
# Smart lic token for CSR VPN
# Provide lic token from your smart account to license the CSRs used to set up
# tunnel across the two datacenters when in dual dc mode.
csr_vpn_license: ChangeMe
#Platform hardware throughput is set to 100MB by default, ensure that smart license account has licenses to authorize this throughput.
csr_vpn_throughput: 100
# Smart lic token for CSR HUB
# Provide lic token from your smart account to license the CSR HUB
csr_hub_license: ChangeMe
#Platform hardware throughput is set to 100MB by default, ensure that smart license account has licenses to authorize this throughput.
csr_hub_throughput: 100
# iPnP SSL HARDENING variable
# 0 - no hardening, 1- medium (TLSv 1.1 and 1.2), 2- high (TLSv 1.2)
SSL_HARDENED: 1
# Data volume details
node_volume_size: 50
elasticsearch_volume_size: 100
# ES log retention period
#IMPORTANT: Increasing the retention period will require additional space defined in elasticsearch_volume_size (see above)
elasticsearch_log_retention_days: 3
passwd_file: ./group_vars/all/passwords.yml
password_restore_path: ./vms-backup/password_restore_dir/
#SAML configuration variables
#Set to true at install time if enabling saml
#Provide configuration values for IDP below
enable_saml: false
#idp metadata_filepath HAS to be a URL from where the metadata file can be downloaded eg: https://test.ciscomsx.com/metadata
idp0_metadata_filepath: ChangeMe
idp0_entityid: ChangeMe
idp0_externalidname: ChangeMe
idp0_idpname: ChangeMe
idp0_subdomain: ChangeMe
# Used for email notifications
smtp_server: ChangeMe
notification_support_email: ChangeMe
notification_user_name: ChangeMe
notification_port: ChangeMe
notification_tlssl_required: "true"
notification_auth_required: "true"
PROVIDER_NAME: admin
PROVIDER_DISPLAY_NAME: admin
PROVIDER_DESCRIPTION: admin
# The provider email address should be changed per environment setups
PROVIDER_EMAIL: ChangeMe
PROVIDER_LOCALE: en_US
PROVIDER_NOTIFICATION_TYPE: email
Configuring Backup Cron Jobs
MSX includes some basic cron-based functionality that provides the ability to automate backups on a schedule as opposed to
running manual backups from the Deployer node. Backup jobs are configured using UTC, and timing can be customized per database.
Backups include persistent data from Cassandra, Consul, CockroachDB, Elasticsearch, NSO, and Action Orchestrator, which contains
ArangoDB and Postgres.
To enable the cron job backups:
Edit group_vars/all/cron_backup_vars.yml to set your cron schedules and other information. Refer to the comments in the yml file below for guidance.
Note
Lookup variables are used if they are set in the environment or if you run the export command. For example, you might export
these keys: S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY. You will not need to do this in a default AWS setup unless you are
backing up to a different account.
---
### NOTE: cron_schedule should be set using UTC
cron_backup_components:
consul:
cron_schedule: "1 0 * * *"
nso:
cron_schedule: "30 0 * * *"
cassandra:
cron_schedule: "5 0 * * *"
elasticsearch:
cron_schedule: "15 0 * * *"
### Elasticsearch does incremental snapshots by default. Select
### the timeframe to make a full snapshot:
### daily/weekly/monthly (daily is the default)
full_backup: "daily"
cockroachdb:
cron_schedule: "0 1 * * *"
etcd:
cron_schedule: ""
arangodb:
cron_schedule: "0 2 * * *"
### For OpenStack deployments or AWS deployments where you will backup to an S3 bucket in another account,
### uncomment and edit the below options where necessary.
### endpoint (required)
### **NOTE:** for AWS S3 based endpoints use the region specific endpoint ie s3.us-west-2.amazonaws.com
### region (required for AWS S3 based endpoints)
### bucket (required)
### access_key (required)
### secret_key (required)
### session_token (optional)
### base_path (optional)
### protocol (optional, defaults to https. endpoint must not be self-signed for https)
### proxy_xxx (optional), but if you need to define a proxy the _host, _port, _scheme are required
# s3_backup:
# endpoint: '<s3-endpoint>'
# region: '<s3-region>'
# bucket: '<bucket-name>'
# access_key: "{{ lookup('env', 'S3_ACCESS_KEY_ID') }}"
# secret_key: "{{ lookup('env', 'S3_SECRET_ACCESS_KEY') }}"
# session_token: "{{ lookup('env', 'S3_SESSION_TOKEN') }}"
# base_path: '<base-path>'
# protocol: 'https'
# proxy_host: "{{ proxy | urlsplit('hostname') }}"
# proxy_port: "{{ proxy | urlsplit('port') }}"
# proxy_scheme: "{{ proxy | urlsplit('scheme') }}"
# proxy_username: "{{ lookup('env', 'PROXY_USERNAME') }}"
# proxy_password: "{{ lookup('env', 'PROXY_PASSWORD') }}"
Change to the Ansible directory.
cd /msx-4.1.0/ansible
Set the environment variable for the Ansible Vault password:
Where <vault_pwd_path> = /msx-4.1.0/ansible/vault/password.txt
Run cron-backup.yml to start the backup schedule.
ansible-playbook cron-backup.yml
The process will take approximately 15 minutes (with an Elasticsearch restart). Otherwise, it will only take a couple of minutes
if no restart is needed.
Log in to the Kubernetes cluster so that you can check that the jobs are running according to schedule.
#login to k8s master
ssh -F ssh.cfg centos@<master-ip>
On the Kubernetes master, change user to sudo.
sudo su
List all the cron jobs in Kubernetes.
kubectl -n vms get cronjobs
The following is the output of the command:
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
curator 1 0 * * * False 0 <none> 13h
Replacing the Default SSH CIDR
For both AWS and OpenStack, you must change the default 0.0.0.0/0 CIDR for inbound SSH connections to the MSX Inception node.
Edit ansible/group_vars/all/inception_ssh_cidrs.yml file to create an allowed list that matches your access requirements.
For example:
inception_ssh_cidrs:
- 192.0.2.0/24
- 198.51.100.0/24
- 203.0.113.0/24
Creating the AWS Credentials File
To access AWS from the command line, you need to create a credential file and save it as /root/.aws/credentials.
The access_key_id and secret_access_key are generated from within AWS.
Configuring the Vault Password
An Ansible vault password is required each time you use an Ansible playbook to install the MSX platform and service packs.
The default vault password can be obtained from your Cisco representative.
You can change the default password and add it as an environment variable to avoid specifying the password for each ansible
playbook installation. If you do not add the vault password as an environment variable, the system prompts you to enter the
vault password during the execution of each Ansible playbook.
Use this procedure to change the default vault password:
From this point on, the new vault password is used until you exit the terminal session.
Pre-flight Validation Checks
MSX provides both platform and service pack pre-flight validation in order to ensure the current environment matches the
minimum requirements of MSX resources.
The procedure to check the platform:
Procedure
Step 1
From the installation container, run the command:
ansible-playbook pre-flight-platform-check.yml
Step 2
The CLI output displays the status depending on whether the requirements are met:
If a requirement is met, the status is displayed as ‘Pass' .In this case, no further action is required.
If a requirement is not met, the status is displayed as ‘Fail’ or ‘Warn’. In this case, follow the instructions displayed
in the Recommended Action column of the output screen.
Note
A similar pre-flight check can be run for the service packs before deploying them.