Overview
Cisco Policy Suite offers a carrier-grade, high capacity, high performance, virtualized software solution, capable of running on VMware, OpenStack/KVM hypervisors or cloud infrastructures. To meet the stringent performance, capacity, and availability demands, the Cisco software requires that all allocated hardware system resources be 100% available when needed, and not oversubscribed or shared across separate VM's.
The following steps outline the basic process for a new installation of CPS:
-
Review virtual machine requirements
-
Orchestration Requirements
-
Install OpenStack
-
CPU Pinning
-
Configure OpenStack Users and Networks
-
Define Availability Zones
-
Download the required CPS images
-
Import images to Glance
-
Create Cinder Volumes
-
Verify or updated Default Quotas
-
Create Flavors
-
Set up Access and Security
-
Create CPS VMs using Nova Boot Commands
-
Create CPS VMs using Heat
-
Deploy CPS
-
Validate CPS Deployment
-
SR-IOV Support
-
Enable Custom Puppet to Configure Deployment
-
HTTPS Support for Orchestration API
-
Installation APIs
-
Upgrade APIs
-
Unmount ISO
-
Mount ISO
-
Upgrade CPS
-
Upgrade Status
-
-
System Configuration APIs
Virtual Machine Requirements
For customers operating a cloud infrastructure, the infrastructure must be configured to guarantee CPU, memory, network, and I/O availability for each CPS VMs. Oversubscription of system resources will reduce the performance and capacity of the platform, and may compromise availability and response times. CPU core requirements are listed as pCPUs (physical cores) not vCPUs (hyper-threaded virtual cores).
In addition, the CPS carrier-grade platform requires:
-
RAM reservation is enabled for all memory allocated to the CPS VM.
-
CPU Hyperthreading must be ENABLED. To prevent over-subscription of CPU cores, CPU pinning should be ENABLED.
-
CPU benchmark of at least 13,000 rating per chip and 1,365 rating per thread.
-
The total number of VM CPU cores allocated should be 2 less than the total number of CPU cores per blade.
-
Monitor the CPU STEAL statistic. This statistic should not cross 2% for more than 1 minute.
Note
A high CPU STEAL value indicates the application is waiting for CPU, and is usually the result of CPU over allocation or no CPU pinning. CPS performance cannot be guaranteed in an environment with high CPU STEAL.
-
CPU must be a high performance Intel x86 64-bit chipset.
Note
BIOS settings should be set to high-performance values, rather than energy saving, hibernating, or speed stepping (contact hardware vendor for specific values).
-
For deployments which cannot scale by adding more VMs, Cisco will support the allocation of additional CPUs above the recommendation to a single VM, but does not guarantee a linear performance increase.
-
Cisco will not support performance SLAs for CPS implementations with less than the recommended CPU allocation.
-
Cisco will not support performance SLAs for CPS implementations with CPU over-allocation (assigning more vCPU than are available on the blade, or sharing CPUs).
-
Scaling and higher performance can be achieved by adding more VMs, not by adding more system resources to VMs.
-
RAM latency should be lower than 15 nanosecond.
-
RAM should be error-correcting ECC memory.
-
Disk storage performance should be less than 2 millisecond average latency.
-
Disk storage performance needs to support greater than 5000 input/output operations per second (IOPS) per CPS VM.
-
Disk storage must provide redundancy and speed, such as RAID 0+1.
-
Hardware and hardware design must be configured for better than 99.999% availability.
-
For HA deployments, Cisco requires the customer designs to comply with the Cisco CPS HA design guidelines.
-
At least two of each CPS VM type must be deployed: Policy Server (qns), Policy Director (lb), OAM (pcrfclient), Session Manager (sessionmgr).
-
Each CPS VM type must not share common HW zone with the same CPS VM type.
-
-
The number of CPU cores, memory, NICs, and storage allocated per CPS VM must meet or exceed the requirements.
The following table provides information related to vCPU requirements based on:
-
Hyper-threading: Enabled (Default)
-
CPU Pinning: Enabled
-
CPU Reservation: Yes (if allowed by hypervisor)
-
Memory Reservation: Yes (if allowed)
-
Hard Disk (in GB): 100
Physical Cores / Blade |
VM Type |
Memory (in GB) |
Hard Disk (in GB) |
vCPU |
Configuration |
---|---|---|---|---|---|
Blade with 16 CPUs |
Policy Server VMs (QNS) |
16 |
100 |
12 |
Threading = 200 Mongo per host = 10 Criss-cross Mongo for Session Cache = 2 on each VM |
Blade with 16 CPUs |
Session Manager VMs |
128 |
100 |
6 |
|
Blade with 16 CPUs |
Control Center (OAM) VMs |
16 |
100 |
6 |
|
Blade with 16 CPUs |
Policy Director VMs (LB) |
32 |
100 |
12 |
|
Blade with 24 CPUs |
Policy Server VMs (QNS) |
16 |
100 |
10 |
Threading = 100 Mongo per host = 10 Criss-cross Mongo for Session Cache = 2 on each VM Hyper-threading = Default (Enable) |
Blade with 24 CPUs |
Session Manager VMs |
80 |
100 |
8 |
|
Blade with 24 CPUs |
Control Center (OAM) VMs |
16 |
100 |
12 |
|
Blade with 24 CPUs |
Policy Director VMs (LB) |
32 |
100 |
12 |
Physical Cores / Blade |
VM Type |
Memory (in GB) |
Hard Disk (in GB) |
vCPU |
Configuration |
---|---|---|---|---|---|
Blade with 16 CPUs |
Policy Server VMs (QNS) |
16 |
100 |
12+ |
Threading = 200 Mongo per host = 10 Criss-cross Mongo for Session Cache = 2 on each VM |
Blade with 16 CPUs |
Session Manager VMs |
128 |
100 |
6+ |
|
Blade with 16 CPUs |
Control Center (OAM) VMs |
16 |
100 |
6+ |
|
Blade with 16 CPUs |
Policy Director VMs (LB) |
32 |
100 |
8+ |
|
Blade with 24 CPUs |
Policy Server VMs (QNS) |
16 |
100 |
10+ |
Threading = 100 Mongo per host = 10 Criss-cross Mongo for Session Cache = 2 on each VM |
Blade with 24 CPUs |
Session Manager VMs |
80 |
100 |
8+ |
|
Blade with 24 CPUs |
Control Center (OAM) VMs |
16 |
100 |
12+ |
|
Blade with 24 CPUs |
Policy Director VMs (LB) |
32 |
100 |
12+ |
Note |
For large scale deployments having Policy Server (qns) VMs more than 35, Session Manager (sessionmgr) VMs more than 20, Policy Director (lb) VMs more than 2, recommended RAM for OAM (pcrfclient) VMs is 64GB. |
Note |
For large scale deployments having Policy Server (qns) VMs more than 32, Session Manager (sessionmgr) VMs more than 16, Policy Director (lb) VMs more than 2, recommended vCPU for OAM (pcrfclient) VMs is 12+. |
Orchestration Requirements
The following orchestration capabilities are required to administer a CPS deployment in OpenStack:
- Ability to independently create/delete/re-create the Cluster Manager VM.
- Ability to snapshot Cluster Manager VM and restore the Cluster Manager VM from snapshot.
- Ability to attach and detach the ISO cinder volume to/from the Cluster Manager VM.
- CPS recommends that the CPS software ISO be mapped to a cinder volume. In deployments where this recommendation is used, prior to installation or upgrade or migration, the ISO cinder volume must be attached to the Cluster Manager so that the ISO can be mounted inside the Cluster Manager. The sample HEAT template provided in this document demonstrates how to automate mounting the ISO inside Cluster Manager. In deployments where this recommendation is not used, the CPS software ISO must be made available inside Cluster Manager VM and mounted using the method implemented by the customer.
- The Config drive must be used to pass in files such as userdata and the Config drive must be mounted to CPS VM in the 'iso9660' format.
- Any cinder volume required by the product code must be attached first to the VM and any customer environment specific cinder volumes should be attached after. One exception is the ISO cinder volume attached to Cluster Manager VM. In cases where ISO cinder volume is attached in a different order, the API to mount the ISO needs to be supplied with the right device name in the API payload.
- eth0 needs to be on the 'internal' network for inter-VM communication.
- On all CPS VMs, the Cluster Manager IP needs to be injected in /etc/hosts to ensure connectivity between each host and the Cluster Manager.
- CPS VM's role needs to be
injected in
/etc/broadhop/.profile
, for example:NODE_TYPE=pcrfclient01
- For upgrades/migration and rollbacks, the orchestrator must
have the ability to independently create/delete/re-create half/all of the
following CPS VMs:
-
Policy Server (qns)
-
Policy Director (lb and iomanager)
-
OAM (pcrfclient)
-
Session Manager (sessionmgr)
During a rollback, half of SM VMs must be deleted during Rollback procedure. As a result, the replica sets must be configured such that not all members of the replica set are in one half or the other. Replica set members must be distributed across the two upgrade sets so that the replica set is not entirely deleted. Refer to the CPS Migration and Upgrade Guide for more details.
-
- For scaling, the
orchestrator must have the ability to independently create/delete/re-create
half/all of the following CPS VMs in each scaling unit:
-
Policy Server (qns)
-
Session Manager (sessionmgr)
-