Cisco SD-WAN Cloud OnRamp for Colocation Solution Guide, Release 19.1

Monitor Operational Status of Cloud OnRamp for Colocation Devices from vManage

Monitoring Cloud OnRamp for Colocation devices is the process of reviewing and analyzing a Cloud OnRamp for Colocation device, such as Cloud Services Platform (CSP) devices and Cisco Colo Manager for health, inventory, availability, and other operation-related processes. You can also monitor the components of devices such as CPU, memory, fan, temperature, and so on. For more information about the monitoring dashboard, see the Cisco SD-WAN Configuration Guides configuration guides.

All notifications are sent to the vManage notification stream. To view the notification stream, see Cisco SD-WAN Command Reference.

Procedure

Step 1	Log into Cisco vManage, at the URL HTTPS: //vManage-ip-address/, as a user with "admin" credentials.
Step 2	To view a list of all devices, click Monitor > Network. A table lists information about all devices. If vManage cannot reach CSP devices and Cisco Colo Manager cannot reach switches, the devices are shown as unreachable.
Step 3	To monitor a device, in the WAN Edge tab, click a CSP device or Cisco Colo Manager from the list by clicking its hostname. By default, the System Status screen or Application screen appears. A horizontal bar at the top of the screen with the device drop-down, device name, device IP address, device site location, device model, and more information drop-down is displayed.
Step 4	If necessary, select a different device that you want to monitor.
Step 5	From the Select Device bar, click the More Info drop-down located to the right of the bar. vManage NMS opens a drop box with a summary information about the device. In the left pane, the following are the categories of information about the device that you can view: VNF Status—view performance specifications, required resources, and component network functions for each VNF See View Information About VNFs from vManage. Interface—view Interface status and statistics See the "View Interfaces" topic in the Cisco SD-WAN Configuration Guides. Control Connections—view status and statistics for control connections See the "View Control Connections" topic in the Cisco SD-WAN Configuration Guides. System Status—view reboot and crash information, hardware component status, and CPU and memory usage See the "View Control Connections" topic in the Cisco SD-WAN Configuration Guides Colo Manager—view Cisco Colo Manager health status See View Cisco Colo Manager Health from vManage. Events—view latest syslog events. See the "View Events" topic in the Cisco SD-WAN Configuration Guides. Troubleshooting—view information about pings and traceroute traffic connectivity tools See the "Troubleshoot a Device" topic in the Cisco SD-WAN Configuration Guides. Real Time—view real-time device information for feature-specific operational commands. See the "View Real-Time Data" topic in the Cisco SD-WAN Configuration Guides.
Step 6	To monitor clusters, click the Colocation Clusters tab from the Monitor\|Network screen. All clusters with relevant information are displayed in a tabular format. Click a cluster. See Monitor Cloud OnRamp Colocation Clusters from vManage for more information.

View Information About VNFs from vManage

Table 1. Feature History
Feature Name	Release Information	Description
VNF States and Color Codes	Cisco SD-WAN Release 20.1.1	This feature allows you to determine the state of a deployed VM using color codes, which you can view on the Monitor > Network page.

Table 2. Feature History
Feature Name	Release Information	Description
Network Utilization Charts for SR-IOV Enabled NICs and OVS Switch	Cisco SD-WAN Release 20.1.1	This feature allows you to view network utilization charts of VM VNICs connected to both SR-IOV enabled NICs and OVS switch.

You can view performance specifications and required resources for each VNF. Reviewing this information can help you to determine which VNF to use when you are designing a network service. To view information about VNFs, perform the following steps:

Procedure

Step 1

Log into Cisco vManage with "admin" credentials using the URL HTTPS: //vManage-ip-address/.

Step 2

In Cisco vManage, click Monitor > Network.

The right pane displays VNF information in a tabular format. The table includes information such as CPU use, memory consumption, and disk, and other core parameters that define performance of a network service.

Step 3

Click a CSP device from the table.

Step 4

From the left pane, click VNF Status.

Step 5

From the table, click the VNF name. The right pane displays information about the specific VNF. You can click the network utilization, CPU utilization, memory utilization, and disk utilization to monitor the resources utilization of a VNF.

The primary part of the right pane contains the following VNF information:

Table 3. VNF Information
Chart options bar	VNF information in graphical format	VNF information in color coded format
Chart Options drop-down—Click Chart Options drop-down list to select the type of data to display. Time periods—Click either a predefined time period, or a custom time period for which to display data.	Choose a VNF from the Select Device drop-down list to display information for the VNF.	The VNFs are shown in specific colors based on the following operational status of the VNF life cycle: Green—VNF is healthy, deployed, and successfully booted up. Red—VNF deployment or any other operation fails, or VNF stops. Yellow—VNF is transitioning from one state to another.

The detail part of the right pane contains:

Filter criteria
VNF table that lists information about all VNFs or VMs. By default, the first six VNFs are checked. The network utilization charts for VNICs connected to SR-IOV enabled NICs and OVS switch are displayed.

Figure 1. VNF Information

The graphical display plots information for the checked VNFs
- Click the checkbox at the left to select and deselect VNFs. You can select and display information for a maximum of six VNFs at one time.
- To change the sort order of a column, click the column title.

View Cisco Colo Manager Health from vManage

You can view Cisco Colo Manager health for a device, Cisco Colo Manager host system IP, Cisco Colo Manager IP, and Cisco Colo Manager state. Reviewing this information can help you to determine which VNF to use when you are designing a network service. To view information about VNFs, perform the following steps:

Procedure

Step 1	Log into Cisco vManage, at the URL HTTPS: //vManage-ip-address/, as a user with "admin" credentials.
Step 2	In vManage, click Monitor > Network. The right pane displays VNF information in a tabular format. The table includes information such as CPU use, memory consumption, and disk, and other core parameters that define performance of a network service.
Step 3	Click a CSP device from the table.
Step 4	From the left pane, click Colo Manager. The right pane displays information about the memory usage, CPU usage, uptime, and so on, of the colo manager.

Monitor Cloud OnRamp Colocation Clusters from vManage

Table 4. Feature History
Feature Name	Release Information	Description
Network Assurance –VNFs: Stop/Start/Restart	Cisco SD-WAN Release 20.3.1 Cisco vManage Release 20.3.1	This feature provides the capability to stop, start, or restart VNFs on Cisco CSP devices from the Colocation Clusters tab. You can easily perform the operations on VNFs using Cisco vManage.

You can view the cluster information and their health states. Reviewing this information can help you to determine which Cisco CSP device is responsible for hosting each VNF in a service chain. To view information about a cluster, perform the following steps:

Procedure

Step 1

Log into Cisco vManage with "admin" credentials, at the URL HTTPS: //vManage-ip-address/.

Step 2

In Cisco vManage, click Monitor > Network.

Step 3

To monitor clusters, click Colocation Clusters.

All clusters with relevant information are displayed in a tabular format. Click a cluster name. You can monitor cluster from the Config View and Port Level View.

Config. View: The primary part of the left pane displays the appliances and switches that form the cluster. In the right pane, you can view the cluster information such as the available and total CPU resources, available and allocated memory, and so on, based on Cloud OnRamp for Colocation size.

The detail part of the left pane contains:

Filter criteria: Type a field value to filter and display the information.
A table that lists information about all devices in the cluster (Cisco CSP devices, PNFs, and switches).

Click a Cisco CSP device. VNF information is displayed in tabular format. The table includes information such as VNF name, service chains, number of CPUs, memory consumption, and other core parameters that define performance of network service. See View Information About VNFs from vManage .

To start, stop, or reboot a VNF, click More Action next to the VNF name, and choose one of the following operations:
- Start.
- Stop.
- Restart.
Figure 2. Start, Stop, or Restart VNFs

Note

Ensure that service chain provisioning has been completed and VMs are deployed, before issuing start, stop, restart operations on any of the VNFs in that chain.

After you choose an operation on a VNF, wait until the operation is complete before you issue another operation. You can view the progress of an operation from the task view window.

Port Level View: After you activate the cluster, to view the port connectivity details, click Port Level View.

You can view detailed port connectivity information for the switches and CSP devices in a color coded format based on SR-IOV and OVS modes.

To view the mapping of ports between the Catalyst 9500 switches and CSP devices, click or hover over a CSP device.

Figure 3. Monitor Port Connectivity Details of a Cluster

Step 4

Click Services.

In this tab, you can view:

The monitoring information of a service chain can be viewed in tabular format. The first two columns display the name and description of the service chain within the service group and the remaining columns mention about the VNF, PNF statuses, monitoring service enablemement, and the overall health of a service chain. The various health statuses and their representations are:

Healthy—An up arrow in green A service chain is in 'Healthy' status when all the VNF, PNF devices are running and are in healthy state. Ensure that you configure the routing and policy correctly.
Unhealthy—A down arrow in red. If one of the VNFs or PNFs are in unhealthy state, the service chain is reported to be in 'Unhealthy' status. For example, after deploying a service chain, if one of the network function IP address changes on the WAN or LAN side, or the firewall policy is not configured to let the traffic pass through, then unhealthy state is reported. This is because the network function or overall service chain is Unhealthy or both are in Unhealthy state.

Undetermined—Down arrow in yellow. This is a third state that is reported when the health of the service chain cannot be determined. This state is also reported when there is no status such as healthy or unhealthy available for the monitored service chain over a time period. You cannot query or search a service chain with undetermined status.

If a service chain consists of a single PNF and PNF is orchestrated outside of vManage, then it cannot be monitored. If a service chain consists of a single network function, firewall that has VPN termination on both sides which cannot be monitored, then it is reported as Undetermined.

Note

If the status of a service chain is undetermined, you cannot choose the service chain to view the detailed monitoring information.

Figure 4. Service Chain Health Monitoring Results

Click a service group that is in Healthy or Unhealthy state. The primary part of the service chain monitoring in the right pane contains the following elements:

Figure 5. Service Chain Health Monitoring Status

Graphical display that plots the latency information of the service chain, VNFs, PNFs.

The detail part of the right pane contains:
- Filter criteria
- A table that lists information about all service chains, VNFs, PNFs, their health status, and types.
  - Check the checkbox at the left of a row to select and deselect a service chain, VNF, PNF.
  - To change the sort order of a column, click the column title.
In the following image, the status details column indicates the monitored data path and it provides the per hop analysis.
Click the Diagram button and view the service group with all its service chains and VNFs in the design view window.
Click a VNF. You can view CPU, memory, and disk allocated to the VNF in a dialog box.
Select a service group from the Service Groups drop-down. The design view displays the selected service group with all its service chains and VNFs.

Step 5

Click Network Functions.

In this tab, you can view:

All the virtual or physical network functions in a tabular format. From the Show button, you can choose to display either a VNF or PNF.

VNF information is displayed in a tabular format. The table includes information such as VNF name, service chains, CPU use, memory consumption, and other core parameters that define performance of network service. To view more information about the VNF, click a VNF name. See View Information About VNFs from vManage .
PNF information is displayed in tabular format. The table includes information such as the serial number and PNF type. To view and note configuration of a specific PNF, click the desired PNF serial number. Ensure that you manually note all the configuration of the PNFs and then configure the PNF devices. For example, the following are some of the PNF configuration where you position the PNF at various locations in the service chain. See Custom Service Chain with Shared PNF Devices to create services chains by adding PNFs. Also, see the ASR 1000 Series Aggregation Services Routers Configuration Guides and Cisco Firepower Threat Defense Configuration Guides to configure the PNFs manually.

Figure 6. PNF in the First Position with Service Chain Side Parameters

Figure 7. PNF in the First Position with Outside Neighbor Information

Figure 8. PNF Shared Across Two Service Chains

The ServiceGroup2_chain3 is a PNF-only service chain and therefore no configuration gets generated. The PNF is in the last position of the ServiceGroup2_chain1, so only INSIDE variables gets generated.

Figure 9. PNF Shared Across Two Service Chains with Outside Neighbor Information

Cisco Colo Manager States for Switch Configuration

The various Cisco Colo Manager states and transitions when you trigger various processes from vManage are:

INIT state—when the Cisco Colo Manager container is successfully initialized.
IN-PROGRESS state—when any configuration push is not possible.
SUCCESS state—when the Cisco Colo Manager container has successfully translated and pushed the intent that is received from vManage to Cisco Catalyst 9500-40X or Cisco Catalyst 9500-48Y4C switches.
FAILURE state—If there is any failure in processing or configuration push in Cisco Colo Manager.

When vManage pushes the Cloud OnRamp for Colocation configuration intent to the Cisco Colo Manager for the first time it moves from INIT to IN-PROGRESS state. After Cisco Colo Manager pushes the configuration, it goes back to the SUCCESS or FAILURE state. For every incremental configuration push, it goes to IN-PROGRESS state. If any of the configurations pushes fail, Cisco Colo Manager goes into FAILURE state.

Note

A notification is sent when Cisco Colo Manager state changes. See Cisco Colo Manager Notifications.

Cisco Colo Manager States and Transitions from Host

vManage depends on various CSP hosts state for the Cisco Colo Manager to be brought up, which are:

Starting—when Cisco Colo Manager has been brought up and health check script has not been run. During this phase, vManage waits for CSP state to change to Healthy.
Healthy—when the health check script has been run and it has passed the checks. This state implies that the operational model for configuration status can be queried or configuration can be pushed. During this phase, if Cisco Colo Manager is in INIT state, vManage pushes the device list. If Cisco Colo Manager is not in INIT state, Cloud OnRamp for Colocation may be in degraded state and recovery flow must happen.
Unhealthy—when all the necessary packages in Network Services Orchestrator (NSO) are not up. This state can be due to various reasons such as, NSO did not come up, Cisco Colo Manager package did not come up, or other reasons. This state implies that the operational model for configuration status is not up and configuration cannot be pushed.

Cisco Colo Manager Notifications

You can view the Cisco Colo Manager notifications from Cisco Colo Manager console by using the show notification stream viptela command.

The various Cisco Colo Manager internal state machines are:

Cisco Colo Manager States

Notification Trigger

Notification Output Example

INIT

Init: Cloud OnRamp for Colocation is activated and vManage brings up Cisco Colo Manager on Cisco CSP.

Note

The Cisco Colo Manager state must be in "Init" only when the docker container is initially brought up and must not be in this state unless container is deleted and brought up again.

admin@ncs# show notification stream viptela last 50
notification
 eventTime 2019-04-08T17:15:15.982292+00:00
 ccmEvent
  severity-level minor
  host-name ccm
  user-id vmanage_admin
  config-change false
  transaction-id 0
  status SUCCESS
  status-code 0
  status-message init
  details Initializing CCM
  event-type CCM-STATUS
 !
!

IN-PROGRESS

vManage pushes intent and Cisco Colo Manager moves to in-progress state.

Note

Cisco Colo Manager generates multiple in-progress notifications for the switches that are brought up.

notification
 eventTime 2019-04-08T17:37:54.536953+00:00
 ccmEvent
  severity-level minor
  host-name ccm
  user-id vmanage_admin
  config-change false
  transaction-id 0
  status SUCCESS
  status-code 0
  status-message IN-PROGRESS
  details Received configuration from vManage
  event-type CCM-STATUS
 !

SUCCESS

During cluster activation, after Cisco Catalyst 9500 switches have been successfully onboarded, status moves to SUCCESS. For any incremental configuration, status moves to SUCCESS only if configuration has been saved successfully in the switch devices.


notification
 eventTime 2019-04-08T17:51:48.044286+00:00
 ccmEvent
  severity-level minor
  host-name ccm
  user-id vmanage_admin
  config-change false
  transaction-id 0
  status SUCCESS
  status-code 0
  status-message SUCCESS
  details Devices done onboarding
  event-type CCM-STATUS
 !
!
admin@ncs#

FAILURE

If onboarding of switches fails during cluster activation, CCM status moves to FAILURE. If any incremental configurations are not saved, CCM status moves to FAILURE.

Note

The failure state cannot transition to another state without end-user intervention.


notification
 eventTime 2019-04-08T18:01:44.943198+00:00
 ccmEvent
  severity-level critical
  host-name ccm
  user-id vmanage_admin
  config-change false
  transaction-id 0
  status FAILURE
  status-code 0
  status-message FAILURE
  details SVL bringup not successful. Could not sync TenGigabitEthernet2/0/* interfaces.
  event-type CCM-STATUS
 !
!
admin@ncs#

Onboarding of switches fails during cluster activation due to wiring errors in flexible connections, and CCM status moves to FAILURE.

admin@ncs# show notification stream viptela last 100 | include Step
notification
 details Step 5 of 7: Device switch1 : 192.168.100.21 (C9500-48Y4C-CAT2324L2HM) connected after SVL reload.  
details Step 6 of 7: Started sync-from for primary device switch1 : 192.168.100.21 (C9500-48Y4C-CAT2324L2HM)  
details Step 6 of 7: Sync-from done for primary device switch1 : 192.168.100.21 (C9500-48Y4C-CAT2324L2HM) 
Device list : switch1 : 192.168.100.21 (C9500-48Y4C-CAT2324L2HM), switch2 : 192.168.100.19 (C9500-48Y4C-CAT2316L2F2)  
details Step 6 of 7: Devices ready for LLDP query Device list : switch1 : 192.168.100.21 (C9500-48Y4C-CAT2324L2HM), switch2 : 192.168.100.19 (C9500-48Y4C-CAT2316L2F2)  
details Step 6.1 of 7: LLDP Query Details: csp2 has 8/8 interfaces connected, 2/4 sriov, 2/4 fortville to primary switch; 2/4 sriov, 2/4 fortville to secondary switch;  
Found devices with not optimum connections:- csp1 has 6/8 interfaces connected, 2/4 sriov, 2/4 fortville to primary switch; 2/4 sriov, 0/4 fortville to secondary switch; 
Minimum Requirement is to have 8/8 interfaces per CSP in cluster. Recommended action: Please refer to recommended topologies and minimum requirements  details Step 7 of 7: 
Devices done onboarding Device list : switch1 : 192.168.100.21 (C9500-48Y4C-CAT2324L2HM), switch2 : 192.168.100.19 (C9500-48Y4C-CAT2316L2F2)

VM Alarms

The following are VM alarms and they can be viewed from vManage, when alarms are received.

Table 6. Alarms
Alarm	Trigger Condition	Syslog Messages
INTF_STATUS_CHANGE	interface status change	nfvis %SYS-6-INTF_STATUS_CHANGE: Interface eth0, changed state to up
VM_STOPPED	vm stopped	nfvis %SYS-6-VM_STOPPED: VM stop successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c
VM_STARTED	vm started	nfvis %SYS-6-VM_STARTED: VM start successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c
VM_REBOOTED	vm rebooted	nfvis %SYS-6-VM_REBOOTED: VM reboot successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c
VM_RECOVERY_INIT	vm recovery initiation	nfvis %SYS-6-VM_RECOVERY_INIT: VM recovery initiation successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c
VM_RECOVERY_REBOOT	vm recovery reboot	nfvis %SYS-6-VM_RECOVERY_REBOOT: VM recovery reboot successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c
VM_RECOVERY_COMPLETE	vm recovery complete	nfvis %SYS-6-VM_RECOVERY_COMPLETE: VM recovery successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c
VM_MONITOR_UNSET	vm monitoring unset	nfvis %SYS-6-VM_MONITOR_UNSET: Unsetting VM monitoring successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c
VM_MONITOR_SET	vm monitoring set	nfvis %SYS-6-VM_MONITOR_SET: Setting VM monitoring successful: SystemAdminTena_ROUTER_0_df6733c1- 0768-4ae6-8dce-b223ecdb036c

See Cisco NFVIS Configuration Guide for more information about syslog support and VM alarms.

Cloud Services Platform Real-Time Commands

Table 7. Real-Time Commands
System Information
Container status
show control connections
Control connection history
Control local properties
Control summary
Control statistics
Control valid vEdges
valid vManage ID
HW Alarms
HW Environments
PNICs
System Status
Host System Mgmt Info
Host System settings
Host System processes
Resource CPU allocation
RBAC Authentication
Resource CPU VNFs
Hardware Inventory
Hardware Temperature thresholds
Control affinity stats

Book Title

Chapter Title

Results

Chapter: Monitor Cisco SD-WAN Cloud OnRamp for Colocation Devices

Monitor Cisco SD-WAN Cloud onRamp for CoLocation Solution Devices

Monitor Operational Status of Cloud OnRamp for Colocation Devices from vManage

Procedure

View Information About VNFs from vManage

Procedure

View Cisco Colo Manager Health from vManage

Procedure

Monitor Cloud OnRamp Colocation Clusters from vManage

Procedure

Cisco Colo Manager States for Switch Configuration

Cisco Colo Manager States and Transitions from Host

Cisco Colo Manager Notifications

VM Alarms

Cloud Services Platform Real-Time Commands

Was this Document Helpful?

Contact Cisco

Related Cisco Community Discussions