Table Of Contents
Cisco WAN Manager Peer-to-Peer Communication
Displaying the CWMGateway Monitor
Establishing Primary CWM and Secondary CWM Priority
Re-establishing Peer-to-Peer Communication Priority
Secondary CWM Graceful Shutdown
Configuring Peer-to-Peer Communication
Primary CWM Has Lost IP Connectivity to the Only Secondary CWM in a Domain
Bidirectional Heartbeat Messages
Limitations for CWM to CWM Communications
Enabling CWM to CWM Communications
Steps for Executing the usertblDBsync and usertblDBcmp Scripts
Cisco WAN Manager Peer-to-Peer Communication
Release 12 of Cisco WAN Manager has been designed to enable multiple CWM workstations to manage a network with improved synchronization and scalability. Due to the size and growth of networks, it is faster to retrieve initial user data from another CWM workstation that is already running and synchronized with the network. An industry standard CORBA architecture is used to implement the communications between two or more CWM workstations through a server-client structure for communications between the CWM server and client processes.
CWM workstations use peer-to-peer communications to synchronize user data with each other. When user data is provisioned or changed, the CWM workstations will propagate the new data to the other CWM workstations. The user is able to continue the provisioning of network data, even when communications between a Primary CWM and Secondary CWM have been interrupted. If for any reason the communications between CWM servers are interrupted, the provisioning of the user data will be suspended on the Secondary CWM, but user data provisioning will continue on the Primary CWM, as user data provisioning will continue on the Primary CWM. During that time, the provisioning of user data and monitoring of the network are not impacted. This is called the Degraded Mode of Operation.
Release 12 of Cisco WAN Manager allows for better handling of peer-to-peer communications failure scenarios. An automatic switchover feature (AutoSwitchOver) has been designed to help a Secondary CWM get out of the degraded mode of operation, without requiring a restart of the CWM core on either the Primary CWM or Secondary CWM. In most cases, a Secondary CWM can get out of the degraded mode of operation automatically within a limited period of time.
In addition to the AutoSwitchOver feature, peer-to-peer communications enhancements include a bidirectional heartbeat feature which allows for better efficiency in the sending and checking of heartbeat messages, and also makes it possible for a Primary CWM to detect and report the connectivity status of Secondary CWMs. This enhanced communication between the Primary CWM and Secondary CWM aids in avoiding a degraded mode of operation. Also, a more reliable mechanism for reporting communication problems through SNMP traps generation for critical CWM-CWM events can expedite users reaction to the problems that could affect peer-to-peer communications.
Note
Peer-to-Peer communications does not affect network sync-up.
CWM Domain
A CWM domain consists of all CWM workstations in a network that are in communication with each other. Each CWM workstation functions as a CWM gateway, however one CWM workstation is designated as the Primary CWM for keeping the user data, and is the source of all user data that the Secondary CWMs can sync with.
You can define a CWM domain by specifying a list of CWM workstations that will communicate with each other as described in the "Configuring Peer-to-Peer Communication" section. All CWM workstations in a given domain must be connected to the same set of networks to ensure that their databases remain consistent.
Note
CWMGateway Process
The CWMGateway process provides a communications gateway for CMW workstations. Processes owned by one CWM workstation can communicate with processes owned by another CWM workstation using the CWMGateway process.
CWM workstations within the same CWM domain communicate with other CWM workstations by transferring information between the CWMGateway processes of each CWM workstation. The communication between CWM workstations is set up transparently by the CWMGateway processes in each CWM workstation.
CWM core processes communicate with the CWMGateway process of the CWM workstation to request information from and to send information to another CWM workstation.
CWMGateway Functions
With the use of the CWMGateway Process, CWM to CWM communications allows CWM to have the following functionality:
•
Note
•
•
•
Apart from the redundancy aspect, one additional benefit of the CWM gateway is the ability for multiple CWM workstations to share User Data, as well as maintain synchronization with the network. The CWM Gateway process maintains consistency of user data across the CWM domain, while the proprietary Robust Trap mechanism and SNMP maintain the CWM database consistent with the network data.
Network Data is defined as data that originates in the network and is communicated to the CWM workstation(s) by means of the proprietary Robust Trap mechanism or by SNMP, depending on the network element concerned. An example of network data is a change in alarm status for a user port or access line.
User Data is information that is supplied by a CWM user, or by an external OSS, and which cannot be stored in a network element and was therefore not visible to other CWM workstations prior to Release 10.4. Examples of User Data include connection templates, Service Class Templates, SNMP community strings, and the unique node IDs generated by CWM during the network discovery process.
With the CWM gateway function, User Data is propagated between the CWM workstations in a domain, thereby maintaining consistency. The Primary CWM acts as an arbitrator to prevent contention between CWM workstations for the same element of User Data. This means, for example, that if a user on one CWM workstation wants to modify a particular connection template, the Connection Template Manager process on that CWM workstation must request the Primary CWM workstation to lock that resource to prevent concurrent modification by two users.
Displaying the CWMGateway Monitor
You can display all the CWM workstations in the domain, role, and user_table DSSync Status.
To display the gateway monitor, choose Apps > Gateway Monitor.
Figure A-1 displays the CWM role as primary.
Figure A-1 Gateway Monitor
Establishing Primary CWM and Secondary CWM Priority
In a given wide area network managed by Release 12 of CWM, the first CWM workstation to begin operation will assume the role of Primary CWM. As other CWM workstations become active, they will take on Secondary CWM workstation roles. The only difference in function between Primary and Secondary CWM workstations is that the Primary CWM workstation would provide the Secondary CWM workstations with user data when the Secondary CWM workstation launches.
Priority numbers of all Secondaries are assigned by the Primary at the time a Secondary registers with the Primary. It is based on "first-come-first-serve" logic. All the Secondaries have the same privilege except that the Secondary with priority 1 will take over as the Primary if the Primary shuts down.
The priority numbers of Secondaries might change in a Secondary CWMGateway process's lifecycle.
In the following cases, the priority numbers among Secondary CWMGateways might change randomly:
•
•
•
This is the result of re-registration with the new Primary CWMGateway process. Priority numbers will be re-assigned by the new Primary CWMGateway process when a Secondary registers with the new Primary. In other words, for whatever reason the Primary CWMGateway process changes, the priority numbers of the Secondaries will be subject to re-assignment.
It is possible that an S2 (Secondary with Priority 2) can become an S1 after the Primary CWMGateway is restarted (by watchdog). It is also possible that an S3 can become an S1 after the switchover.
In the following cases, the priority numbers among Secondary CWMGateways will NOT change randomly:
•
•
In these cases, if Sn (Secondary with priority n) is restarted or shutdown, any Sm (m>n) will become S(m-1), while any Sm (m<n) will remain as Sm.
The difference between these scenarios is that in a Shutdown Secondary case, the Primary CWMGateway process did not change.
Re-establishing Peer-to-Peer Communication Priority
This section describes how to re-establish peer-to-peer communication priority.
Primary CWM Graceful Shutdown
The first CWM to be launched assumes the role of the primary by default. Subsequently, other CWMs launched will register with the primary CWM and each Secondary CWM is assigned a priority number which identifies the order in which the Secondary CWM was launched with respect to the primary CWM. Before being shutdown gracefully by the user, the CWMGateway on the primary CWM will invoke the IDL interface (used to communicate between two CWMGateway processes), of each of the objects corresponding to a Secondary CWMGateway. This will notify the Secondary CWM that the primary CWM is about to go away and that it will in turn nominate the second CWM in line (priority 1) to take over as the new primary. The Secondary CWMGateway with priority number 1 (SP1) will now take over as the new Primary, and any remaining CWMGateways will subsequently register with the new Primary CWM. The new CWMGateway priority numbers will be based on "first-come-first-serve" logic.
Note
Secondary CWM Graceful Shutdown
A Secondary CWMGateway will notify its primary counterpart before it's shutdown gracefully. The primary CWMGateway will assign a new priority number to the remaining Secondary CWMGateway whose priority number is greater than the priority number of the CWMGateway that has just gone away. The new priority number is one less than the previously assigned priority number. For example, if the CWMGateway with priority number 1 has gone away, the CWMGateway with priority number 2 will be changed to 1 so that it becomes next in line to take over the role of the primary CWM when the current primary CWM goes away.
Note
Configuring Peer-to-Peer Communication
When a CWM workstation is launched, it reads a configuration file called CWMGateway.conf to determine its initial setup and default configurations. Following is an example of a CWMGateway.conf file with a DomainGatewayList of a domain consisting of four CWM workstations named cwmws1, cwmws2, cwmws3, and cwmws4:
DomainGatewayList cwmws1 cwmws2 cwmws3 cwmws4ForcedSwitchOver cwmws2HeartBeatInterval 20SP1_AutoSwitchOver ONPrimaryRestartInterval 15Parameters in this CWMGateway.conf file must be specified to enable inter-CWM communication; it is the only list of other CWM gateways available to this local host. Each host or CWM workstation listed in the DomainGatewayList must be reachable from the local host. Confirm this by using the UNIX "ping" command. The host names listed in the DomainGatewayList can be presented in any order.
The CWMGateway.conf file in /usr/users/svplus/config/ specifies the following parameters:
•
Note
•
Three conditions are required for a forced switchover to take effect:
•
•
•
This option is not read at startup or in response to an HUP or USR1 signal, but processed only in response to a USR2 signal.
To initiate a forced switchover follow these steps:
a.
b.
c.
•
•
•
Degrade Mode
The user is able to continue the provisioning of network data, even when communications between a Primary CWM and Secondary CWM have been interrupted. If for any reason the communications between CWM servers are interrupted, user data provisioning will be suspended on the Secondary CWM, but user data provisioning will continue on the Primary CWM. During that time, the provisioning of user data and monitoring of the network are not impacted. This is called the Degraded Mode of Operation.
In order to provide peer-to-peer communications, a CWM workstation must be able to determine, transparently to the network, if another CWM workstation is currently running in the network. This requires IP connectivity between all CWM workstations. If all Secondary CWMs have lost IP connectivity with the Primary CWM, then all Secondary CWMs will function in the degraded mode of operation.
Degrade mode is defined by a loss of connectivity between the Primary CWM and any Secondary CWMs, in which all Secondary CWMs are unable to provision user data, including the adding, deleting, or modifying of user data, while waiting for a connection with the Primary CWM to be restored. In the degrade mode a Secondary CWM can still manage the network data (not user data), and provisioning of network data can still proceed.
Release 12 of Cisco WAN Manager allows for better handling of peer-to-peer communications failure scenarios. An automatic switchover feature (AutoSwitchOver) has been designed to help a Secondary CWM get out of the degraded mode of operation, without requiring a restart of the CWM core on either the Primary CWM or Secondary CWM. In most cases, a Secondary CWM can get out of the degraded mode of operation automatically within a limited period of time.
Recovering From Degrade Mode
There is one case in which a CWM cannot get out of the degraded mode of operation with the AutoSwitchOver feature, and instead requires a forced switchover by the user. This happens when there is an interruption in communications between CWMs that may be due to a Primary CWM that has lost IP or physical connectivity with the only Secondary CWM in a domain.
If there is a problem with the Primary CWM, then the Secondary CWM will take over as the new Primary after the user does a forced switch over. The user will then need to fix the downed Primary CWM and re-establish IP connectivity.
Primary CWM Has Lost IP Connectivity to the Only Secondary CWM in a Domain
Note
–
–
Forced Switchover
Prior to Release 12 of CWM, a user could only do a forced switchover on a Primary CWM. If any Secondary CWMs went into the degraded mode of operation due to a disconnect with the Primary CWM, the Secondary CWM had to wait for the problem that caused the disconnect to be resolved first before getting out of the degraded mode of operation. If the connection problem couldn't be resolved promptly, the Secondary CWM had to be restarted in order to isolate the current Primary CWM.
Release 12 of CWM has improved the CWMGateway by providing the user a method to force a Secondary CWM's switchover to a Primary CWM without restarting the running CWM through this expanded forced switchover capability.
Note
Bidirectional Heartbeat Messages
A bidirectional heartbeat feature is part of the peer-to-peer communications enhancements for
Release 12 of CWM which allows for better efficiency in the sending and checking of heartbeat messages. This feature also makes it possible for a Primary CWM to detect and report the connectivity status of Secondary CWMs. This enhanced communication between the Primary CWM and Secondary CWM aids in avoiding a degraded mode of operation. Also, a more reliable mechanism for reporting communication problems through SNMP traps generation for critical CWM-CWM events can expedite users reaction to the problems that could affect peer-to-peer communications.Prior to Release 12 of CWM, the heartbeat messages were only sent from the Primary CWM to the Secondary CWM through a oneway CORBA interface. Release 12 of CWM provides for a non-oneway CORBA interface so that messages are now sent bidirectionally.
Limitations for CWM to CWM Communications
•
Note
•
Enabling CWM to CWM Communications
In Release 12, CWM provides a script to sync up a CWM database with another remote CWM workstation without running the two CWMs in primary-secondary mode.
Note
The script synchronizes the following user-related tables in the Stratacom database:
•
•
•
•
•
•
•
•
•
•
•
•
•
Note
Steps for Executing the usertblDBsync and usertblDBcmp Scripts
Execute the following steps to copy the remote table containing user data information to the database on the local machine by running the usertblDBsync script.
Step 1
% usertblDBsync <remote_CWM_workstation_name>
Example:
mmen% usertblDBsync mmenu10
Note
•
•
•
•
•
•
•
•
•
•
•
•
•
Continue to sync? [No]y
***********Syncing user tables with CWM on host [cwmtopo62]***********
Syncing Table node_info@mmendsl2 with
node_info@cwmtopo62.....................[DONE]
Syncing Table user_info@mmendsl2 with
user_info@cwmtopo62.....................[DONE]
Syncing Table sec_profile@mmendsl2 with
sec_profile@cwmtopo62.................[DONE]
Syncing Table xpvc_preferred@mmendsl2 with
xpvc_preferred@cwmtopo62...........[DONE]
Syncing Table xpvc@mmendsl2 with
xpvc@cwmtopo62...............................[DONE]
Syncing Table xpvc_segment@mmendsl2 with
xpvc_segment@cwmtopo62...............[DONE]
Syncing Table sct@mmendsl2 with
sct@cwmtopo62.................................[DONE]
Syncing Table sct_cosb@mmendsl2 with
sct_cosb@cwmtopo62.......................[DONE]
Syncing Table sct_vc@mmendsl2 with
sct_vc@cwmtopo62...........................[DONE]
Syncing Table sct_usage@mmendsl2 with
sct_usage@cwmtopo62.....................[DONE]
Syncing Table conn_template@mmendsl2 with
conn_template@cwmtopo62.............[DONE]
Syncing Table conn_templ_param@mmendsl2 with
conn_templ_param@cwmtopo62.......[DONE]
Syncing Table user_conn_desc@mmendsl2 with
user_conn_desc@cwmtopo62...........[DONE]
mmendsl2-11->
Executing this script copies the following user-related tables in the stratacom database from the remote CWM workstation specified by <remote_CWM_workstation_name> to the local machine:
•
•
•
•
•
•
•
•
•
•
•
•
•
Note
Execute the following steps to compare the remote table containing user data information to the database on the local machine by running the usertblDBcmp script.
Step 1
% usertblDBcmp <remote_CWM_workstation_name>
Example:
mmen% usertblDBcmp mmenu10
•
•
•
•
•
•
•
•
•
•
•
•
•
cwmtopo62-10-> usertblDBcmp mmen
***********Comparing user tables with CWM on host [mmen]***********
Comparing Table node_info@cwmtopo62 with
node_info@mmen...................[SAME]
Comparing Table user_info@cwmtopo62 with
user_info@mmen...................[SAME]
Comparing Table sec_profile@cwmtopo62 with
sec_profile@mmen...............[SAME]
Comparing Table xpvc_preferred@cwmtopo62 with
xpvc_preferred@mmen.........[SAME]
Comparing Table xpvc@cwmtopo62 with
xpvc@mmen.............................[SAME]
Comparing Table xpvc_segment@cwmtopo62 with
xpvc_segment@mmen.............[SAME]
Comparing Table sct@cwmtopo62 with
sct@mmen...............................[SAME]
Comparing Table sct_cosb@cwmtopo62 with
sct_cosb@mmen.....................[SAME]
Comparing Table sct_vc@cwmtopo62 with
sct_vc@mmen.........................[SAME]
Comparing Table sct_usage@cwmtopo62 with
sct_usage@mmen...................[SAME]
Comparing Table conn_template@cwmtopo62 with
conn_template@mmen...........[SAME]
Comparing Table conn_templ_param@cwmtopo62 with
conn_templ_param@mmen.....[SAME]
Comparing Table user_conn_desc@cwmtopo62 with
user_conn_desc@mmen.........[SAME]
cwmtopo62-10->
