This document describes the troubleshooting method for the problem identification of Unified Contact Center Enterprise (UCCE) Peripheral Gateway (PG) remaining in isolated enabled state, especially after a fresh PG installation or the upgrade attempt. Typical causes of this problem are related to the incorrect PG port numbers assigned by the UCCE application installer during installation and upgrade run. This document also summarizes a list of ports used for maintaining PG duplex operations.
For duplex PG pairs, when you run OPCTEST status command on either side of the PG, OPC status is visible only for the one side of PGs, and the other side is not reachable. As if each PG server of the PG pairs is running under simplex mode on its own.
Multilayer Data Switch (MDS) process log indicates MDS initiates registration process of all its clients, and duplex operation is requested but eventually failed. MDS goes into simplex operation.
Open Peripheral Controller (OPC) process log indicates the pairing side of OPC went offline due to MDS problem.
PGAG process logs also indicate TOS events, and eventually receiving reports from MDS process that the other side is not reachable, MDS in simplex mode operations etc.
Since these are catastrophic failures, these errors are obvious and readily available even with default traces turned up on PG services,
Troubleshoot steps first involves some basic network checks:
Checking IP addresses configuration on Pub and Private NICs
Window route table inspections
However, there are cases where the problem persists once all the steps are reviewed. Then our topic starts at this point onwards.
Note: All ports as referenced in this article are TCP ports.
Before the installer lays out the port range schema for a PG, it has to first inspect existing PG installation on the server to ensure that there are no port range conflicts after the installation. In some cases, especially after an upgrade, migration, and re-installation of PG servers, assigned ports could be mismatched between a pairing PG servers. The mismatches are likely caused by the existing ICM installations or the remanence of it on the servers, and coverage installations are performed on top of existing installation.
Important system numbers
PG port ranges are assigned by ICM installer during installation time, and the values are based on certain factors, Enterprise Instance Number is one of such factor and the first thing to verify.
Enterprise Instance Number
Inherited from old Geotel Intelligent Call Management (ICM) software architecture, instance number is used to differentiate the software contexts. The default value is 0. It is usually determined during installation time. if the Instance Number is not consistently maintained across the entire enterprise setup of the software, port number mismatches will definitely occur.
port number (instance n) = port number (instance 0) + 40 * n.
PG DMP system id
PG DMP system id determines the designation of a PG pair in the UCCE Call Router, misconfiguration of this value will cause conflict in terms of communication failure between Call Routers and PGs. Different values if are configured across a PG pair by mistake will not allow PG pair to form duplex obviously.
By the design, there could only be 2 PG nodes on each host server. If the installer found an existing PG installation on the host server, it tries to use different sets of ports for the new PG.
Regardless the PG DMP or system id of the PG, Installer installs 43XXX range of these ports to the first pg installation on a host server. The 2nd PG node installed on the same host server uses the 45XXX port range. ie. if PG2 was installed before PG1 on the host server, the relevant ports associated to PG2 nodes take up 43XXX port range, whereas PG1 is given 45XXX port range.
This is where port mismatch could occur even with the order of PG installation.
For instance 0 and 1st PG, the StateXferPort value is 43023 on both sides of PG For instance 0 and 2nd PG, the StateXferPort value is 45023 on both sides of PG
For PG duplex operation, state transfer issues, if the problems occur after a major upgrade, migration and coverage installation of PG servers, review above-mentioned ports and numbers, see if one of them are mismatched across both sides of PG pairs.