The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
Use this procedure to troubleshoot connectivity problems with Web Services (WS) Nodes when the hostname or IP address of the Application Node has changed.
Under normal circumstances, after changing the hostname and/or IP address of the Application Node, WS Node connectivity can be restored by running the set hcs cluster config command from the Application Node CLI. However, if this command fails to correct all connectivity issues, other commands are available to correct the problems.
It is strongly recommended to make a DRF backup of your cluster, once WS Node connectivity has been restored.
For information about Backup and Restore error messages, see Cisco Hosted Collaboration Mediation Fulfillment Maintain and Operate Guide, Release 10.6(1).
Step 1 | Issue the utils service list command to verify that the Cisco HCS IPA Service is running. |
Step 2 | From the Infrastructure Manager interface, select to review the Status Info and Recommended Action in the job results table. |
Step 3 | Review the Infrastructure Platform Automation log files located in activelog hcs/ipa. For more information, see Trace management on Cisco HCM-F platform and Table 1. |
Step 1 | Issue the utils service list command to verify that the following services are running: |
Step 2 | Issue the show hcs link auto-vm-linkage command to verify that the Cisco Hosted Collaboration Mediation Link Auto-VM-Linkage is enabled. |
Step 3 | Issue the utils diagnose hcs fulfillment command to view diagnostics for the Fulfillment service. |
Step 4 | Review the Fulfillment alarms located in activelog syslog/CiscoSyslog. See Alarms management on Cisco HCM-F platform. |
Step 5 | Review the Fulfillment log files located in activelog hcs/fulfillment. See Trace management on Cisco HCM-F platform and Table 1. |
Step 1 | Issue the utils service list command to verify that the following services are running: |
Step 2 | Issue the show hcs link auto-primecollab-linkage command to verify that the Cisco Hosted Collaboration Mediation Link Auto-PrimeCollab-Linkage is enabled. |
Step 3 | Issue the utils diagnose hcs dmasa command to view diagnostics for the DMA-SA service. |
Step 4 | Review the DMA-SA alarms located in activelog syslog/CiscoSyslog. See Alarms management on Cisco HCM-F platform. |
Step 5 | Review the DMA-SA log files located in activelog hcs/dmasa. See Trace management on Cisco HCM-F platform and Table 1. |
The following sections identify common HLM errors.
Symptom: Failed to change HCM-F global deployment mode value.
Resolution: Determine if there are Prime License Manager instances in the HCS space. Delete all Prime License Manager instances before you change the global deployment mode.
Symptom: Cannot create a Prime License Manager in the HCS space.
Symptom: Cannot delete a Prime License Manager instance from the HCS space.
Resolution: Determine if a Unified Manager cluster is still assigned to the Prime License Manager.
Symptom: The provisioned Unified Communications cluster doesn't appear in the eligible list for assigning to a Prime License Manager.
Resolution: Ensure that the Unified Communications cluster is either a Unified Communications Manager cluster or a Cisco Unity Connection cluster. Verify that the cluster version is 9.0 or above. Verify that the UC application is a publisher node in HCM-F.
Symptom: An eligible Unified Communications cluster assignment operation is not allowed.
Resolution: Verify that the number of clusters assigned to the target Prime License Manager does not exceed 1000.
Symptom: Assigning/Unassigning a cluster failed.
Symptom: Receiving a HLMAuditWarningAlarm.
Resolution: Verify that the cluster was added back to Prime License Manager by HLM if SDR reported an additional cluster was assigned to a Prime License Manager but Prime License Manager does not have it. Manually remove the cluster from Prime License Manager if Prime License Manager reported an additional cluster assigned to it but this cluster was not assigned by HLM. Verify that the assigned cluster's platform credential is the same as its credential in Prime License Manager if auditing reported a mismatch.
Symptom: Receiving a HLMChangeNotifAlarm.
Resolution: Review the HLM log to identify which component was affected. If a cluster was deleted, log into Prime License Manager(s) to ensure that the cluster is not assigned. Remove the cluster from Prime License Manager if it is in the inventory. If a customer was deleted, log in to Prime License Manager(s) to ensure none of this customer's clusters are still assigned. Remove the clusters from Prime License Manager if any of them appear in the inventory.
Symptom: Receiving a HLMClusterPLMIncompatibleAlarm after assigning a cluster.
Resolution: Log into Prime License Manager and perform a synchronization. Verify that the newly assigned cluster's license is compliant. If the license compliance fails, unassign the cluster then upgrade the cluster to the same version of Prime License Manager. Or assign the cluster to the Prime License Manager with same version.
Step 1 | Issue the utils service list command to verify that the following services are running: |
Step 2 | Issue the utils diagnose hcs sdrcnf command to view diagnostic information on the SDRCNF service. |
Step 3 | Review the SDRCNF alarms located in activelog syslog/CiscoSyslog. For more information, see Alarms management on Cisco HCM-F platform. |
Step 4 | Review the SDRCNF log files located in activelog hcs/sdrcnf. See Trace management on Cisco HCM-F platform and Table 1. |
This procedure is for troubleshooting synchronization problems between HCM-F and Cisco Unified Communications Domain Manager 8.1(x).
Step 1 | Issue the utils service list command to verify that the following services are running: |
Step 2 | Review the diagnostics information for the CUCDMSync service. |
Step 3 | From the Infrastructure Manager interface, select to run a manual CUCDMSync. |
Step 4 | From the Infrastructure Manager interface, select to view the job status table and detailed job information. |
Step 5 | Review the CUCDMSync alarms located in activelog syslog/CiscoSyslog. See Alarms management on Cisco HCM-F platform. |
Step 6 | Review the CUCDMSync log files located in activelog hcs/cucdmsync. See Trace management on Cisco HCM-F platform and Table 1. |
Step 7 | Use HCM-F RTMT to connect to the HCM-F and check the overall health of the system. Using HCM-F RTMT, you can browse, query, and collect trace and log files. You can also view alarms and enable remote real time tracing. |
The following section identifies some common CUCDMSync errors.
Symptom: Unable to perform manual or automatic sync request.
Resolution: Verify that CUCDM is configured correctly in HCM-F. Verify that webservice user is configured correctly on the CUCDM server.
Step 1 | Issue the utils service list command to verify that the following services are running: |
Step 2 | Review the diagnostics information for the UCSMSync service. |
Step 3 | From the Infrastructure Manager interface, select to run a manual UCSMSync. |
Step 4 | From the Infrastructure Manager interface, select to review the Status Info and Recommended Action in the job results table. |
Step 5 | Review the UCS Manager alarms located in activelog syslog/CiscoSyslog. See Alarms management on Cisco HCM-F platform. |
Step 6 | Review the UCSMSync log files located in activelog hcs/ucsmsync. See Trace management on Cisco HCM-F platform and Table 1. |
Step 7 | Use HCM-F RTMT to connect to the HCM-F and check the overall health of the system. Using HCM-F RTMT, you can browse, query, and collect trace and log files. You can also view alarms and enable remote real time tracing. |
Step 1 | Issue the utils service list command to verify that the following services are running: |
Step 2 | Review the diagnostics information for the VCenterSync service. |
Step 3 | From the Infrastructure Manager interface, select to run a manual VCenterSync. |
Step 4 | From the Infrastructure Manager interface, select to review the Status Info and Recommended Action in the job results table. |
Step 5 | Review the VCenterSync alarms located in activelog syslog/CiscoSyslog. See Alarms management on Cisco HCM-F platform. |
Step 6 | Review the VCenterSync log files located in activelog hcs/vcentersync. See Trace management on Cisco HCM-F platform and Table 1. |
Step 7 | Use HCM-F RTMT to connect to the HCM-F and check the overall health of the system. Using HCM-F RTMT, you can browse, query, and collect trace and log files. You can also view alarms and enable remote real time tracing. |
Step 1 | Issue the utils service list command to verify that the Cisco Platform Manager Service is running. |
Step 2 | Check for communication problems between the servers. Verify that the Platform SOAP Services or Platform Administrative Web Service is active on any of the Cisco Unified Presence, Cisco Unified Communications Manager, or Cisco Unity Connection servers you want Platform Manager to access. |
Step 3 | If you have an issue with an upgrade task, check the server type in the server group. For upgrade tasks, the publisher server and subscriber servers cannot be in the same group. |
Step 4 | Review the Platform Manager error messages. See Common Platform Manager errors. |
Step 5 | Review the Platform Manager log files. See: |
Ensure that the connections between all Unified Communications applications and Platform Manager are established before you run a task. All Unified Communications applications must be contacted at least once before you run the first Platform Manager task. SFTP or FTP and Unified Communications applications must be routable from Platform Manager; for example, if NATs are used, use public IP. Likewise, the SFTP or FTP servers must be routable from the Unified Communications applications.
A backup task cannot be edited after it is created; however, you can edit a backup schedule task.
Backup schedule tasks can be cancelled while in process. Any servers in the group, however, that have started backing up when you cancel the task do not stop mid-task. These servers are still backed up.
If a backup schedule task fails, review the DRS logs.
"Skipped" servers have a first node server. After the first node server is backed up, all other servers in the group are skipped. Backing up the first node server ensures that the other servers in the group are also backed up. These servers appear as "Skipped" in the Backup Task List. A server also appears Skipped if it has a pre-9.x release version. Backup Scheduler can only backup servers with software versions higher than 9.0.
When you troubleshoot issues for Platform Manager, you can access the following log files on the Cisco HCM-F platform at the following locations:
file get activelog tomcat/logs/platform-api/log4j/*—This log file includes information generated by the Unified Communications applications. The same log file is also stored on the application server. Using this command, you can view:
file get activelog tomcat/logs/pm/log4j/*—This log file includes information from Platform Manager. Using this command, you can view:
REST traffic between the browser and Platform Manager including inputs, results, errors, and messages
SOAP messages to the Unified Communications applications including inputs, results, errors, and messages
Database access including updates, queries, and results
Task-related events like scheduling, starting, and updating tasks
Background jobs like synchronization tasks
Step 1 | Issue the utils service list command to verify that the Service Inventory service is running. Depending on the reports requested, verify that the following services are also running: |
Step 2 | Check the status of the Service Inventory jobs by selecting Only one report can run at a time. A report that is currently running will show Status as "In Progress". Queued reports will show Status as "Requested". from the Infrastructure Manager menu. Service Inventory Report jobs who the Entity Name as "ONDEMAND" and "DAILY_SCHEDULE". |
Step 3 | If you believe a job is stuck, you can remove the job by selecting in it on the Jobs page and clicking Delete Selected. Restarting the HCS Service Inventory service from the CLI will also fail any in progress jobs and cancel any requested jobs. |
Step 4 | Enabling aggregate reports by selecting Each reseller and customer when scheduling a report may slow down the report processing and require additional resources, so check this only when required. Enabling aggregate reports will create additional report files for each reseller and customer apart from the regular .SI file, and will also create a Provider Level Summary file. All these reports will be pushed the SFTP and Backup SFTP, if those have been configured under Service Inventory. |
Step 5 | If you have Cisco Unified Communications Domain Manager 8.1(x) configured, select CUCDM for the Report Source when setting up a scheduled or on demand report. If Cisco Unified Communications Domain Manager 8.1(x) is not configured and UC Applications are configured in Infrastructure Management, then select UCAPP for the Report Source. If the Report Source is CUCDM, be sure to select 8.6.2 or 9.0.1 for the Report Format Version. If the Report Source is UCAPP, be sure to select 9.1.1 for the Report Format Version. |
Step 6 | To generate a Location report, at least one SI report has to have been previously generated. |
Step 7 | Review the Cisco HCS Service Inventory alarms located in activelog syslog/CiscoSyslog. See Alarms management on Cisco HCM-F platform. |
Step 8 | Review the
Cisco HCS Service Inventory log files located in:
See Trace management on Cisco HCM-F platform and Table 1. |
The following section contains common Service Inventory errors.
Service Inventory job scheduling relies on the service provider name for Cisco Unified Communications Domain Manager, which appears in the Infrastructure Manager administrative interface (Service Provider > Administration) and exists in the Shared Data Repository.
If Cisco Unified Communications Domain Manager 10.6(1) is used, the Service Provider is automatically pushed to HCM-F.
If Cisco Unified Communications Domain Manager 8.1(x) is used, the Service Provider is retrieved by using the Cisco HCS CUCDMSync service (strongly recommended).
If Cisco Unified Communications Domain Manager 10.6(1) is not used, the Service Provider can be manually configured in Infrastructure Manager.
Tip | For deployments using Cisco Unified Communications Domain Manager 8.1(x), use the Cisco HCS CUCDMSync service to synchronize data configuration between Cisco Unified Communications Domain Manager and Cisco HCM-F. If you use the Cisco HCS CUCDMSync service, Service Inventory requires that at least one synchronization has occurred before a report is generated. If necessary, perform a manual synchronization after you verify that the Cisco HCS CUCDMSync service is running. Be aware that if you manually configure the service provider in HCM-F and then activate the Cisco HCS CUCDMSync service for Cisco Unified Communications Domain Manager 8.1(x), or configure the service provider on Cisco Unified Communications Domain Manager 10.6(1), the information that you manually configured is overwritten with the information that comes directly from Cisco Unified Communications Domain Manager. |
Perform the following tasks, depending on your configuration:
Verify that the service provider name appears correctly in the Infrastructure Manager administrative interface. In addition, open Cisco Unified Communications Domain Manager and verify that the service provider name for Cisco Unified Communications Domain Manager is correct.
If the name is correct in Cisco Unified Communications Domain Manager 8.1(x) and you use the Cisco HCS CUCDMSync service, perform troubleshooting tasks for the Cisco HCS CUCDMSync service, as described in the Troubleshooting Cisco HCS CUCDMSync Service.
If the name is correct in Cisco Unified Communications Domain Manager 8.1(x), perform troubleshooting tasks for the Cisco HCS NBI REST SDR Web Service.
If you do not use Cisco Unified Communications Domain Manager 8.1(x) or Cisco Unified Communications Domain Manager 10.6(1), update the configuration in the HCM-F Infrastructure Manager interface. To manually configure the Service Provider name in the Infrastructure Manager interface, select .
The Service Inventory service encountered a runtime error and cannot validate the request from Cisco Unified Communications Domain Manager; therefore, the Service Inventory service cannot receive the files from Cisco Unified Communications Domain Manager.
Verify that you configured the username and password correctly for Cisco Unified Communications Domain Manager in the Infrastructure Manager administrative interface.
The Service Inventory service encountered a runtime error and cannot connect to the Cisco Unified Communications Domain Manager; therefore, the Service Inventory service cannot receive the files from Cisco Unified Communications Domain Manager.
Verify that Cisco Unified Communications Domain Manager is up and running.
In the Infrastructure Manager administrative interface, verify that you correctly configured the SOAP port for Cisco Unified Communications Domain Manager.
In the Infrastructure Manager administrative interface, verify that you correctly configured the hostname and IP address for Cisco Unified Communications Domain Manager.
Service Inventory did not receive the files from Cisco Unified Communications Domain Manager after one hour passed.
Log in to Cisco Unified Communications Domain Manager. Select General
. Review the transactions to determine if the transactions failed.For example, the following message may appear in the transaction: "Unhandled exception encountered while performing request processing. Report generation failure. FTP failure. Is there a file of the same name on the FTP server?"
In the Service Inventory administrative interface, verify that you configured the SFTP port correctly on the Configuration page.
For example, the following message may appear in the transaction: "Unhandled exception encountered while performing request processing. Report generation failure. FTP failure. Authentication failed."
On the Configuration page in the Service Inventory administrative interface, correct the user credentials for Cisco Unified Communications Domain Manager SFTP access to Service Inventory.
For example, the following message may appear in the transaction: "Unhandled exception encountered while performing request processing. Report generation failure. FTP failure. Is there a file of the same name on the FTP server? [Errno -2] Name or service not known."
Ensure that the Cisco HCM-F platform network connection works as expected.
If Cisco Unified Communications Domain Manager has DNS enabled, run the nslookup command on Cisco Unified Communications Domain Manager to ensure that the Service Inventory hostname resolves.
If Cisco Unified Communications Domain Manager does not use DNS, verify that you configured the IP address for the Service Inventory server on the Configuration page in the Service Inventory administrative interface.
Service Inventory received the file from Cisco Unified Communications Domain Manager, but the file is empty or does not contain all data.
In the Infrastructure Manager administrative interface, verify that the Provider configuration matches the configuration that is in Cisco Unified Communications Domain Manager.
The remote SFTP server did not receive the report that Service Inventory transferred.
On the Configuration page in the Service Inventory administrative interface, verify that the remote SFTP server configuration is correct.
If the an aggregate report is required, but Service Inventory is not generating it, check the report settings in Service Inventory. Verify that the Each reseller and customer checkbox is checked.