Troubleshoot Common Issues Seen in a CCE Environment when Integrating with Webex Connect

Introduction

This document describes how to troubleshoot common issue scenarios seen in a Contact Center Enterprise (CCE) solution integrated with Webex Connect.

Prerequisites

Requirements:

Cisco recommends that you have knowledge of these topics:

• Cisco Packaged Contact Center Enterprise (PCCE)
• Cisco Unified Contact Center Enterprise (UCCE)
• Webex Connect

Component Used:

The information in this document is based on these software and hardware versions:

• CCE Version: 12.6(2)
• Finesse Version: 12.6(2)
• Webex Control Hub
• Webex Connect Portal
• Webex Engage Portal

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Note: Agents handling digital channel task must be Single Sign-On (SSO) enabled.

Background

Webex Connect allows the Contact Center business and its customers to interact using digital channels such as email, chat, and SMS. The Contact Center Enterprise (CCE) solution integrates with Webex Connect to create a seamless omnichannel experience for customers. This integration helps customers to interact across voice and digital channels of communication. Webex Connect offers a rich self-service and bot integration to empower customers to get answers to some common questions. It provides a unified solution for integrated routing, Agent Desktop, and reporting service. Webex Connect provides a simplified framework that helps partners and customers interact through digital channels.

How to access and collect logs across various components

Webex Connect Portal - Flows

Note: Ensure you have descriptive logging enabled for the flow before initiating an activity.

Enable Descriptive Logs for your specific flow

• Select the gear icon at the top of the page to open the Flow Settings
• In the General tab, enable Descriptive Logs
• Enter 1000 in the 'Enabled for' filed
• Click Save.

Debugger Logs

Step 1: Log into your Connect Portal and navigate to Services and click on appropriate service.

Step 2: Next, click on 'Flows' and then select the appropriate flow that would like to debug:

Step 3: Now, within the select flow:

1. Click on the debugger icon to the right. 
2. Next, click on ‘Decrypt Logs’.
3. Find the timestamp of your most recently initiated Chat and click on the corresponding ‘Transaction ID’.

Step 4: Click on each of the node to view the details of the events that transpired on each of the node.

Cloud Connect Server

Chat sessions can be tracked in the digitalrouting.log file on the Cloud Connect Server using the TransactionID fetched from the Debugger.

• Log File Name: digitalrouting.log
• Via CLI: file get activelog /hybrid/log/digitalrouting/digitalrouting.log
• Via browser: https://<cloud-connect-server>:8445/drapi/log/digitalrouting

Right click on the log filename and click 'Save Link As'

Agent sync issues can be tracked in the dataconn.log file.

• Log File Name: dataconn.log
• Via CLI: file get activelog /hybrid/log/dataconn/dataconn.log
• Via browser: https://<cloud-connect-server>:8445/drapi/common/log/directory/taos-log-a_hybrid_log_dataconn

Right click on the log filename and click 'Save Link As'

Browser Console Logs

The 'Manage Digital Channel' gadget on the Finesse desktop is hosted on Webex Engage. Hence, the communication is between the browser where Finesse desktop is loaded, and Webex Engage.

In order to troubleshoot gadget issues, browser console logs are needed.

Note: For Manage Digital Channel gadget issues, Finesse Desktop WebServices logs is not be required.

From a Chrome browser

• Click on the Settings icon and navigate to Developer tools > More tools > Developer tools.
• On Developer tools > Console tab, click on the gear symbol and check the options: Preserve log, Show timestamps, Log XMLHttpRequests.
• On Developer tools > Network tab, click on the gear symbol and check the option: Preserve log.
• Close the settings page.
• Now load the URL in the browser.
• The console logs can now be collected by right-clicking on the 'Console’ tab and select Save all as.

From a Firefox browser

• Click on the Applications menu icon and navigate to More tools > Web Developer tools.
• In Network tab, click on the gear symbol and select the option: Persist Logs.
• The console and network logs can now be collected by right-clicking on the respective tabs and select Save all as.

Chrome                                                                                    Firefox

ICM Logs

The activity can be tracked in the rtr and the mr-pim logs using the TransactionID.

• Router logs (rtr)
• MR-PIM logs (mr-pim)

Router Logs

MR-PIM Logs

Common Troubleshooting Scenarios

Scenario 1: Digital Channel Settings on CCE Admin page fails to load with error - Error connecting to Digital Routing Service.

"Error connecting to Digital Routing Service. Make sure it is up and security certificate is valid."

This error is seen in either of these scenarios:

• MR PIM for Digital Routing is not Active.
• Digital Routing service is not running on the Cloud Connect servers.

Use these command to ensure the digitalrouting container is up and running:

• utils cloudconnect list
• show cloudconnect digitalrouting status

Use these commands to stop and start the digitalrouting service

• utils cloudconnect stop digitalrouting
• utils cloudconnect start sigitalrouting

Scenario 2: Chat initiation fails with error - The specified Domain is not valid.

Upon checking the 'Decrypt Logs' on the Webex Connect portal, it can be noticed that the chat fails in the 'Create Conversation' step.

The logs show 'The specified Domain is not valid' error

• {"code":4614,"tenantId":"0ca3e4dc-xxxx-aaaa-1234-f8ef834c84f5","context":null,"description":"The specified Domain is not valid.","event":"conversation:error","value":null,"tid":"ca02cf2f-a87a-4a83-866a-88ccf76d74ce","status":"failure","errorDetails":["The specified Domain is not valid."]}

This indicates a configuration issue; therefore, verify the domain configuration in these 2 places:

1. On Webex Connect Portal, navigate to Service:

• And select the correct Service >> Flows >> Edit flow >> select the gear symbol >> 'Custom Variables' tab >> 'liveChatDomain' - enter the appropriate domain details >> Save

Note:

Do not edit the 'domain' field by mistake and ensure this is left as is - livechat.

After making the change, click on Save >> Make Live >> in the prompt ensure the correct Application is select in the drop down >> Make Live.

b. Log into Webex Control Hub:

• And navigate to Contact Center >> Digital >> Select Webex Engage link >> and now in the Webex Engage portal >> select Assests >> select and edit your LiveChat >> select Websites tab >> Edit >> in the general tab ensure the domain is set correct in the 'Domain' field >> Save changes.

Scenario 3: Chat initiation fails in the CCE Create Task node with error - The specified Domain is not valid.

In Webex Connect portal, decrypt the log and click on the node where the chat fails. In this example, the chat fails in the 'CCE Create Task'.

The logs show a missing or incorrect domain configuration:

• {"code":4614,"tenantId":"cb7f4158-bd62-4a46-a1ae-d2ac704d2144","context":null,"description":"The specified Domain is not valid.","event":"conversation:error","value":null,"tid":"eb7f4658-bc62-4a26-a1ae-d1ad704d2414","status":"failure","errorDetails":["The specified Domain is not valid."]}
• "status : 4002 , desc : value is mandatory , name : Domain [ id : cb7f4158-bd62-4a46-a1ae-d2ac704d2144"

To resolve this:

• On Webex Connect Portal, navigate to Service >> select the correct Service >> Flows >> Edit flow >> select Edit >> double click on 'CCE Create Task' node
• In the Domain field, enter the entry point for Webex Connect to be able to send the route request to Cloud Connect which in this case is a Reverse Proxy
• Entered the Reverse Proxy for the deployment >> Save >> Make Live >> select correct application >> Make Live.

Scenario 4: Manage Digital Channels gadget on Finesse fails to load and login (loads a blank gadget).

• Agents handling digital channel task must be Single Sign-On (SSO) enabled. If they are not SSO enabled, the Manage Digital Channel loads blank.
• If the agent is SSO enabled and the gadget loads blank, collect the browser console logs to troubleshoot.

Scenario 5: Manage Digital Channels on Finesse fails to login with error - Failed to load Manage Digital Channels. Unable to get the gadget sources.

Console logs show: "Digital channel configuration does not have orgID configured"

This error can be seen if the cloud connect configuration is missing under Cisco Finesse Administration page >> Cloud Connect Settings

• Ensure the correct Cloud Connect Server details are entered.

Once the Cloud Connect server setting is verified, check the URL to ensure you see an OrgID:

• https://<finesse>/finesse/api/DigitalChannels/Configuration

Example from a working lab

If you do not see an orgID listed, either:

• The containers on Cloud Connect are not running (utils cloudconnect list)

OR

• Port 8445 between the Finesse and Cloud Connect servers is blocked.

If there is a communication issue between Finesse and CloudConnect, the Cisco Finesse Desktop WebService log file shows the below:

{"seq":732,"level":"ERROR","timestamp":"Dec 12 2024 14:50:04.069 -0500","thread-name":"FINESSE-STATE-CHANGE-THREAD-CLOUD_CONNECT_SETTINGS_POLLER","message":"CloudConnectCommunicator|Failed GET on the publisher for the url: https://<cloud-connect>:8445/cloudconnectmgmt/orgsettings"}

Verify connectivity using these CLI commands:

• From Finesse: utils network connectivity <cloud-connect-fqdn> 8445
• From CloudConnect: utils network connectivity <finesse-fqdn> 8445

Scenario 6: Manage Digital Channels on Finesse fails to login with error - Sign in failed. Invalid token signature.

The browser console logs show an onConversationLoadedError '9005' indicate an issue with the IdS certificate present in the Webex Control Hub.

bundle.js:1 {code: '9005', message: 'could not verify the authenticity of the source app'}
useMediaSetup.js:156 onConversationLoadedError {code: '9005', message: 'could not verify the authenticity of the source app', origin: 'https://<finesse-fqdn>:8445'}
useMediaSetup.js:216 :: useMediaSetup- handleOnLoadError Received errorCode: 9005
useMediaSetup.js:156 onConversationLoadedError could not verify the authenticity of the source app
useMediaSetup.js:216 :: useMediaSetup- handleOnLoadError Received errorCode: undefined

To resolve this:

• Verify that no change has been done to the token csr generated on IdS (show ids token csr).
• Re-upload the signed certificated (generated from the csr) to webex control hub (Control Hub >> Contact Center >> Digital tab)
• Clear browser cache followed by a restart of the browser and re-test.
• Login must now be successful.

If there were any changes made on the IdS side to the ids token csr or if the certificate was corrupted, then the steps from 'Regenerate the public key certificate using Cisco IdS' section need to be followed.

• Note: This can only done during non-production hours as it involves a restart of the IdS server and also requires the agents to log out and log back in.

Scenario 7: Manage Digital Channels on Finesse fails to login with error - Sign in failed. User not found. Please retry the login operation.

The browser console logs show an onConversationLoadedError '9010' indicating that the user does not exist in Engage for that tenant (orgID).

onConversationLoadedError Object
useMediaSetup- handleOnLoadError Received errorCode: 9010
onConversationLoadedError User is unauthorized to access this tenant
useMediaSetup- handleOnLoadError Received errorCode: undefined

Verify:

• The user profile on Webex Engage is active
• The user profile is on the correct tenant (orgID).

Scenario 8: Manage Digital Channels on Finesse fails to login with error - Media channels not available. You are not configured to use Media Channels.

Ensure:

• Agent is configured with correct Webex Connect MRD based Skill Group.
• Agent is configured with and Attribute that enables the user to be part of the Precision Queue (PQ) from the Webex Connect MRD.
• Media Class and MRD are configured correctly.

If this error is seen for agents only configured to use Webex Connect MRD based PQ, this is due to defect - Cisco bug ID CSCwk50394 . .

• Console logs show: useMediaSetup.js:242 No MRDs configured for this user.
• Solution for Cisco bug ID CSCwk50394 : Add the agent to a Skill Group in the Webex Connect specific MRD.

Scenario 9: Accept Chat fails with - Failed to load data. System is unable to load the task due to missing information.

This indicates the Manage Digital Channels gadget has not received the ECC Variable values which is needed to load the chat session.

Browser console logs show: <finesse-fqdn>:8445/desktop/undefined/api/app-config:1 Failed to load resource: the server responded with a status of 404 ()
Decrypt Logs from the Live Chat Inbound Flow show:

{"statusCode":403,"m_strResponseBody":"{\"errors\":[{\"error\":{\"id\":\"20283\",\"status\":403,\"message\":\"Mandatory ECC Variables are not registered.\",\"detail\":\"The variable user_DR_MediaResourceID should be configured for the Digital Routing Service as a pre-requisite.\"}}],\"trackingId\":\"WebexConnect_11f4zzfe-6b45-45c0-83bb-44cc8d0f3ee0\"}","headers":{"Transfer-Encoding":["chunked"],"X-Content-Type-Options":["nosniff"],"Connection":["keep-alive"],"Pragma":["no-cache"],"TrackingId":["WebexConnect_11f4bbfe-6b45-45c0-83bb-44aa8d0f3ee0"],"Date":["Tue, 16 Jul 2024 12:47:38 GMT"],"X-Frame-Options":["DENY"],"Cache-Control":["no-cache, no-store, max-age=0, must-revalidate"],"Content-Security-Policy":["default-src 'none'; frame-ancestors 'self'; frame-src 'none'; script-src 'self'; connect-src 'self'; font-src 'self'; img-src 'self'; style-src 'self'; object-src 'none'; form-action 'self'; block-all-mixed-content; base-uri 'self'"],"Expires":["0"],"X-XSS-Protection":["1; mode=block"],"Content-Language":["en-us"],"Content-Type":["application/json;charset=utf-8"]},"statusText":""}

To resolve this error, ensure:

• The mandatory DR ECC Variables are configured correctly under the Digital Channel Settings configuration gadget on CCE Administration page.
• The mandatory DR ECC Variables are spelled correctly.
• The mandatory DR ECC Variables are added as members in the Expanded Call Variable Payload List configuration under 'List Tools' in the Configuration Manager.

Note: The mandatory ECC variable are case sensitive.

Mandatory ECC Variables:

• user_DR_Primary
• user_DR_Backup
• user_DR_MediaResourceID
• user_DR_CustomerName
• user_DR_MediaChannelName

For more information on the mandatory ECC variables, refer the ECC Variables for Digital Routing Tasks section in the Digital Channels Integration Using Webex Connect chapter of the UCCE 12.6.2 Features Guide.

Scenario 10: Chat fails to load on Manage Digital Channels with error - Failed to load data - No participant(s) added to this conversation.

image

Browser console logs show ""Dispatching event: onConversationLoadedError with payload: {"error":{"status":false,"code":"9015","message":"No participant(s) added to this conversation","result":null},"conversationid":"CSRFZTKAJYIYK973"} bundle.js:1:117180 ""

To resolve this:

• On the Webex Connect portal, check to see if any of the Agnostic Flows are being invoked or not (Created >> Queued >> Routed).
• If not, check the execution conditions for each of the agnostic flows and delete/correct them as required.
• If needed, save and 'Make Live' each of the Agnostic Flows with the correct Application to trigger the agnostic flows.

Track Digital Routing specific ECC Variables across MR PG, Router and Browser Console logs.

For logs, refer the document Track Task Routing Events in a CCE Environment Integrated with Webex Connect

Enable ECC Variable Tracing

ECC Variable tracing can be enabled on the components listed below, in order to track the ECC variables across all the components:

• Router
• OPC on MR PG
• CTISVR

For instructions and commands to enable and disable the ECC variable tracing, refer this document - Troubleshoot Call and ECC Variables Issue.

Here is an example of the commands used to enable and disable ECC variable tracing on the Router (rtr) component.

Related Information

Features Guide - Digital Channels Integration Using Webex Connect:

• UCCE: https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cust_contact/contact_center/icm_enterprise/icm_enterprise_12_6_2/configuration/guide/ucce_b_features-guide-1262/rcct_m_digital-channels-integration-webexconnect_12-6-2.html
• PCCE: https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cust_contact/contact_center/pcce/pcce_12_6_2/maintenance/guide/pcce_b_features-guide-1262/rcct_m_digital-channels-integration-webexconnect_12-6-2.html

Cisco Contact Center Enterprise Manage Digital Channels Gadget User Guide:

• https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cust_contact/contact_center/finesse/finesse_1262/user/guide/rcct_b_12_6_2_manage-digital-channels/rcct_m_1262_manage-digital-channels.html

Administration and Setup Guide for Webex Engage with Cisco Contact Center Enterprise:

• https://www.cisco.com/c/dam/en/us/td/docs/voice_ip_comm/cust_contact/contact_center/icm_enterprise/wxengage_for_digital_channels/admin/guide/Administration_and_Setup_Guide_for_Webex_Engage_with_CCE.pdf