Troubleshoot Call and ECC Variables Issue

Introduction

This document describes how to troubleshoot on-call and Expanded Call Context (ECC) variables in Peripheral Gateway (PG).

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Cisco Unified Contact Center Enterprise (UCCE)
• Cisco Package Contact Center Enterprise (PCCE)
• Cisco Unified Communications Manager (CUCM)

Components Used

The information in this document is based on Unified Contact Center Enterprise Version 12.6(1).

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

The JTAPI Gateway (JGW) process is the one responsible for initializing communications with the CUCM through JTAPI. 3rd also uses CTI to establish a connection with Agent PG. The Call and ECC variables are not enabled as a part of default trace settings.

Overview

that's one moible agent make call to self-mobile phone. the call with ECC but the ECC can not be checked in the TCD table. in this example, we use define ECC paramter in the Variable 4.

High-Level Call Flow

Troubleshoot 

1. These Variables can be Enabled for the Peripheral Gateway and CTI Server Components

OPC: The commands are set using the OPC Test Utility:

C:\>opctest /cust ins1 /node PG1A
21:27:53 Trace: EMT Creating Mutex Global\IMTConnect_DisconnectLock

To enable ECC and Peripheral variables tracing in OPC, enable the trace levels in OPCTest debug command.

opctest: debug /ecc_variables /periph_variables

Use  debug /on  to Restore Default Tracing Levels. To disable the ECC and Peripheral variables tracing in OPC, use one of the commands to disable ECC and Peripheral variable tracing. opctest: debug /on (this sets the trace level back to default)

opctest: debug /on
Use "debug /on" to Restore Default Tracing Levels.
Restoring Default Tracing Levels.

OPC Current Time: 02/27 05:32:22
OPC Local Time: 02/26 21:32:22 (8.0 hr)

OPC Current Tracing Levels

DebugControlFlag1 (0x0):
DebugControlFlag2 (0x10000000): /default

Opctest: debug /noecc_variables (this disables the ECC variable tracing).

Opctest: debug /noperiph_variables (this disables Peripheral variable tracing).

opctest: debug /noecc_variables
Use "debug /on" to Restore Default Tracing Levels.

opctest: debug /noperiph_variables
Use "debug /on" to Restore Default Tracing Levels.

2. CTI Server: The Registry Settings are Modified

1. EMSTracemask=0x04 - Enables tracing of Call variables

2. EMSTracemask=0x02 - Enables tracing of ECC variables

EMSTraceMask

3. Procmon: CallManager PIM and VRU PIM

The commands are used over the procmon utility.

C:\>procmon ins1 PG1A pim1
Trace: EMT Creating Mutex Global\IMTConnect_DisconnectLock
Trace: WARNING!!! - Using Procmon during production hours may have performance impact and unexpected service interruption.

>>>>trace ecc_variables /on - Enables ECC call variables tracing
>>>>trace ecc_variables /off - Disables peripheral call variables tracing

>>>>trace periph_variables /on  - Enables peripheral call variables tracing
>>>>trace periph_variables /off - Disables peripheral call variables tracing

4. RTT Test

Enable: debug /noperiph_variables, debug /noecc_variables.

C:\>rttest /cust ins1 /node RouterA
rttest: debug /periph_variables/ecc_variables
debug_control: Invalid control argument: /periph_variables/ecc_variables
rttest: debug /ecc_variables /periph_variables

Disable: debug /noecc_variables /noperiph_variables

rttest: debug /noecc_variables /noperiph_variables

Log Analyze

1. Non-working Log

PG OPC: Based on the TCD test timestamp and calling phone number 8110343 to find CallingDevice=8110343 and InvokeID=0xf01c6bef.

16:09:14:515 PG4A-opc Trace: TPServices::AddInvocationByCTIInvokeID - CTIInvokeID 0x26208ab8 mapped to OPCInvokeID 0xf01c6bef 
16:09:14:515 PG4A-opc Trace: CSTAMakeCall: TPServices::AddInvocationByCTIInvokeID InvokeID=0xf01c6bef CallingDevice=8110343 CalledDevice=9210501775255427560213183335003 
16:09:14:515 PG4A-opc Trace: PRIVATE_DATA: CallPlacement=0 CallManner=0 ConsultType=0 AlertRings=0 CallOption=0 FacilityType=0 FacilityCode= AuthorizationCode= AccountCode= AnsMachTreat=0 CCT= PV1= PV2=2 PV3= PV4=60213183335003 PV5= PV6= PV7=1560000001038270316 PV8= PV9=22|17752554275|013183335003 PV10=WWSDM204|M|S|COR|| UserToUser= Wrapup= 
16:09:14:515 PG4A-opc Trace: Name Type Index Value 
16:09:14:515 PG4A-opc Trace: user.microapp.FromExtVXML Array 0 013183335003 

3 seconds later, filter by 8110343 to find CID 18603484. tried to send CTI_CALL_DATA_UPDATE_EVENT. But the connection failed. Due to there is no CSTAMakeCallConfEvent. So call PeripheralVar can not be copied.

16:09:17:127 PG4A-opc Trace: CSTA_SERVICE_INITIATED(PID=5006): InitiatedConn=(CID=18603484,Dev=8110343,Type=Static) localConnInfo(1)=CS_INITIATE cause=-1 
16:09:17:127 PG4A-opc Trace: AssignAgentCall: PID=5006 AGID=28946 Periph#=13212933 LineHandle=0 LineType=2 Conn(CID=18603484,Dev=8110343,DevType=Static) SG#=29747 shadow=0 CTI=1 AgentObj=0x581f928 
16:09:17:127 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603484 CallObj=(0x615bb50) NewState=NULL OldState=NONE. 
16:09:17:127 PG4A-opc Trace: CTI_CALL_CREATED_EVENT: CID=18603484 Src:8110343 Dest: RCK=(0-0) 
16:09:17:129 PG4A-opc Trace: CSTA_ORIGINATED(PID=5006): OrigConn=(CID=18603484,Dev=8110343,Type=Static) CallingDev=(DEV)8110343 CalledDev=(DEV)500992160213183335003 localConnInfo(3)=CS_CONNECT cause=-1 
16:09:17:129 PG4A-opc Trace: CTI_CALL_DATA_UPDATE_EVENT: CID=18603484 src=8110343 Dest= RCK=(0-0) 
16:09:32:595 PG4A-opc Trace: CSTA_NETWORK_REACHED(PID=5006): Conn=(CID=18603484,Dev=500992160213183335003,Type=Dynamic) TrunkUsed=(TRUNK)-1 CalledDev=(TRUNK)-1 localConnInfo(-1)=CS_NONE cause=-1 
16:09:32:595 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603484 CallObj=(0x615bb50) NewState=NULL OldState=NULL. 
16:09:32:595 PG4A-opc Trace: ReclassifyLineInfo: (PID=5006) LineHandle=0 LineType=LT_OUTBOUND(SRC) SrcConn(CID=18603484,Dev=8110343,DevType=Static) LineHandle=0 LineType=LT_INSIDE DestConn(CID=-1,Dev=,DevType=Unknown) LineHandle=-1 LineType=LT_UNKNOWN CallObj=0x0615BB50 
16:09:32:595 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603484 CallObj=(0x615bb50) NewState=NULL OldState=NULL. 
16:09:32:595 PG4A-opc Trace: CTI_CALL_DATA_UPDATE_EVENT: CID=18603484 src=8110343 Dest= RCK=(0-0) 
16:10:19:195 PG4A-opc Trace: CSTA_CONNECTION_CLEARED (PID=5006): DroppedConn=(CID=18603484,Dev=8110343,Type=Static) ReleasingDev=(DEV)8110343 localConnInfo(-1)=CS_NONE cause=-1 
16:10:19:195 PG4A-opc Trace: SendCTICallDataUpdateEvent: CID=18603484 (Src:8110343 Dest:) NotifyCTIServer:1 CallContextFlags:0 
16:10:19:195 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603484 CallObj=(0x615bb50) NewState=NONE OldState=NULL. 
16:10:19:195 PG4A-opc Trace: Call::RemoveOfferedSkillGroups - !INFORM! - CallID:18603484 on PID:5006 is REMOVING offered skill groups. Forced=N 
16:10:19:195 PG4A-opc Trace: Cleared(PID=5006): CallObj=(0x0615BB50) SrcConn(CID=18603484,Dev=8110343,DevType=Static) DestConn(CID=-1,Dev=,DevType=Unknown) 
16:10:19:196 PG4A-opc Trace: TerminateCall: Inst=8110343 Line[0,OUT]:TLK->IDL Conn=(CID=18603484,DevID=8110343,DevType=Static) AgentLineObj=0x4294fc0 
16:10:19:196 PG4A-opc Trace: TerminateCall: AGSTID=28946 PeriphNo=13212933 SG=29747 LineType=OUT Talk=62 Wrap=0 Hold=0 Conn=(CID=18603484,DevID=8110343,DevType=Static) AgentObj=0x581f928 CallObj=0x615bb50 
16:10:19:196 PG4A-opc Trace: BuildClosedCallRecord(PID=5006): CallObj=0x615bb50 DateTime=10/25 08:10:18 IntervalDatetime=10/25/2022 08:00:00 MRDomainID=1 CID=18603484 RCK(RtrDate=0,RtrCID=0) RCKSeq#=0 CD=10(DBCD_DROP_NO_ANSWER) CDF=4(DBCDF_ERROR) PeriphCallType=9,DBCD_OUT AppTaskCD=0 ICRCK=1342458107, ICRCKParent=-1 ICRCKChild=-1 NetworkTime=0 Duration=62 RingTime=0 DelayTime=62 TimeToAband=0 HoldTime=0 TlkTime=0 WorkTime=0 LocalQTime=0 NetworkQTime=0 BillRate=-1 SegmentTime=-1 NewTransaction=78 RouteID=-1 AGSTID=28946(13212933) SGSTID=23009(0x7433) SvcSTID=-1(-1) RoutedAGSGID=0 RoutedSGSTID=0 RoutedSvcSTID=0 SRCAGSTID=28946(13212933) DigDialed=500992160213183335003 NetworkTID=-1 TrkGID=-1(-1) InstPort#=8110343 DNIS=(null) UUIData=(null) AnsWithinSvcLevel=N ConfTime=0 Priority=-1 Trk=-1 BadCallTag=N CallTypeID=-1 CED=(null) ANI=8110343 WrapupData=(null) ECCSize=0 NetworkConsult=FALSE CallGUID= LocationParamPKID= LocationParamName= PstnTrkGrpID= PstnTrkGrpChann#=13629160 CallOrginiated=0 CallReferenceID=00000000001BDDDC029D81C900000000 ProtocolID=1 ECCPayloadID=0 ApplicationData='\0' 
16:10:19:196 PG4A-opc Trace: CTI_CALL_TERMINATED_EVENT: CID=18603484 
16:10:19:196 PG4A-opc Trace: CSTA_CALL_CLEARED(PID=5006): clearedCall=(CID=18603484,Dev=,Type=Static) localConnInfo(-1)=CS_NONE cause=-1 
16:10:19:196 PG4A-opc Trace: CSTACallClearedEvent - Call not found for specified connection (CallID=18603484, Device= DevType=Static) on Peripheral 5006. 

PIM: The "Third level Text: Got an exception on a call to 'connect' (method 'run' in class ThreadMakeCall )". Try to send makecall to CUCM failed due to (PERERR_GW_E_THREADMAKECALL_GENERIC_CM_ERROR).

16:09:14:517 PG4A-pim4 Trace: CSTAMakeCall: Peripheral::ProcessCSTAThirdPartyServiceRequest InvokeID=0xf01c6bef CallingDevice=8110343 CalledDevice=9210501775255427560213183335003
16:09:32:125 PG4A-pim4 Trace: DeskLinkPeripheral::CSTAUniversalFailureConfEvent - CSTA Error 1 Per Error 13085 (PERERR_GW_E_THREADMAKECALL_GENERIC_CM_ERROR) Your request to make a call failed. Please check the number and try again. 
16:09:32:125 PG4A-pim4 Trace: DeskLinkPeripheral::CSTAUniversalFailureConfEvent - Second level Text: JTAPI Gateway - Error on MAKE CALL operation - Exception 
16:09:32:125 PG4A-pim4 Trace: DeskLinkPeripheral::CSTAUniversalFailureConfEvent - Third level Text: Got an exception on a call to 'connect'(method 'run' in class ThreadMakeCall ) 
16:09:32:125 PG4A-pim4 Trace: CSTAUniversalFailureConfEvent: TPServices::CSTAUniversalFailureConfEvent InvokeID=0xf01c6bef Error=GENERIC_OPERATION 
16:09:32:125 PG4A-pim4 Trace: PRIVATE_DATA: PeripheralErrorCode=0x331d(13085) 

JGW: You can see the timeout. It cannot meet post conditions of connect(). As an interim workaround, try to adjust JtapiPostConditionTimeout=15 into 20 of the jtapi.ini file. Finally, you need to check the CUCM side.

PG4A-jgw4 Trace: 1179230440: %JTAPI-JTAPI-7-UNK:[(P1-MobileAgentUser) GCID=(1,7363991)->ACTIVE]PlatformExceptionImpl caught: Timeout - Could not meet post conditions of connect(). 
PG4A-jgw4 Trace: Call to call[24141207].connect(LCP8110343,8110343,9210501775255427560213183335003 ) failed. Timeout - Could not meet post conditions of connect().

2. Working Log

OPC log: You can see CSTAMakeCall and CSTAMakeCallConfEvent. So the PeripheralVar4 can be sent to RTR/HLGR.

16:04:32:909 PG4A-opc Trace: TPServices::AddInvocationByCTIInvokeID - CTIInvokeID 0x262088f1 mapped to OPCInvokeID 0xf01c6a6e 
16:04:32:909 PG4A-opc Trace: CSTAMakeCall: TPServices::AddInvocationByCTIInvokeID InvokeID=0xf01c6a6e CallingDevice=8110654 CalledDevice=9210300202250459513315514536555 
16:04:32:909 PG4A-opc Trace: PRIVATE_DATA: CallPlacement=0 CallManner=0 ConsultType=0 AlertRings=0 CallOption=0 FacilityType=0 FacilityCode= AuthorizationCode= AccountCode= AnsMachTreat=0 CCT= PV1= PV2=2 PV3= PV4=13315514536555 PV5= PV6= PV7=1560000001040855196 PV8= PV9=22|15514536555|1234 PV10=DCZZ002Q-E|M|F|COR|| UserToUser= Wrapup= 
16:04:35:671 PG4A-opc Trace: CSTA_SERVICE_INITIATED(PID=5006): InitiatedConn=(CID=18603231,Dev=8110654,Type=Static) localConnInfo(1)=CS_INITIATE cause=-1 
16:04:35:671 PG4A-opc Trace: AssignAgentCall: PID=5006 AGID=26363 Periph#=13210350 LineHandle=0 LineType=2 Conn(CID=18603231,Dev=8110654,DevType=Static) SG#=29747 shadow=0 CTI=1 AgentObj=0x51042a0 
16:04:35:671 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603231 CallObj=(0x6091d30) NewState=NULL OldState=NONE. 
16:04:35:671 PG4A-opc Trace: CTI_CALL_CREATED_EVENT: CID=18603231 Src:8110654 Dest: RCK=(0-0) 
16:04:35:676 PG4A-opc Trace: CSTA_ORIGINATED(PID=5006): OrigConn=(CID=18603231,Dev=8110654,Type=Static) CallingDev=(DEV)8110654 CalledDev=(DEV)333392113315514536555 localConnInfo(3)=CS_CONNECT cause=-1 
16:04:35:676 PG4A-opc Trace: CTI_CALL_DATA_UPDATE_EVENT: CID=18603231 src=8110654 Dest= RCK=(0-0) 
16:04:37:001 PG4A-opc Trace: Call(18603231) PeripheralVar2 CTI:set to [2] 
16:04:37:001 PG4A-opc Trace: Call(18603231) PeripheralVar4 CTI:set to [13315514536555] 
16:04:37:001 PG4A-opc Trace: Call(18603231) PeripheralVar7 CTI:set to [1560000001040855196] 
16:04:37:001 PG4A-opc Trace: Call(18603231) PeripheralVar9 CTI:set to [22|15514536555|1234] 
16:04:37:001 PG4A-opc Trace: Call(18603231) PeripheralVar10 CTI:set to [DCZZ002Q-E|M|F|COR||] 
16:04:37:001 PG4A-opc Trace: CTI_CALL_DATA_UPDATE_EVENT: CID=18603231 src=8110654 Dest= RCK=(0-0) 
16:04:37:001 PG4A-opc Trace: CSTAMakeCallConfEvent: CSTAMakeCallConfEvent InvokeID=0xf01c6a6e NewCall(CID=18603231,Dev=8110654,DevType=Static) 
16:04:37:002 PG4A-opc Trace: CSTA_NETWORK_REACHED(PID=5006): Conn=(CID=18603231,Dev=333392113315514536555,Type=Dynamic) TrunkUsed=(TRUNK)-1 CalledDev=(TRUNK)-1 localConnInfo(-1)=CS_NONE cause=-1 
16:04:37:002 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603231 CallObj=(0x6091d30) NewState=NULL OldState=NULL. 
16:04:37:003 PG4A-opc Trace: ReclassifyLineInfo: (PID=5006) LineHandle=0 LineType=LT_OUTBOUND(SRC) SrcConn(CID=18603231,Dev=8110654,DevType=Static) LineHandle=0 LineType=LT_INSIDE DestConn(CID=-1,Dev=,DevType=Unknown) LineHandle=-1 LineType=LT_UNKNOWN CallObj=0x06091D30 
16:04:37:003 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603231 CallObj=(0x6091d30) NewState=NULL OldState=NULL. 
16:04:37:003 PG4A-opc Trace: CTI_CALL_DATA_UPDATE_EVENT: CID=18603231 src=8110654 Dest= RCK=(0-0) 
16:05:03:146 PG4A-opc Trace: CSTAClearCall: TPServices::AddInvocationByCTIInvokeID InvokeID=0xf01c6a8d Call(CID=18603231,Dev=8110654,Type=Static) 
16:05:03:157 PG4A-opc Trace: CSTA_CONNECTION_CLEARED (PID=5006): DroppedConn=(CID=18603231,Dev=8110654,Type=Static) ReleasingDev=(DEV)8110654 localConnInfo(-1)=CS_NONE cause=-1 
16:05:03:157 PG4A-opc Trace: SendCTICallDataUpdateEvent: CID=18603231 (Src:8110654 Dest:) NotifyCTIServer:1 CallContextFlags:0 
16:05:03:157 PG4A-opc Trace: Call:SetCallState (PID=5006) - CallID=18603231 CallObj=(0x6091d30) NewState=NONE OldState=NULL. 
16:05:03:157 PG4A-opc Trace: Call::RemoveOfferedSkillGroups - !INFORM! - CallID:18603231 on PID:5006 is REMOVING offered skill groups. Forced=N 
16:05:03:157 PG4A-opc Trace: Cleared(PID=5006): CallObj=(0x06091D30) SrcConn(CID=18603231,Dev=8110654,DevType=Static) DestConn(CID=-1,Dev=,DevType=Unknown) 
16:05:03:158 PG4A-opc Trace: TerminateCall: Inst=8110654 Line[0,OUT]:TLK->IDL Conn=(CID=18603231,DevID=8110654,DevType=Static) AgentLineObj=0x4291300 
16:05:03:158 PG4A-opc Trace: TerminateCall: AGSTID=26363 PeriphNo=13210350 SG=29747 LineType=OUT Talk=28 Wrap=0 Hold=0 Conn=(CID=18603231,DevID=8110654,DevType=Static) AgentObj=0x51042a0 CallObj=0x6091d30 
16:05:03:158 PG4A-opc Trace: BuildClosedCallRecord(PID=5006): CallObj=0x6091d30 DateTime=10/25 08:05:02 IntervalDatetime=10/25/2022 08:00:00 MRDomainID=1 CID=18603231 RCK(RtrDate=0,RtrCID=0) RCKSeq#=0 CD=10(DBCD_DROP_NO_ANSWER) CDF=4(DBCDF_ERROR) PeriphCallType=9,DBCD_OUT AppTaskCD=0 ICRCK=1342458021, ICRCKParent=-1 ICRCKChild=-1 NetworkTime=0 Duration=28 RingTime=0 DelayTime=28 TimeToAband=0 HoldTime=0 TlkTime=0 WorkTime=0 LocalQTime=0 NetworkQTime=0 BillRate=-1 SegmentTime=-1 NewTransaction=78 RouteID=-1 AGSTID=26363(13210350) SGSTID=23009(0x7433) SvcSTID=-1(-1) RoutedAGSGID=0 RoutedSGSTID=0 RoutedSvcSTID=0 SRCAGSTID=26363(13210350) DigDialed=333392113315514536555 NetworkTID=-1 TrkGID=-1(-1) InstPort#=8110654 DNIS=(null) UUIData=(null) AnsWithinSvcLevel=N ConfTime=0 Priority=-1 Trk=-1 BadCallTag=N CallTypeID=-1 CED=(null) ANI=8110654 WrapupData=(null) ECCSize=10 NetworkConsult=FALSE CallGUID= LocationParamPKID= LocationParamName= PstnTrkGrpID= PstnTrkGrpChann#=13629160 CallOrginiated=0 CallReferenceID=00000000001BDCDF029D80B500000000 ProtocolID=1 ECCPayloadID=0 ApplicationData='\0' 
16:05:03:158 PG4A-opc Trace: CTI_CALL_TERMINATED_EVENT: CID=18603231 
16:05:03:158 PG4A-opc Trace: CSTA_CALL_CLEARED(PID=5006): clearedCall=(CID=18603231,Dev=,Type=Static) localConnInfo(-1)=CS_NONE cause=-1 
16:05:03:158 PG4A-opc Trace: CSTACallClearedEvent - Call not found for specified connection (CallID=18603231, Device= DevType=Static) on Peripheral 5006.

Related Information

• OPC TEST
• Procmon
• RTT TEST
• Related Trace Level
• Cisco Technical Support & Downloads