Field Notice: FN - 63291 - Cisco Computer Telephony Integration Object Server (CTIOS) CTIOS Server Assertions
Revised January 12, 2010
December 16, 2009
THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Changed link for CTI OS 7.5(5) ES2.
Initial Public Release.
ICM CTI/CTIos Server - 7.2(7), 7.5(4), 7.5(5), 7.5(6)
During high call volume in production, CTIOS may restart or assert with a minidump, failure to accept new client connections due to previous exceptions that may cause agent login failure, button enablement issues related agent state or call control. If CTIOS Server node did not restart automatically, it may need to be cycled via Cisco ICM Service Control to recover from this condition.
Note: We have observed the problem to be more frequent, exposed under load with variable degree of agent counts and call volumes depending on customer environment and call flows.
The problem which causes the CTI OS to generate exceptions and crash was introduced in 7.2(7).
Potential symptoms of the problem addressed by ES5 are:
- Generation of many exceptions will corrupt the memory and eventually will lead to CTI OS Server crash.
- During load CTIOS server crashes.
- The CTIOS Server will stop allowing any more agents to connect/login.
- CTIOS Server on other side of PG only show connection to client being lost.
- Timing issue on CTIOS server causes two call connections to be created for the same device which results in greyed out icons on the CTIOS desktops.
The following is an example of a CTI OS server log showing multiple exceptions:
08:46:11:383 CTIOS1-ctios Trace: CServiceBroker::RemoveObject() FAILED, Object(05D36008) was not found.
CSCtb47394 & CSCtb43783:
11:17:43 CTIOS1-ctios Trace: CRITICAL: ServiceQueuePostThread caught exception while processing a event[eCallDataUpdateEvent], GetLastError() returned 0.
11:17:43 CTIOS1-ctios CTI OS Server has generated an exception in ServiceBroker::ProcessEvent processing eCallDataUpdateEvent.\nDetails:\nLast Error: 0\n
11:17:45 CTIOS1-ctios Trace: CRITICAL: ServiceQueuePostThread caught exception while processing a event[eCallEstablishedEvent], GetLastError() returned 0.
11:17:45 CTIOS1-ctios CTI OS Server has generated an exception in ServiceBroker::ProcessEvent processing eCallEstablishedEvent.\nDetails:\nLast Error: 0\n
7:18:50 CTIOS1-ctios Trace: [call.5000.117661379]::HandleCallEstablishedEvent(): attempt to create Called connection for Device 11809.
17:18:50 CTIOS1-ctios Trace: [call.5000.117661379]::HandleCallDeliveredEvent(): attempt to create Alerting connection for Device 11809.
12:30:01 CTIOS1-ctios Trace: ClientMgr[00E33790]: AddAcceptingClients, caught unknown exception. client, threadid
Note: Call IDs, Device IDs, Object IDs, and Thread IDs mentioned in the above log snippets are only examples.
This issue will be resolved in CTI OS Release 7.2(7) and greater. The solution to this issue for earlier releases of CTI OS is to install the appropriate software engineering special (ES).
On systems that are running CTI OS Servers Release 7.2(7), install CTI OS 7.2(7) ES19.
CTI OS 7.2(7) ES19 can be downloaded from: CTIOS7.2.7_ES19.exe
On systems that are running CTI OS Servers Release 7.5(4), install CTI OS 7.5(4) ES3.
CTI OS 7.5(4) ES3 can be downloaded from: CTIOS7.5.4_ES3.exe
On systems that are running CTI OS Servers Release 7.5(5), install CTI OS 7.5(5) ES2.
CTI OS 7.5(5) ES2 can be downloaded from: CTIOS7.5.5_ES2.exe
On systems that are running CTI OS Servers Release 7.5(6), install CTI OS 7.5(6) ES5.
CTI OS 7.5(6) ES5 can be downloaded from: CTIOS7.5.6_ES5.exe
The following table provides information of the testing of Cisco products for this issue.
||Component Tested (All or Specific)
|CTI OS Server
||All CTI OS 7.x components tested on Windows Server 2003 R2 SP2
|CTI OS Server
|CTI OS Server
|CTI OS Server
* Testing Disposition
I ? In Progress (indicating that testing is in progress and will be updated when complete)
Y ? Yes (tested)
NA ? Not Applicable (none of the security updates are being tested because of a ?Not Applicable? or ?Deferred? assessment)
To follow the bug ID link below and see detailed bug information, you must be a registered customer and you must be logged in.
|CSCsz73388 (registered customers only)
||CTIOS Server stops accepting new connections after "unknown exception"
|CSCtb98855 (registered customers only)
||CTIOS Server show Established before Delivered causing grayed icons
|CSCtb43783 (registered customers only)
||Multiple calls seen on CTIOS desktop during load
|CSCtc32117 (registered customers only)
||CTIOS Server assertion; CServiceBroker::RemoveObject() FAILED
|CSCtb47394 (registered customers only)
||CTIOS server reloads during outbound load creating minidump
For More Information
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:
Receive Email Notification For New Field Notices
Cisco Notification Service - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.