Designing CVP for High Availability
This chapter describes a best-practices Customer Voice Portal (CVP) solution high-availability system design.
The chapter covers the following topics:
•Layer 2 Switch
•CVP Voice Browser
•CVP Application Server
•Content Services Switch (CSS)
•CVP VoiceXML Server
•Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) Server
•Intelligent Contact Management (ICM)
•Standalone Distributed Diagnostics and Services Network (SDDSN)
This design provides the highest level of failure protection. Your solution may vary depending upon the customer's business needs, including:
•Tolerance for call failure
CVP can be deployed in many configurations that use numerous hardware and software components. Each solution must be designed in such a way that a failure impacts the fewest resources in the call center. The type and number of resources impacted depends on how stringent the business requirements are and which design characteristics you choose for the various CVP components, including the network infrastructure. A good CVP design is tolerant of most failures (defined later in this chapter), but sometimes not all failures can be made transparent to the caller.
CVP is a sophisticated solution designed for mission-critical call centers. The success of any CVP deployment requires a team with experience in data and voice internetworking, system administration, and CVP application configuration.
Before implementing CVP, use careful preparation and design planning to avoid costly upgrades or maintenance later in the deployment cycle. Always design for the worst possible failure scenario, with future scalability in mind for all CVP sites.
In summary, plan ahead and follow all the design guidelines and recommendations presented in this guide and in the latest version of the Cisco IP Telephony Solution Reference Network Design (SRND), available at:
For assistance in planning and designing your CVP solution, consult your Cisco or certified Partner Systems Engineer (SE).
A note about the CVP Call Server component: throughout this document we have been treating this device as a single component, because there has not been a need to examine it in any more depth than that. When discussing CVP high availability however, it is important to understand that there are actually two parts to this component: the CVP Voice Browser and the CVP Application Server. The CVP Voice Browser is responsible for H.323 processing, and the CVP Application Server is responsible for the interface to ICM, as well as for the conversion of CVP Microapplications to VoiceXML pages and vice versa. CVP's high availability design treats these two parts as independent processes, allowing calls to be handled by one CVP Call Server's Voice Browser and another CVP Call Server's Application Server at the same time. We will cover this in detail as we discuss these components later in this chapter.
Layer 2 Switch
The figure below shows a high-level physical design layout for a fault-tolerant CVP system. Each component in the CVP site is duplicated for redundancy. The quantity of each of these components varies based on the expected busy hour call attempts (BHCA) for a particular deployment. The following sections describe the failover strategy for each of these components.
Figure 5-1 Redundant CVP System
In , two Layer 2 switches comprise a single VLAN. There are two reasons for this design:
•If one switch fails, only a subset of the components becomes inaccessible. The components connected to the remaining switch can still be accessed for call processing.
•A Content Services Switch (CSS) and its redundant partner must reside on the same VLAN in order to send keep-alive messages to each other via Virtual Router Redundancy Protocol (VRRP), a protocol similar to Hot Standby Router Protocol (HSRP). If one of the Later 2 switches fails, one CSS is still functional.
Note NIC teaming is not supported in the CVP solution. Gateway/Gatekeeper NIC redundancy is discussed in the next section.
The function of the originating gateway in a CVP solution is to accept calls from the PSTN and direct them to CVP for treatment.
This section covers the following topics:
For the most current information on how to provide redundancy and reliability for originating gateways and T1/E1 lines, refer to the latest version of the Cisco IPCC Enterprise Solution Reference Network Design (SRND), available at
In addition, consider the following CVP-specific issues when designing gateways for high availability in a CVP solution:
•When used in ICM-integrated models, the originating gateway communicates with CVP using the H.323 protocol. Therefore, unlike many Cisco CallManager deployments where the gateway and Cisco CallManager typically communicate via MGCP, the originating gateway must be configured for H.323 when communicating with CVP in ICM-integrated CVP deployments. The H.323 protocol, unlike MGCP, has no inherent provisions for call survivability. Further details are discussed in CVP Voice Browser.
•When configuring the gateway for H.323, it is best to bind the H.323 interface to the virtual loopback interface, as illustrated in the following configuration example:
ip address 18.104.22.168 255.255.255.255
h323-gateway voip interface
h323-gateway voip id Dir_GK ipaddr 22.214.171.124 1719 <<<<<<< GK Info
h323-gateway voip h323-id paris
h323-gateway voip bind srcaddr 126.96.36.199
This configuration makes H.323 independent of the gateway physical interfaces. In this way, if one NIC card fails, the other NIC card can start processing H.323 traffic.
•As shown in the Redundant CVP System diagram, each gateway NIC card is connected to a different physical switch to provide redundancy in the event that one switch or interface fails. One of the switches must additionally be configured with a second VLAN to which one of the NIC cards must be connected. The loopback interface virtually ties together the two gateway NIC cards which are now residing in two different VLANs.
If the originating gateway fails, the following conditions apply to call disposition:
•Calls in progress are dropped. There is nothing that can be done to preserve these calls because the PSTN switch has lost the D-channel to all T1/E1 trunks on this gateway.
•New calls are directed to a T1/E1 at an alternate gateway, provided the PSTN switch has its trunks and dial plan configured to do so.
An H.323 gatekeeper is always required in all ICM-integrated deployment models except Model #4, NIC-based Call Control with CVP Queue, Collect and Self Service, which does not use CVP for call control at all. The CVP Voice Browser requires the use of a gatekeeper, and a CVP Voice Browser is used in all of these other models. Note, however, that a gatekeeper is an optional (although recommended) component for call routing by the originating gateway in those deployments; an originating gateway can perform all of its H.323 call routing by using VoIP dial-peers that contain static IP addresses, whereas the CVP Voice Browser must always perform a gatekeeper Remote Access Service (RAS) lookup to route calls.
Note In one particular situation, when using the VBAdmin SetTransferLabel option, the Voice Browser ignores the IP address result returned from the gatekeeper, and instead routes the IVR call leg back to the same originating gateway from which the call arrived. This feature ensures that no WAN bandwidth is incurred during IVR treatment/queuing. A gatekeeper is still required in this situation because the Voice Browser needs to perform the gatekeeper lookup function to obtain possible alternate endpoints in the event that the attempt to transfer the call to the originating gateway fails.
Gatekeepers can use one of three types of high-availability mechanisms:
Only HSRP and alternate gatekeeper are supported by CVP. Alternate gatekeeper support was introduced in CVP 3.1 SR1.
HSRP is a Cisco proprietary routing protocol that provides backup to a gatekeeper in the event of failure. Using HSRP, two gatekeepers work together to present the appearance of a single virtual router on the LAN. The gatekeepers share the same IP and MAC addresses. Therefore, if one of the gatekeepers fails, the hosts on the LAN are able to continue forwarding packets to a consistent IP and MAC address. The process of transferring the routing responsibilities from one device to another is transparent to the user. The H.323 endpoints (such as the CVP Voice Browser, Cisco CallManager, and gateways) register to a virtual IP address that represents the HSRP gatekeeper pair. If one gatekeeper fails, its partner assumes primary control. The major disadvantage of HSRP is that both gatekeepers in the HSRP failover pair must reside on the same LAN segment, and therefore they generally cannot be separated geographically. When there are multiple data centers, position an HSRP pair at each data center. It may also be possible to provide a high-speed VLAN connection between the two data centers and split the HSRP pair between the data centers.
Unlike with HSRP, when there are multiple data centers, only one gatekeeper needs to positioned at each data center when using alternate gatekeeper. The CVP Voice Browser can be configured with a list of alternate gatekeepers (as many as needed as there is no limit). When the Voice Browser starts up, it attempts to register to the first gatekeeper in the list. If the registration is not successful, it proceeds to try the remainder of the gatekeepers sequentially in the list until a successful registration occurs. The VB stays registered to that gatekeeper until either:
•That gatekeeper has some type of failure. The VB recognizes a gatekeeper failure in the following ways:
–The periodic RAS RRQ (registration request) to the gatekeeper times out or is rejected.
–An ARQ (admission request) on a transfer times out.
–The gatekeeper pro-actively tells the VB to unregister, such as when the administrator does a shutdown on the gatekeeper configuration.
•The user does another setGK from VBAdmin. This causes the VB to register with the first gatekeeper in the list, if that gatekeeper is available, otherwise it once again does a sequential attempt.
Although the CVP Voice Browser does not specifically support GUP clustering, there is no reason that the alternate gatekeepers cannot be defined as part of a GUP cluster. In this way, other H323 endpoints that do support clustering (such as Cisco Call Manager and IOS gateways) can take advantage of the benefits of gatekeeper clustering. CVP simply ignores clustering messages, such as when one of the gatekeepers in the cluster becomes overloaded. CVP uses one or more of the gatekeepers in the cluster as the alternate gatekeepers in its list and detects failure according to the rules mentioned in the preceding bullets.
This section covers the following topics:
Configure the HSRP gatekeepers according to the instructions in the latest version of the CVP Configuration and Administration Guide, available at
Using CVP Voice Browser VBAdmin:
1. set GK "10.86.129.33, 10.86.129.34, 10.86.129.35"
This sets up 3 gatekeepers to which the VB could possible register. In each case, the VB registers to the first local zone that is configured in that gatekeeper. It also uses the default RAS port 1719.
2. setGK "10.86.129.33:zonename1:1718, 10.86.129.34"
This causes the VB to first attempt to register to gatekeeper 10.86.129.33 on port 1718 with local zone "zonename1". If that gatekeeper fails, the VB subsequently attempts to register to 10.86.129.34 on port 1719 with the first local zone defined on that gatekeeper.
A gatekeeper can fail in any of the following ways. The call dispositions below apply to both HSRP and Alternate Gatekeeper.
•The primary gatekeeper fails.
–Some calls in progress may not be transferred during the period that the endpoints are re-registering to the backup gatekeeper. After the failed transfer, an error is returned to the ICM. If the ICM script is coded to return an error (an END node does this) and survivability is configured on the gateway, the call is default-routed.
–New calls arriving at the incoming gateway and CVP are serviced correctly, although it is possible that some of the calls may invoke survivability during the period that the endpoints are re-registering to the backup gatekeeper.
•All gatekeepers fail.
–The CVP Voice Browsers goes out of service.
–Calls in progress are not transferred. After the failed transfer, an error is returned to the ICM. If the ICM script is coded to return an error (an END node does this) and survivability is configured on the gateway, the call is default-routed.
–New calls arriving at the gateway are default-routed if survivability is configured on the gateway.
•The primary gatekeeper degrades but does not fail.
There are two conditions that usually cause this behavior: low memory due to memory leaks or excessive debug levels causing CPU overload.
–In this situation, call processing behavior is unpredictable due to the fact that there may be no clean failover to the backup gatekeeper. If survivability is configured on the gateway, calls are default-routed.
CVP Voice Browser
There are two components in a CVP Call Server: Voice Browser and Application Server. Each has its own set of high-availability characteristics. In general, every data center that houses CVP call servers should follow the N+1 redundancy rule. For example, if the data center requires five call server boxes, install at least six.
CVP Voice Browser high availability is configured on the originating gateways.
Configuring High Availability for New Calls
A gatekeeper knows which CVP Voice Browsers are in service and out of service. It is therefore important to let a gatekeeper route incoming calls to a CVP Voice Browser. By default, CVP Voice Browsers register to the gatekeeper with a technology prefix (tech-prefix) of 2#. A technology prefix is a way for the gatekeeper to categorize registering endpoints by functionality. In general, no additional configuration is needed on the gatekeeper for incoming calls. The Voice Browser registers to the gatekeeper with 2#, and the originating gateway prepends a 2# to the incoming Dialed Number Identification Service (DNIS) digits. The gatekeeper automatically knows how to match the gateway request to an available Voice Browser. On the gatekeeper, the command show gatekeeper gw-type-prefix displays the route plan that the gatekeeper uses to route calls.
On the originating gateways, define the dial-peer for the CVP Voice Browsers as follows:
dial-peer voice 11111 voip
session target ras (Tells the gateway to do a gatekeeper lookup to route the
tech-prefix 2# (Tells the gateway to prepend 2# to the incoming DNIS string
before passing it to the CVP Voice Browser. The CVP Voice Browser will strip
the 2# before sending the request to the ICM.)
Configuring High Availability for Calls in Progress
In the event that a CVP Voice Browser fails with calls in progress, it is possible to salvage all calls provided certain gateway configuration steps have been taken. A Voice Browser can fail in one of several ways:
•The server can crash
•The process can crash
•The process can hang
•An Ethernet cable can become disconnected.
The configuration discussed in this section protects against all of these situations. However, there are two situations that cannot be protected against:
•Someone stops the process with calls in progress. This situation occurs when a system administrator forgets to put the Voice Browser out-of-service first to allow calls in progress to finish before stopping the process.
•The Voice Browser exceeds the recommended call rate. Although there is a throttle for the absolute number of calls allowed in the Voice Browser, there is no throttle for call rate. In general, exceeding 5 calls per second (cps) for an extended period of time causes the Voice Browser to have erratic and unpredictable behavior. This situation can be prevented by proper sizing of the CVP systems.
For call survivability, configure the originating gateways as described in the latest version of the CVP Configuration and Administration Guide, available at
In the event of most downstream failures (including a CVP Voice Browser failure), the call is default-routed by the originating gateway. Note that survivability is not applicable in the CVP Standalone and NIC-routing models because there is no CVP Voice Browser involved anywhere in those models.
On the originating gateways, also configure the following commands:
no h225 timeout keepalive
hp rtcp report interval 3000
If the CVP Voice Browser fails, the following conditions apply:
•Calls in Progress
If the CVP Voice Browser fails after the caller has been transferred, (transfers include transfer to an IP phone, VoiceXML gateway, or other egress gateway), the call continues normally until a subsequent transfer activity (if applicable) is required from the CVP Voice Browser. If the caller has not hung up and is awaiting further activity, there is a period of 9 to 18 seconds of silence before the caller is default-routed by survivability to an alternate location.
If the call has not yet been transferred, the caller hears 9 to 18 seconds of silence before being default-routed by survivability to an alternate location. (Survivability does not apply in the CVP Standalone and NIC-routing models.)
New calls are directed by the gatekeeper to an alternate CVP Voice Browser. If no Voice Browsers are available, the call is default-routed to an alternate location by survivability. (Survivability does not apply in CVP Standalone and NIC Routing models)
CVP Application Server
High availability for the CVP Application Server is configured on the CVP Application Server itself, the CVP Voice Browsers, the VoiceXML gateways, and the Content Services Switch (CSS).
A CVP Voice Browser selects an application server for the switch leg of the call. Configure the CVP Voice Browser as described in the latest version of the CVP Configuration and Administration Guide.
One important addition to note is that the Voice Browser immediately reverts to using an application server earlier in the list if one becomes available. Although there is no limit to the number of application servers in the list, it is best to list only two. Otherwise, in some instances the caller has to wait for each application server in the list to time-out before being default-routed by survivability. (Survivability does not apply in the CVP Standalone and NIC Routing models.) For example, if the data center has three CVP servers (A, B, and C), then the application server lists could be configured as follows on the Voice Browsers:
Voice Browser A: setAppServerList localhost, B
Voice Browser B: setAppServerList localhost, C
Voice Browser C: setAppServerList localhost, A
On the AppAdmin Call Definitions page in the Application Server, provisions must be made for overflow calls from a failed application server. Use the following best-practice recommendation:
•Maximum number of calls allowed: 425
CVP Call Server capacity is limited to 400 simultaneous calls regardless of CPU or memory resources on the server, but it is prudent to allow for some overlap between calls which are terminating and new calls which are starting up. Note that in the most common Deployment Models, #3a and #3b, a call that is receiving IVR treatment counts as two calls: one call in the 100-port group and one call in the 200-port group.
•100-port group: 400
•200-port group: 400
It is generally best to make the 100-port and 200-port group values equal to accommodate any ratio of IVR treatment and transferred calls.
The VoiceXML gateway sends HTTP requests to a CVP Application Server to obtain a VoiceXML document. It uses the following VoiceXML gateway configuration parameters to locate a server when not using a CSS. Note that the backup server is invoked only if the primary server is not accessible, and if this is not a load-balancing mechanism. Additionally, every call first attempts to access the primary before failing over to the backup.
hp host isn-vxml <ip-address-of-primary-application-server>
hp host isn-vxml-backup <ip-address-of-secondary-application-server>
The VoiceXML gateway also uses the following VoiceXML gateway configuration parameters to locate a server when using a CSS. Because the CSS almost always locates an application server on the first request, a backup server is rarely invoked but should always be configured.
ip host isn-vxml <ip-address-of-CSS-VIP-for-app-server>
ip host isn-vxml-backup <ip-address-of-CSS-VIP-for-app-server>
There are several files in flash memory on the gateway that are also involved with high availability: handoff.tcl, recovery.vxml, and several .wav files. Use Trivial File Transfer Protocol (TFTP) to load the proper files into flash memory according to instructions found in the latest version of the CVP 3.1 Configuration and Administration Guide.
The CSS, if used, provides load balancing and failover for CVP Application Servers (as well as some other components, which are discussed later). Configure the CSS according to the instructions in the latest version of the CVP Configuration and Administration Guide.
If the CVP Application Server fails, the following conditions apply to the call disposition:
•Calls in progress are default-routed to an alternate location by survivability on the originating gateway. (Survivability does not apply in the CVP Standalone and NIC Routing models.)
•New calls are directed to an in-service CVP Application Server.
The VoiceXML Gateway parses and renders VoiceXML documents obtained from one of several sources: the CVP Application Servers, the CVP VoiceXML Servers, or some other external VoiceXML source. Rendering a VoiceXML document means retrieving and playing prerecorded audio files, collecting and processing user input, and/or connecting to an ASR/TTS server for voice recognition and dynamic text-to-speech.
High availability configuration for VoiceXML gateways is controlled by the gatekeeper and/or the CVP Voice Browser depending on the geographic location of the VXML gateways (centralized or distributed).
Centralized VoiceXML gateways
This refers to the case where VoiceXML gateways reside in the same data center as the CVP Call Server. On the gatekeeper, configure a zone prefix list that contains the H.323 IDs of all VoiceXML gateways at the data center. For example, assume that there are three VoiceXML gateways in the data center with H.323 IDs of VoiceXMLgw1, VoiceXMLgw2, and VoiceXMLgw3, and that the ICM label for the type-7 Network VRU is 9999999. In this example, the gatekeeper distributes calls in essentially a round-robin scheme among all three VoiceXML gateways, provided they are all in service:
zone prefix gkzone-name 9999999* gw-priority 10 VoiceXMLgw1 VoiceXMLgw2
Distributed VoiceXML gateways (ingress gateway and VoiceXML same gateway)
This refers to the case where the gateway that processes the incoming call from the PSTN is geographically separated from the CVP servers and the VoiceXML gateway that is used is the same as the ingress gateway. This is done to keep the media stream at the edge and not consume bandwidth on the WAN. Use the SetTransferLabel in VBAdmin (not gatekeeper zone prefixes) to control the selection of the VXML gateway.
Distributed VoiceXML gateways (ingress gateway and VoiceXML different gateway)
This refers to the case where the gateway that processes the incoming call from the PSTN is geographically separated from the CVP servers and the VoiceXML gateway that is used is different than the ingress gateway but located as the same site as the ingress gateway. This is done to keep the media stream at the edge and not consume bandwidth on the WAN and to optimize VoiceXML gateway sizing when it is appropriate to separate ingress and VXML gateways. In this case, setTransferLabel cannot be used, since one does not want the IVR leg of the call to go back to the ingress gateway. Additionally, it is also impractical to configure gatekeeper zone prefixes to control the call routing since one would need to configure separate network VRUs and labels in ICM for each remote site. Instead, use VBAdmin SetSigDigits functionality.
Using this method, the CVP Voice Browser strips leading significant digit(s) from the incoming DNIS. The value that is stripped is saved and prepended with a # sign before subsequent transfers for the call occur. The VoiceXML gateway at the remote site should register to the gatekeeper with the same tech-prefix as the leading significant digit(s) that were stripped from the DNIS. The gatekeeper then routes the IVR leg of the call to the correct VoiceXML gateway. If using a Call Manager, remember that CVP indiscriminately prepends the sigdigits value to all transfers, including those to Call Manager. It is therefore necessary when using Call Manager in this scenario to define a gatekeeper-controlled trunk for each of the VoiceXML gateway tech-prefixes and to add zone prefix configuration to the gatekeeper for the Call Manager agents.
dial-peer voice NNNN voip
tech-prefix 2# (gets the call to CVP)
Apply a translation-rule to the incoming DNIS to prepend the value "3".
Assuming DNIS is 8002324444, the final DNIS routed to CVP is 2#38002324444
setTechPrefix 2# (default)
setSigDigits 1 (strip one digit from the DNIS after stipping the 2# tech
Register to gatekeeper with tech-prefix 3#
Create a separate gatekeeper-controlled trunk corresponding to each of the
tech-prefixes used by the VXML gateways.
Gatekeeper (only if using Call Manager):
Define zone prefixes to appropriately route calls to Call Manager agents.
•Call arrives at CVP with DNIS 2#38002324444
•CVP first strips the tech-prefix (2#), leaving 38002324444
•CVP then strips one digit (3) from the beginning of the DNIS, leaving 8002324444.
•8002324444 is passed to ICM for call routing.
•When it is time to transfer, assume ICM returns the label 99999998877. CVP prepends 3#, giving 3#99999998877. This value is then passed to the gatekeeper for address resolution.
•Gatekeeper resolves this label to the VoiceXML gateway which registered as 3#.
•VoiceXML gateway should strip the 3# on the way in, leaving 99999998877 for the destination phone address.
In all the cases described above, provide alternate endpoints for each of the VoiceXML gateways in case the VoiceXML gateway rejects the incoming request (perhaps due to configuration error or overload):
endpoint alt-ep h323id VoiceXMLgw1 ip-address-VoiceXMLgw2
endpoint alt-ep h323id VoiceXMLgw2 ip-address-VoiceXMLgw3
endpoint alt-ep h323id VoiceXMLgw3 ip-address-VoiceXMLgw1
In the event that a CVP Voice Browser is unable to connect to a VoiceXML gateway, an error is returned to the ICM script. Separate the Send to VRU node from the first Run External script node in order to catch the VoiceXML gateway connection error. If an END script node is used off the X-path of the Send to VRU node, the call is default-routed by survivability on the originating gateway. (Survivability does not apply in the CVP Standalone and NIC Routing models.) A Queue to Skill group node could also be used, but that method is effective only if there is an agent available. Otherwise, ICM tries to queue the caller and that attempt fails because the CVP Voice Browser is once again unable to connect to a VoiceXML gateway. An END node could then also be used off the X-path of the Queue to Skill Group node to default-route the call.
If the VoiceXML Gateway fails, the following conditions apply to the call disposition:
•Calls in progress are default-routed to an alternate location by survivability on the ingress gateway. (Survivability does not apply in CVP Standalone and NIC Routing models.)
•New calls find an alternate VoiceXML gateway.
Hardware Configuration for High Availability on the Voice Gateways
The individual hardware components have the following high-availability options:
•Redundant power supplies and on-hand spares
•Separate components for higher availability
•Dedicated components, which have fewer interaction issues
Example 1: Separate PSTN Gateway and VoiceXML Gateway
A PSTN gateway and a separate VoiceXML gateway provide greater availability than a combine PSTN and VoiceXML gateway.
Example 2: Duplicate Components for Higher Availability
•Two 8-T1 PSTN gateways provide greater availability than one 16-T1 PSTN gateway.
•Two 96-port VoiceXML servers provide greater availability than one 192-port VoiceXML server.
•Larger designs can use N+1 spares for higher availability.
Example 3: Geographic Redundancy for Higher Availability
Geographical redundancy and high availability can be achieved by purchasing duplicate hardware for Side A and Side B.
Content Services Switch (CSS)
The VoiceXML gateway is the only box in the CVP solution that makes requests to the CSS. When the VoiceXML gateway needs to make a request for media, ASR/TTS or VoiceXML, it looks in its configuration to see to where it should make the request. When a CSS is used, the IP address that is configured on the VoiceXML gateway is a virtual IP address that points to a service configured on the CSS. There are four types of services that the VoiceXML gateway client can request from the CSS:
•CVP VoiceXML Server
•CVP Application Server
If the primary CSS that is servicing these requests should fail, the client VoiceXML gateway must still be able to obtain media and VoiceXML from the servers.
You can configure high availability for the CSS by using the Virtual IP (VIP) Redundancy method, as described in the latest version of the CVP Configuration and Administration Guide.
Also refer to the latest version of the CSS Redundancy Configuration Guide, available at:
Essentially, a master/backup pair of CSSs functions very much like an HSRP gatekeeper pair. They must reside on the same VLAN and exchange heartbeats using Virtual Router Redundancy Protocol (VRRP), a protocol very similar to HSRP. If the primary CSS fails, the backup CSS recognizes the failure within three seconds and begins processing client requests to its configured virtual IP addresses. The configuration of the master and backup CSSs must always be kept in syncronization.
If the master CSS fails, then the following conditions apply to the call disposition:
•Calls in progress encounter various behaviors, depending on the type of service the VoiceXML gateway client requested:
–Media server requests are unaffected. The VoiceXML gateway has a very short-lived interaction with the CSS for audio files. Upon receiving a media server request from the gateway, the CSS simply provides an HTTP redirect IP address for the VoiceXML gateway. At that point, the gateway fetches the audio file directly from the media server, bypassing any further interaction with the CSS. Additionally, media file requests to the CSS are very infrequent because the VoiceXML gateway caches previously retrieved media files.
–CVP Application Server requests are unaffected. Only the initial VoiceXML document request to a CVP Application Server uses the CSS. The CSS first picks a CVP Application Server to service the request. The first document passes through the CSS on its return to the VoiceXML gateway. However, subsequent VoiceXML requests are made directly from the VoiceXML gateway client to the CVP Application Server. If the CSS fails during the very brief period that the first VoiceXML document is being returned, the VoiceXML gateway simply retries the request. If the backup (now primary) CSS selects the same CVP Application Server as the previous one, there is an error due to a duplicate call instance. In that case, the caller is default-routed by survivability on the originating gateway. (Survivability does not apply in the CVP Standalone model.)
–ASR/TTS requests typically fails but might be recoverable. When the VoiceXML gateway first makes an ASR/TTS request to the CSS, a TCP connection is opened from the VoiceXML gateway to the Media Resource Control Protocol (MRCP) server. That TCP connection goes through the CSS and persists until the caller disconnects or is transferred to an agent. If the primary CSS fails, that TCP connection is terminated. The VoiceXML gateway returns an error code that you could write a script to work around. The worst-case scenario is that the caller is default-routed to an alternate location by survivability on the originating gateway. (Survivability does not apply in the CVP Standalone model.)
–CVP VoiceXML Server requests may fail. The VoiceXML Gateway is "sticky" to a particular CVP VoiceXML Server for the duration of the VoiceXML session. It uses CSS cookies to provide that stickiness. If the CSS fails, the backup CSS has no knowledge of the cookie. Subsequent requests in the session might go to the correct CVP VoiceXML Server, but there is no guarantee. The VoiceXML gateway returns an error code that you could write a script to work around. The worst-case scenario is that the caller is default-routed to an alternate location by survivability on the originating gateway. (Survivability does not apply in the CVP Standalone model.)
•New calls are directed transparently to the VIPs on the backup CSS, and service is unaffected.
Audio files can be stored locally in flash memory on the VoiceXML gateway or on an HTTP/TFTP file server. By definition, audio files stored locally are highly available. However, HTTP/TFTP file servers provide the advantage of centralized administration of audio files.
Configuration when Using CVP Microapplications
The VoiceXML gateway sends HTTP requests to an HTTP media server to obtain audio files. It uses the following VoiceXML gateway configuration parameters to locate a server when not using a CSS. Note that the backup server is invoked only if the primary server is not accessible, and this is not a load-balancing mechanism. Once failover occurs, all calls continue to use the backup server until that server becomes inaccessible. Note that media-server is not a fixed name. It simply needs to match whatever name was assigned to the mediaserver ECC variable in the ICM script.
ip host mediaserver <ip-address-of-primary-media-server>
ip host mediaserver-backup <ip-address-of-secondary-media-server>
The VoiceXML gateway also uses the following VoiceXML gateway configuration parameters to locate a server when using a CSS. Because the CSS almost always locates a media server on the first request, a backup server is rarely invoked but should always be configured. The CSS, if used, provides load-balancing and failover for HTTP media servers.
ip host mediaserver <ip-address-of-CSS-VIP-for-media-server>
iip host mediaserver-backup <ip-address-of-CSS-VIP-for-media-server>
Configure the CSS according to the instructions in the latest version of the CVP Configuration and Administration Guide, available at:
Call Disposition when Using CVP Microapplications
If the media server fails, the following conditions apply to the call disposition:
•Calls in progress should recover automatically. The high-availability configuration techniques described above should make the failure transparent to the caller. If the media request does fail, use scripting techniques to work around the error (for example, retry the request, transfer to an agent or label, or use TTS).
•New calls are directed transparently to the backup media server, and service is not affected.
•If the media server is located across the WAN from the VXML gateway and the WAN dies, the gateway continues to use prompts from gateway cache until such time that the requested prompt becomes stale, at which time the gateway attempts to re-fetch the media and the call fails if survivability is not enabled. If survivability is enabled, the call are default-routed.
Configuration when using CVP VoiceXML Studio scripting
When scripting in CVP VoiceXML Studio, unlike with ICM scripting, there is no concept of "-backup" for media files. The best the script writer can do is to point Properties->AudioSettings->Default Audio Path URI in the application to a single media server or the CSS VIP address for a farm of media servers.
CVP VoiceXML Server
The VoiceXML gateway makes HTTP requests to the CVP VoiceXML Server to obtain VoiceXML documents.
The CVP VoiceXML Server high-availability configuration and behavior differ between Standalone deployments and deployments which are integrated with ICM, as described in the following sections.
Standalone Self Service Deployments
For instructions on configuring primary and backup CVP VoiceXML Servers, refer to section "CVP VoiceXML Server Standalone Solution" in the latest version of the CVP Configuration and Administration Guide, available at
Specifically, it is the CVPPrimaryVXMLServer and CVPBackupVXMLServer gateway parameters that control the high availability characteristics of VoiceXML server. If VoiceXML server load balancing and more robust failover capabilities are desired, a CSS may be used. To configure the CSS for CVP VoiceXML Server, refer to Appendix D, section "CVP VoiceXML Server Configuration" at the preceding link. Load balancing can also be achieved without a CSS by varying the primary and backup CVP VoiceXML Server configurations across multiple gateways.
Deployments using ICM
For instructions on configuring the CVP VoiceXML Server in an ICM-integrated deployment, refer to section "CVP VoiceXML Server with ICM" in the latest version of the CVP Configuration and Administration Guide.
In the ICM script, first attempt to connect to VoiceXML Server A (172.21.5.32). If the application fails out the X-path of the VoiceXML Server ICM script node, VoiceXML Server B (172.20.1.32) should be tried. These IP addresses can also represent VoiceXML Server VIP's on the CSS. Note the optional time check in the script to see how long the caller had been in the IVR when the application failed. If the caller had been in the IVR a significant amount of time, it may seem rather odd to the caller to restart them from the beginning of the application.
Figure 5-2 ICM Deployment Script
If the CVP VoiceXML Server fails, the following conditions apply to the call disposition:
•Calls in progress in a Standalone deployment are disconnected. Calls in progress in an ICM-integrated deployment can be recovered using scripting techniques to work around the error as shown in the script above (for example, retry the request, transfer to an agent or label, or force an error with an END script node to invoke survivability on the originating gateway).
•New calls are directed transparently to an alternate CVP VoiceXML Server. Note that without a CSS, callers may experience a delay at the beginning of the call waiting for the system to timeout while trying to connect to the primary VoiceXML server.
Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) Server
The VoiceXML gateway sends MRCP requests to the ASR/TTS servers in order to perform voice recognition and text-to-speech instructions that are defined in a VoiceXML document.
The ASR/TTS high-availability configuration and behavior differ between Standalone and ICM-integrated deployments, as described in the following sections.
Standalone Self Service Deployments
A CSS is required in Standalone deployments to provide failover capabilities for ASR/TTS. For instructions on configuring the ASR/TTS Server in a Standalone deployment, refer to section "ASR, TTS and Media Server Redundancy for Cisco CVP VoiceXML Server Applications" in the latest version of the CVP Configuration and Administration Guide, available at:
For instructions on configuring the CSS for ASR/TTS, see Appendix D in the latest version of the CVP Configuration and Administration Guide at the above link.
Deployments Using ICM
The VoiceXML gateway uses gateway configuration parameters to locate an ASR/TTS server both when using a CSS and when not using a CSS. Note that the backup server is invoked only if the primary server is not accessible, and if this is not a load-balancing mechanism. Once failover occurs, all calls continue to use the backup server until that server becomes inaccessible.
The hostnames (such as asr-en-us) are fixed and cannot be modified. The only portion that may be modified is the locale. In the following example, there is a set of primary and backup English ASR/TTS servers and a set of Spanish servers. Configure the CSS, if used, according to Appendix D in the latest version of the CVP Configuration and Administration Guide, available at
When a CSS is used, the IP addresses mentioned the following example would be the virtual IP address for the ASR/TTS service on the CSS.
ip host asr-en-us <ip-address-of-primary-English-ASR-server>
ip host asr-en-us-backup <ip-address-of-secondary-English-ASR-server>
ip host tts-en-us <ip-address-of-primary-English-TTS-server>
ip host tts-en-us-backup <ip-address-of-secondary-English-TTS-server>
ip host asr-es-es <ip-address-of-primary-Spanish-ASR-server>
ip host asr-es-es-backup <ip-address-of-secondary-Spanish-ASR-server>
ip host tts-es-es <ip-address-of-primary-Spanish-TTS-server>
ip host tts-es-es-backup <ip-address-of-secondary-Spanish-TTS-server>
If the ASR/TTS MRCP server fails, the following conditions apply to the call disposition:
•Calls in progress in Standalone deployments are disconnected. Calls in progress in ICM-integrated deployments can be recovered using scripting techniques to work around the error (for example, retry the request, transfer to an agent or label, switch to prerecorded prompts and DTMF-only input for the remainder of the call, or label or force an error with an END script node to invoke survivability on the originating gateway).
•New calls in Standalone or ICM-integrated deployments are directed transparently to an alternate ASR/TTS server if a CSS is being used. New calls in ICM-integrated deployments are directed transparently to an alternate ASR/TTS server if "-backup" gateway hostnames have been used.
CVP transfers callers to IPCC Enterprise agent phones or desktops using the H.323 protocol. The CVP Voice Browser gets an agent label from the ICM and queries the gatekeeper with the label. The gatekeeper then returns an IP address of one Cisco CallManager in the cluster, and the CVP Voice Browser calls that IP address and connects the caller to the agent. The CVP Voice Browser proxies the signaling channel, so it remains in the call signaling loop after the transfer is completed. However, the RTP stream flows directly from the originating gateway to the phone. This fact becomes very significant in discussions of high availability.
For the most current information on providing Cisco CallManager for high availability, refer to the latest version of the Cisco IPCC Solution Reference Network Design (SRND), available at
In addition, configure the originating gateway parameters according to the guidelines in the section on Configuring High Availability for Calls in Progress.
If the Cisco CallManager process fails on the server that is either hosting the call or hosting the phone, the following conditions apply to the call disposition:
•Calls in progress are preserved. Skinny Client Control Protocol (SCCP) phones have the ability to preserve calls even when they detect the loss of their Cisco CallManager. The caller-and-agent conversation continues until either the caller or agent goes on-hook. The CVP Voice Browser recognizes that Cisco CallManager has failed, assumes the call should be preserved, and maintains the signaling channel to the originating gateway. In this way, the originating gateway has no knowledge that Cisco CallManager has failed. Note that additional activities in the call (such as hold, transfer, or conference) are not possible. Once the parties go on-hook, the phone then re-homes to another Cisco CallManager server. When the agent goes on-hook, Real-Time Control Protocol (RTCP) packets ceases transmitting to the originating gateway, which causes the gateway to disconnect the caller 9 to 18 seconds after the agent goes on-hook. If survivability has been configured on the gateway and the caller is waiting for some additional activity (the agent might think caller is being blind-transferred to another destination), the caller is default-routed to an alternate location.
•New calls are directed to an alternate Cisco CallManager server in the cluster.
Intelligent Contact Management (ICM)
Cisco Intelligent Contact Management (ICM) software provides enterprise-wide distribution of multichannel contacts (inbound/outbound telephone calls, Web collaboration requests, email messages, and chat requests) across geographically separated contact centers. ICM software is an open standards-based solution whose capabilities include routing, queuing, monitoring, and fault tolerance.
For the most current information on configuring ICM for high availability, refer to the latest version of the Cisco IPCC Enterprise Solution Reference Network Design (SRND), available at
There are many components in Cisco ICM, and call disposition varies depending on the component that fails. Although there are a few exceptions, the following conditions apply to the call disposition:
•If the Voice Response Unit (VRU) Peripheral Gateway (PG) or any component on the VRU PG fails, calls in progress are default-routed by survivability on the originating gateway.
•If the Logger fails, calls in progress are unaffected.
•If the primary router fails, calls in progress are unaffected. If both the side A and side B routers fail, calls in progress are default-routed by survivability on the originating gateway.
•New calls are directed to the backup ICM component.
Standalone Distributed Diagnostics and Services Network (SDDSN)
SDDSN is a component that provides alarm reporting for CVP using a variety of mechanisms. The CVP Voice Browser and Application Server send process state information to SDDSN, which in turn generates alarms that can be distributed to one of several alarm tracking mechanisms. Common alarm tracking mechanisms include Cisco ICM Alarm Tracker, CiscoWorks2000, and Hewlett-Packard OpenView. Note that alarms are limited to CVP process-level alarms and interface alarms (that is, are the processes up and in service, can the Voice Browser contact a gatekeeper, can the Application Server contact the ICM, and so forth). Call-level alarms and server characteristics (memory and CPU) are not monitored.
For the most current information on configuring SDDSN for high availability, refer to the latest version of the CVP Configuration and Administration Guide, available at:
If the primary SDDSN server fails, CVP begins transmitting alarm data to the backup SDDSN server if one is configured. The following conditions also apply to call disposition:
•Calls in progress are unaffected.
•New calls are unaffected.