Call Throttling and the Code Yellow State
Call throttling allows Cisco Unified Communications Manager to automatically throttle (deny) new call attempts when it determines that various factors, such as heavy call activity, low CPU availability to Cisco Unified Communications Manager, routing loops, disk I/O limitations, disk fragmentation or other such events, could result in a potential delay to dial tone (the interval users experience from going off hook until they receive dial tone).
This chapter provides the following information about call throttling:
•Introducing Call Throttling
•Troubleshooting Call Throttling
Introducing Call Throttling
Call throttling occurs automatically when Cisco Unified Communications Manager determines such conditions to be present, and the system exits throttling automatically when such conditions are alleviated. You can configure the parameters that are associated with entering and exiting call throttling through several service parameters in Cisco Unified Communications Manager Administration (System > Service Parameters) although Cisco does not advise modification of these parameters unless recommended by Cisco customer support. See Service Parameters Configuration in the Cisco Unified Communications Manager Administration Guide for information on accessing and configuring service parameters.
Cisco Unified Communications Manager uses the values that are specified in the call-throttling-related parameters to evaluate the possibility of a delay to dial tone and also to determine when conditions no longer necessitate call throttling. When throttling is necessary to prevent excessive delay to dial tone, Cisco Unified Communications Manager enters a Code Yellow state, and new call attempts are throttled (denied). You can disable call throttling via the System Throttle Sample Size service parameter, but Cisco does not recommend disabling call throttling. The following list defines several of the call throttling-related service parameters:
•Code Yellow Entry Latency defines the maximum allowable delay, in milliseconds, to handle SDL messages that are sent to Cisco Unified Communications Manager by the various devices in the system as well as the wealth of internal messages that are received and sent by Cisco Unified Communications Manager for various activities such as KeepAlives, change notification, and many more types of internal messaging. If the calculated average expected delay is more than the value that is specified in this service parameter, Cisco Unified Communications Manager enters a Code Yellow state to initiate call throttling, and stops accepting new calls.
•Code Yellow Exit Latency Calculation determines the acceptable percentage of Code Yellow Entry Latency to specify exit criteria for leaving the Code Yellow state (Code Yellow exit latency) when Cisco Unified Communications Manager has initiated call throttling. The basis for the value that you specify in this parameter comprises a formula that uses the value in the Code Yellow Entry Latency parameter, which specifies the delay in milliseconds. To arrive at a percentage, use the following formula: Code Yellow Entry Latency value multiplied by the Code Yellow Exit Latency value. For example:
Code Yellow Entry Latency service parameter value: 20 msec
Code Yellow Exit Latency service parameter value: 40%
Code Yellow Exit Latency value = 20 X 0.4 = 8 msec, which means Cisco Unified Communications Manager exits Code Yellow state if the calculated message latency drops to 8 msec or lower.
To get out of the Code Yellow state, Cisco Unified Communications Manager ensures that the average expected delay is less than the value of the Code Yellow exit latency.
•Code Yellow Duration specifies the number of minutes that a Cisco Unified Communications Manager system can remain in a Code Yellow state (call throttling). If this duration is met and the system is still in Code Yellow state, Cisco Unified Communications Manager enters a Code Red state, which indicates that Cisco Unified Communications Manager has remained in a Code Yellow state for an extended period and cannot recover. When Cisco Unified Communications Manager enters a Code Red state, the Cisco CallManager service restarts, which also produces a memory dump that may be helpful for analyzing the failure.
•System Throttle Sample Size indicates the size of the sample, in seconds, that is used to calculate the average expected delay for Cisco Unified Communications Manager to handle an SDL message. For example, a sample size of 10 means that Cisco Unified Communications Manager must calculate a non-zero latency value for 10 consecutive seconds before it will calculate the average expected delay and compare it to the value in the CodeYellow Entry Latency parameter. You can disable call throttling via this parameter.
When delay to dial tone is calculated to be over the threshold that is configured in the call-throttling-related service parameters, Cisco Unified Communications Manager begins rejecting new calls. When call throttling is engaged, a user who attempts a new call will receive reorder tone and, depending on the phone model, may also receive a prompt on the phone display. Call throttling effectively avoids the problem in which a user tries to place a new call, but the length of delay between going off-hook and receiving dial tone is excessive enough to cause a reaction in the user (such as complaining to the system administrator or questioning whether the system is down or the phone is broken, for example). Cisco Unified Communications Manager uses a complex algorithm to constantly monitor the system to anticipate when such latency could occur.
When the delay to dial tone is within the guidelines of the call-throttling-related service parameters, Cisco Unified Communications Manager ceases throttling calls by exiting the Code Yellow state and new calls events are again allowed.
Troubleshooting Call Throttling
CCM/SDI and SDL trace files record call throttling events and can provide useful information. Also, you generally will require performance monitoring data for debugging. The Cisco CallManager System Performance object (viewable in the Real-Time Monitoring Tool) includes a counter called ThrottlingSampleActivity, which indicates whether Cisco Unified Communications Manager has calculated a non-zero value for latency and helps you understand how busy the system is. Frequent non-zero values in this counter could indicate a potential overload condition on the system. To try to circumvent the possibility of a Code Yellow event, consider the possible causes of a system overload, such as heavy call activity, low CPU availability to Cisco Unified Communications Manager, routing loops, disk I/O limitations, disk fragmentation or other such events, and begin to investigate those possibilities.
Generally, repeated call throttling events require assistance from the Cisco Technical Assistance Center (TAC). TAC will likely request these trace files for closer examination.
•Service Parameters Configuration, Cisco Unified Communications Manager Administration Guide