Guest

Cisco Services Modules

Release Note for the Cisco Application Control Engine Module (Software Version A2(2X))

Table Of Contents

Release Note for the Cisco Application Control Engine Module

Contents

Supervisor Engine and Cisco IOS Support for the ACE Module

Virtual Switching System Support

ACE Module Troubleshooting Wiki

New Software Features in Version A2(2.1)

Configuring the ACE to Ignore Authentication Failures Due to CDP Errors

Configuring Persistence with Load Balancing on Each HTTP Request

Using the "\xST" Metacharacter in Regular Expressions for Layer 4 Generic Data Parsing

Overview

"\xST" Metacharacter Regex Usage Considerations

Configuration Examples

New Software Features in Version A2(2.0)

Displaying the Layer 7 Match HTTP URL Statement Hit Counts Feature

Configuring KAL-AP Tags per VIP Address Feature

Configuring the VIP Address Match Statement

Associating a KAL-AP Tag to a VIP Class Map

Displaying the Load Information for a VIP KAL-AP Tag

Bulk Importing of SSL Certificates and Key Pair Files

Rejecting Server Certificates Because of Expired CRL

Using CRLs for Server Authentication

Configuring Downloaded CRLs for Server Authentication

Configuring Downloaded CRLs through LDAP for Client and Server Authentication

Displaying Detailed CRL-Downloading Statistics

System Log Messages

New Syslog Messages

253011

305009

305010

305011

305012

Revised Syslog Messages

253003

253004

253006

Features in Software Version A2(1.1) through A2(1.3)

Configuring the Reverse IP Stickiness Feature

Overview of Reverse IP Stickiness

Configuration Requirements and Restrictions

Configuring Reverse IP Stickiness

Displaying Reverse IP Sticky Status and Statistics

Reverse IP Stickiness Configuration Examples

Configuring the Switch Mode Feature

ACE Operating Considerations

Software Version A2(2.2) Resolved Caveats and Open Caveats

Software Version A2(2.2) Resolved Caveats

Software Version A2(2.2) Open Caveats

Software Version A2(2.1) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(2.1) Resolved Caveats

Software Version A2(2.1) Open Caveats

Command Changes in Software Version A2(2.1)

Displaying Detailed CRL-Downloading Statistics

System Log Messages

New Syslog Message

Revised Syslog Messages

Software Version A2(2.0) Resolved and Open Caveats

Software Version A2(2.0) Resolved Caveats

Software Version A2(2.0) Open Caveats

Command Changes from Software Version A2(1.1) to A2(2.0)

Available ACE Licenses

Ordering an Upgrade License and Generating a License Key

Upgrading Your ACE Software

Changing the Admin Password

Changing the www User Password

Checking Your Configuration for FT Priority and Preempt

Creating a Checkpoint

Updating Your Application Protocol Inspection Configurations

Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration

Before You Begin

Downgrade Procedure

ACE Documentation Set

Obtaining Documentation and Submitting a Service Request


Release Note for the Cisco Application Control Engine Module


Release: August 31, 2009
Revised: September 23, 2009


Note The most current Cisco documentation for released products is available on Cisco.com.


Contents

This release note applies to the following software versions for the Cisco Application Control Engine Module (ACE), models ACE10 (ACE10-6500-K9) and ACE20 (ACE20-MOD-K9).

A2(2.2)

A2(2.1)

A2(2.0)

It also includes new features and command changes from software version A2(1.1) to A2(2.0). For information on the ACE module features and configuration details, see the ACE documentation located at:

http://www.cisco.com/en/US/products/ps6906/tsd_products_support_model_home.html

This release note contains the following sections:

Supervisor Engine and Cisco IOS Support for the ACE Module

Virtual Switching System Support

ACE Module Troubleshooting Wiki

New Software Features in Version A2(2.1)

New Software Features in Version A2(2.0)

Features in Software Version A2(1.1) through A2(1.3)

ACE Operating Considerations

Software Version A2(2.2) Resolved Caveats and Open Caveats

Software Version A2(2.1) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(2.0) Resolved and Open Caveats

Command Changes from Software Version A2(1.1) to A2(2.0)

Available ACE Licenses

Ordering an Upgrade License and Generating a License Key

Upgrading Your ACE Software

Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration

ACE Documentation Set

Obtaining Documentation and Submitting a Service Request

Supervisor Engine and Cisco IOS Support for the ACE Module

Table 1 and Table 2 summarize the supervisor engine model and Cisco IOS version support for the ACE module in the Catalyst 6500 series switch and the Cisco 7600 series router, respectively.

Table 1 Supervisor Engine and Cisco IOS Support for the ACE Module in a Catalyst 6500 Series Switch with a Multilayer Switch Feature Card (MSFC3)

Supervisor Engine Model
Minimum Required IOS Version
Other IOS Version Support

WS-SUP720

12.2(18)SXF4 (or later)

12.2(33)SXH (or later), 12.2(33)SXI1 (or later)

WS-SUP720-3B

WS-SUP720-3BXL

VS-S720-10G-3C

12.2(33)SXH (or later)

VS-S720-10G-3CXL

1 Minimum required IOS version for VSS support. See the Virtual Switching System Support section.


Table 2 Supervisor Engine, Route Switch Processor (RSP), and Cisco IOS Support for the ACE Module in a Cisco 7600 Series Router with an MSFC3

Supervisor Engine or RSP
Minimum Required IOS Version
Other IOS Version Support

WS-SUP720

12.2(18)SXF4 (or later)

12.2(33) SRB (or later)

Not supported: 12.2(33)SXH1

WS-SUP720-3B

WS-SUP720-3BXL

RSP720

12.2(33)SRC (or later)

None

1 Cisco IOS release 12.2(33)SXH runs only on the Catalyst 6500 series switch. Therefore, the Supervisor 720-10GE engines are not supported in the Cisco 7600 series router.


For more information about Cisco IOS releases, see the Release Notes for Cisco IOS Release 12.2SXF and Rebuilds and the Release Notes for Cisco IOS Release 12.2(33)SXH and Later Releases.

Virtual Switching System Support

The ACE10 and the ACE20 running ACE software version A2(1.2) or later and installed in a Catalyst 6500 series switch running IOS software version 12.2(33)SXI or later support the Virtual Switching System (VSS). VSS is a system virtualization technology that allows the pooling of multiple Catalyst 6500 switches into a single virtual switch for increased operational efficiency by simplifying the network. Inter-chassis Supervisor switchover (SSO) boosts non-stop communication. For more information about VSS, see the Cisco IOS Version 12.2(33)SXI Configuration Guide.

ACE Module Troubleshooting Wiki

The ACE documentation set now includes the ACE Module Troubleshooting Wiki. This wiki is a collaborative site that describes the basic procedures and methodology to assist you in troubleshooting the most common problems that you may encounter while you are operating your ACE.

As a registered user of Cisco.com, we strongly encourage you to add content to this site in the form of troubleshooting tips, procedures, or even entire sections. When you add content to the site, you should adhere to the format that has been established for the wiki. To access the ACE Module Troubleshooting Wiki on Cisco DocWiki, click the following URL:

http://docwiki.cisco.com/wiki/Cisco_Application_Control_Engine_(ACE)_Module_Troubleshooting_ Guide,_Release_A2(x)

New Software Features in Version A2(2.1)

The A2(2.1) software release provides the following new features:

Configuring the ACE to Ignore Authentication Failures Due to CDP Errors

Configuring Persistence with Load Balancing on Each HTTP Request

Using the "\xST" Metacharacter in Regular Expressions for Layer 4 Generic Data Parsing

Configuring the ACE to Ignore Authentication Failures Due to CDP Errors

By default, when you configure the crl best-effort command for client or server certificate revocation checks and the ACE detects CRL distribution point (CDP) errors in the presented certificates or errors occur during a CRL download, the ACE rejects the SSL connection.

Per CSCsz83339, the new cdp-errors ignore command allows you to configure an SSL parameter map that ignores CDP errors when the crl best-effort command is configured. When you configure the cdp-errors ignore command, the ACE allows SSL connections when it detects CDP errors in the presented certificates or it could not download a valid certificate revocation list (CRL) from valid CDPs on the certificates. The syntax for this command in parameter map SSL configuration mode is as follows:

cdp-errors ignore

For example, to configure the ACE to ignore CDP errors, enter:

host1/Admin(config)# parameter-map type ssl PARAMMAP_SSL
host1/Admin(config-parammap-ssl)# cdp-errors ignore

When you configure the SSL parameter map, you associate it with the SSL proxy server service by using the ssl advanced-options command in SSL proxy configuration mode.

To reset the default behavior where the ACE rejects an SSL connection when CDP errors occur, use the no form of the cdp-errors ignore command. For example, enter:

host1/Admin(config-parammap-ssl)# no cdp-errors ignore

To display the number of times that the ACE ignored CDP errors in the presented SSL certificate and allowed the SSL connection, use the show crypto cdp-errors command. This command displays the output of the Best Effort CDP Errors Ignored field.

Configuring Persistence with Load Balancing on Each HTTP Request

The persistence-rebalance command does not load balance successive GET requests on the same TCP connection. Per CSCsy21634, the new strict option for this command allows you to configure the ACE to load balance each subsequent GET request on the same TCP connection independently. This feature allows the ACE to load balance each HTTP request to a potentially different Layer 7 class and/or real server.

By default, persistence rebalance is disabled. To enable the strict persistence rebalance feature, use the persistence-rebalance strict command in HTTP parameter-map configuration mode. The syntax of this command is as follows:

persistence-rebalance strict

For example, to enable the strict persistence rebalance feature, enter:

host1/Admin(config)# parameter-map type http http_parameter_map
host1/Admin(config-parammap-http)# persistence-rebalance strict

To reset persistence to the default setting of disabled, enter:

host1/Admin(config-parammap-http)# no persistence-rebalance

To change to persistence rebalance behavior that does not load balance successive requests to the same connection, use the persistence-rebalance command.

Using the "\xST" Metacharacter in Regular Expressions for Layer 4 Generic Data Parsing

This section describes the use of the new "\xST" metacharacter for regular expressions that are used as part of Layer 4 generic data parsing.

It includes the following topics:

Overview

"\xST" Metacharacter Regex Usage Considerations

Configuration Examples

Overview

The "\xST" (STop) metacharacter is now available in software version A2(2.1) for all regular expressions (regexes) that are supported by the ACE. This new metacharacter has been provided for specific use cases that utilize the maximum parse length to terminate parsing. However, the "\xST" metacharacter is specifically designed for use by applications that involve the generic data parsing of a Layer 4 payload.

If you intend to use the "\xST" metacharacter for regex matches on packets from protocols, we recommend that you use this metacharacter only for the following protocols in the generic data parsing of a Layer 4 payload:

SSL session-ID stickiness—To perform sticky hashing on the initial packets in an SSL handshake, allowing the ACE to stick the same client to the same SSL server based on the SSL session ID.

Financial Information eXchange (FIX) type `A' Logon message—To define load-balancing criteria while setting up the outbound path of a connection.

In earlier releases of the ACE software, without the ability to include the "\xST" metacharacter in regexes, there are certain SSL session-id and FIX packets that may get stuck in the ACE HTTP engine and eventually time out the connection. The inclusion of the "\xST" metacharacter will now aid the ACE in properly load-balancing SSL session-id and FIX packets.

The "\xST" metacharacter has been added to software version A2(2.1) per CSCsh04655.

"\xST" Metacharacter Regex Usage Considerations

The new "\xST" metacharacter has the following usage guidelines related to its inclusion in regex matching:

If the input matches a regex pattern that includes the "\xST" metacharacter, the regex engine will halt upon finding the character directly next to the '\xST' in the regex string (2nd '\x01' in the match statement).

No additional input data will be considered by the ACE once the matching pattern is seen which may affect other regexes that are configured elsewhere in the policy. In this case, the "\xST" metacharacter should be used only once in the policy.

The "\xST" metacharacter should only be used at the end of a regex pattern and not at the beginning. In this case, the ACE will display the "Error: Invalid regular expression" error message.

The "\xST" metacharacter should not be added directly after a * wildcard match. For example, "abc.*\xST" would not be a recommended regex.

Configuration Examples

The following configuration examples show the use of the "\xST" metacharacter in two very specific regexes:

SSL session-ID Stickiness Configuration Example

parameter-map type generic SESSID-PARAM

set max-parse-length 76

sticky layer4-payload SESSID-STICKY

serverfarm SF1

response sticky

layer4-payload offset 43 length 32 begin-pattern "(\x20|\x00\xST)"

FIX Protocol Configuration Example

sticky layer4-payload FIX-STICKY
  serverfarm FIX-SF1
  layer4-payload begin-pattern "\x0149=" end-pattern "\x01"

class-map type generic match-all FIX-CM
  2 match layer4-payload regex ".*\x0110=...\x01\xST"

New Software Features in Version A2(2.0)

The A2(2.0) software release, which includes any maintenance releases since A2(1.0), provides the following new features:

Displaying the Layer 7 Match HTTP URL Statement Hit Counts Feature

Configuring KAL-AP Tags per VIP Address Feature

Bulk Importing of SSL Certificates and Key Pair Files

Rejecting Server Certificates Because of Expired CRL

Using CRLs for Server Authentication

Configuring Downloaded CRLs for Server Authentication

Configuring Downloaded CRLs through LDAP for Client and Server Authentication

Displaying Detailed CRL-Downloading Statistics

System Log Messages

Displaying the Layer 7 Match HTTP URL Statement Hit Counts Feature

The Layer 7 match HTTP URL statement hit count feature allows you to display the number of times that a connection is established (hit count) based on match HTTP URL statements for a class map in a Layer 7 HTTP policy map. The show service-policy url-summary command displays this information. The syntax of this command is as follows:

show service-policy [policy_name [class-map class_name]] url-summary

The options are as follows:

policy_name—(Optional) Name of an existing Layer 3 and Layer 4 HTTP policy map. Enter an unquoted text string with no spaces. If you do not enter a policy map name with this command, the ACE displays the match URL statement hit counts for all class maps in L7 HTTP policy maps.

class-map class_name—(Optional) Displays the statement hit counts for the specified class map associated with the policy. Enter the name as an unquoted text string with no spaces.

For example, to display the hit count for the match HTTP URL statements for all class maps in all policy maps, enter the following command:

host1/Admin# show service-policy url-summary

Table 3 describes the fields in the show service-policy url-summary command output.

Table 3 Field Descriptions for the show service-policy url-summary Command Output 

Field
Description

Service Policy

Unique identifier of the policy map.

L3-Class

Name of the Layer 3 class map associated with the service policy.

L7-Class

Identifier of the Layer 7 class map.

match http url

The HTTP URL match statement.

hit

The number of times that a connection is established based on a specific URL match statement.

Note The URL hit counter is per match statement per load-balancing Layer 7 policy. If you are using the same combination of Layer 7 policy and class maps with URL match statements in different VIPs, the count is combined. If the ACE configuration exceeds 64K URL and load-balancing policy combinations, this counter displays NA.


Configuring KAL-AP Tags per VIP Address Feature

A keepalive-appliance protocol (KAL-AP) on the ACE allows communication between the ACE and the Global Site Selector (GSS), which sends KAL-AP requests, to report the server states and loads for global-server load-balancing (GSLB) decisions. The ACE uses KAL-AP through a UDP connection to calculate weights and provide information for server availability to the KAL-AP device. The ACE acts as a server and listens for KAL-AP requests. When KAL-AP is initialized on the ACE, the ACE listens on the standard 5002 port for any KAL-AP requests. You cannot configure any other port.

The ACE supports VIP-based and tag-based KAL-AP probes. Previously, the ACE supported only tag-based KAL-AP for domains associated with VIP addresses. Through the domain, you could associate multiple VIP addresses with a tag with a maximum of 64 KAL-AP domain tags per context (see the Cisco Application Control Engine Module Server Load-Balancing Guide).

The KAL-AP tags per VIP address feature allows you to associate a KAL-AP tag with a VIP address in a policy map configuration. You can configure multiple VIP addresses to a tag or a VIP address to multiple tags. The ACE supports 4,096 VIP tags.

For information on configuring a VIP KAL-AP tag and displaying its load information, see the following sections:

Configuring the VIP Address Match Statement

Associating a KAL-AP Tag to a VIP Class Map

Displaying the Load Information for a VIP KAL-AP Tag


Note For the domain load calculation, the ACE considers the Layer 3 class map, server farm, and real server objects. All other objects under the domain are ignored during the calculation. For the ACE A2(2.0) release, the calculation of the Layer 3 class-map has changed. Previously, the calculation considered each VIP address that is configured in the class map. A VIP-based KAL-AP calculation is run on each address. Now, the calculation consider all Layer 3 rules (a Layer 3 class map within a Layer 3 policy map) defined by the class map and sums up the total number of servers and the number of servers in the Up state. After determining these sums, the ACE multiplies them by the number of VIP addresses configured in the class map.


Configuring the VIP Address Match Statement

Before you configure the VIP KAL-AP tag, configure a Layer 3 class map that contains a VIP address match statement. You can define a 3-tuple flow of VIP address, protocol, and port as matching criteria by using the match virtual-address command in class map configuration mode. You can configure multiple match criteria statements to define the VIP for server load balancing. The syntax of this command is as follows:

[line_number] match virtual-address vip_address {[mask] | any | {tcp | udp {any | eq port_number | range port1 port2}} | protocol_number}

For detailed information on the keywords and arguments for this command, see the Cisco Application Control Engine Module Server Load-Balancing Guide.


Note For KAL-AP, the ACE verifies whether the VIP addresses are active in all Layer 3 class maps that are configured with the addresses. It ignores all other protocol-specific information for the VIP addresses.


For example, to create a class map VIP-20 that matches traffic destined to VIP address 10.10.10.10 with a wildcard value for the IP protocol value (TCP or UDP), enter the following command:

host1/Admin(config)# class-map VIP-20
host1/Admin(config-cmap)# match virtual-address 10.10.10.10 any

Associating a KAL-AP Tag to a VIP Class Map

After you configure a Layer 3 class map that contains a KAL-AP VIP address match statement, you can associate a KAL-AP tag with the address in the class map by using the kal-ap-tag command in policy map class configuration mode. The syntax for this command is as follows:

kal-ap-tag tag_name

The tag_name is the name of the KAL-AP tag. Enter the name as an unquoted text string with no spaces and a maximum of 76 alphanumeric characters.

Note the following restrictions:

You cannot associate the same tag name to more than one Layer 3 class map.

You cannot associate the same tag name to a domain and a Layer 3 class map.

You cannot configure a tag name for a Layer 3 class map that already has a tag configuration as part of a different Layer 3 policy map configuration, even if it is the same tag name.

For example, to associate the VIP-20 class map with the l3_policy20 policy map by using the class command in policy map configuration mode and access policy class configuration mode, enter the following command:

host1/Admin(config)# policy-map multi-match l3_policy20
host1/Admin(config-pmap)# class VIP-20
host1/Admin(config-pmap-c)#

To associate the KAL-AP-TAG2 tag with the class map, enter the following command:

host1/Admin(config-pmap-c)# kal-ap-tag KAL-AP-TAG2

To remove the KAL-AP-TAG2 tag from the class map, enter the following command:

host1/Admin(config-pmap-c)# no kal-ap-tag

Displaying the Load Information for a VIP KAL-AP Tag

To display the latest load information for a VIP tag name provided to the KAL-AP request, use the show kalap udp load command in Exec mode. The syntax of the command to display VIP tag information is as follows:

show kalap udp load {all | vip tag name}

The keywords and arguments are as follows:

all—Displays the latest load information for all VIP addresses, VIP tags, and domains configured on the ACE.

vip tag name—Displays the latest load information for the specified VIP tag name.

Table 4 list the field and output descriptions for the show kalap udp load all command.

Table 4 Field Output Descriptions for the show kalap udp load all Command 

Field
Description

VIP-Addr

VIP address of the KAL-AP request based on a VIP address.

VIP Tag Name

Tag name for a KAL-AP request based on a VIP tag and its associated VIP address.

Domain Name

Name of the domain for a KAL-AP request.

VIP

VIP address for the VIP tag or domain KAL-AP request.

Port

Port number for the KAL-AP request.

Load Value

Load number that the ACE calculates. The number is from 0 to 255 and reports the server availability of the VIP to the KAL-AP device. A load value of 0 indicates that the VIP address is not available. A load value of 2 indicates that the VIP is least loaded and a load value of 255 indicates that the VIP is fully loaded. A load value of 1 is reserved to indicate that the VIP is offline and not available for use.

Time Last Updated

Time when the KAL-AP request occurred.


For example, to display the latest load information for all VIP addresses, domains, and VIP tags, enter the following command:

host1/Admin# show kalap udp load all

To display the latest load information to the KAL-AP request for the VIP KAL-AP-TAG2 tag, enter the following command:

host1/Admin# show kalap udp load vip tag KAL-AP-TAG2

Bulk Importing of SSL Certificates and Key Pair Files

The bulk import feature allows you to import multiple SSL certificates and key-pair files at the same time. Because this feature imports files with the names that they have on the remote server, consider the following:

The ACE fetches all files on the remote server that matches the wildcard criteria. However, it imports only files with names that have a maximum of 39 characters. If the name of a file exceeds 40 characters, the ACE does not import the file and discards it.

If you attempt to import a file that has the same filename of an existing local file, the ACE does not overwrite the existing file. Before importing the updated file, you must either rename the imported file or delete the local file.

The crypto import command has been expanded to include a bulk keyword and its options and arguments. The syntax of this command is as follows:

crypto import [non-exportable] bulk sftp [passphrase passphrase] ip_addr username remote_path

The keywords, options, and arguments are as follows:

non-exportable—(Optional) Marks the imported file as nonexportable, which means that you cannot export the file from the ACE.

bulkSpecifies the importing of multiple certificate or key pair files simultaneously.

sftp—Specifies the Secure File Transfer Protocol file transfer process.

passphrase passphrase—(Optional) Indicates that the file was created with a passphrase, which you must submit with the file transfer request in order to use the file.The passphrase pertains only to encrypted PEM files and PKCS files. The passphrase should apply to all files being imported.

ip_addr—IP address of the remote server. Enter an IP address in dotted-decimal notation (for example, 192.168.12.15).

username—Username required to access the remote server. When you execute the command, the ACE prompts you for the password of the username on the remote server. Enter a name with a maximum of 64 characters. Do not include spaces or the following special characters:

;<>\|`@$&()

remote_path—Remote path to the certificate or key pair files that reside on the remote server. The ACE fetches only files specified by the path; it does not recursively fetch remote directories. Enter a filename path including wildcards (for example, /remote/path/*.pem). The ACE supports POSIX pattern matching notation, as specified in section 2.13 of the "Shell and Utilities" volume of IEEE Std 1003.1-2004. This notation includes the "*," "?" and "[" metacharacters.

To fetch all files from a remote directory, specify a remote path that ends with a wildcard character (for example, /remote/path/*). Do not include spaces or the following special characters:

;<>\|`@$&()


Note After the crypto import bulk command initially executes, pressing Ctrl-C may not cancel it.

The ACE does not a execute any crypto commands or the show crypto commands in Table 8 at the same time. See Table 8 for more information.


For example, to import all files from an SFTP server., enter the following command:

host1/Admin# crypto import bulk sftp 1.1.1.1 JOESMITH /USR/KEYS/*
Initiating bulk import. Please wait, it might take a while...
Connecting to 1.1.1.1...
JOESMITH@1.1.1.1's Password: password
...
Bulk import complete. Summary:
Network errors:  										0
Bad file URL: 										0
Specified local files already exists:										0
Invalid file names: 										1
Failed reading remote files: 										5
Failed reading local files: 										0
Failed writing local files: 										0
Other errors: 										0
Successfully imported: 										10
host1/Admin#

For the complete syntax of and more information about the crypto import command, see the Cisco Application Control Engine Module SSL Configuration Guide for software version A2(1.0).

Rejecting Server Certificates Because of Expired CRL

When you configure Certificate Revocation Lists (CRLs) on the ACE for server authentication, as described in the "Using CRLs for Server Authentication" section, the CRLs contain an update field that specifies the date when a new version will be available. By default, the ACE continues to use CRLs that contains an update field with an expired date and, thus, does not reject incoming server certificates using the CRL.

To configure the ACE to consider a server certificate as revoked when the CRL in use has expired, use the expired-crl reject command in parameter map SSL configuration mode. The syntax of this command is as follows:

expired-crl reject

For example, enter the following command:

host1/Admin(config-parammap-ssl)# expired-crl reject

To reset the default behavior of the ACE of not considering a server certificate as revoked after the CRL in use has expired, enter the following command:

host1/Admin(config-parammap-ssl)# no expired-crl reject

Using CRLs for Server Authentication

By default, the ACE does not use certificate revocation lists (CRLs) during server authentication. You can configure the SSL proxy service to use a CRL in one of the following ways:

The ACE can scan each server certificate for the service to determine if it contains a CDP pointing to a CRL in the certificate extension and then retrieve the CRL from that location if the CDP is valid.

You can manually configure the CRL to download to the ACE (see the "Configuring Downloaded CRLs for Server Authentication" section).


Note By default, the ACE does not reject server certificates when the CRL in use has passed its update date. To configure the ACE to reject certificates when the CRL is expired, use the expired-crl reject command. For more information, see the "Rejecting Server Certificates Because of Expired CRL" section.


You can determine which CRL information to use for server authentication by using the crl command in SSL proxy configuration mode. The syntax of this command is as follows:

crl crl_name | best-effort

The argument and keyword are as follows:

crl_name—Name that you assigned to the CRL when you downloaded it with the configuration mode crypto crl command. See the "Configuring Downloaded CRLs for Server Authentication" section.

best-effort—Specifies that the ACE scans each server certificate to determine if it contains a CDP pointing to a CRL in the certificate extension and then retrieves the CRLs from that location, if the CDP is valid.

For example, to enable the CRL1 CRL for server authentication on an SSL proxy service, enter the following command:

host1/Admin(config-ssl-proxy)# crl CRL1

When the ACE accepts a server certificate in the downloaded CRL database, a successful SSL connection to an SSL real server increments the following show stats crypto client counters:

Total SSL server authentications

SSL static CRL lookups

To scan the server certificate for CRL information, enter the following command:

host1/Admin(config-ssl-proxy)# crl best-effort

When the ACE accepts a server certificate on a best-effort-CRL-enabled connection and the certificate is not found in the downloaded CRL database, a successful SSL connection to an SSL real server increments the following show stats crypto client counters:

Total SSL server authentications

SSL best effort CRL lookups

After the certificate is validated and cached in the ACE, subsequent SSL connections without session reuse to the same SSL server increments the following show stats crypto client counters:

Total SSL server authentications

SSL best effort CRL lookups

SSL CRL lookup cache hits

SSL authentication cache hits

If a valid non-expired CRL is cached in the ACE, no CRL lookups are performed and the following show stats crypto client counters will not increment together by the same connection:

SSL best effort CRL lookups

SSL CRL lookup cache hits

When the SSL connection to the SSL real server fails because of a revoked server certificate, the following show stats crypto client counters increment:

SSL alert CERTIFICATE_REVOKED sent

Total SSL server authentications

Failed SSL server authentications

SSL best effort CRL lookups or SSL static CRL lookups

To disable the use of a downloaded CRL during server authentication, enter the following command:

host1/Admin(config-ssl-proxy)# no crl CRL1

To disable the use of server certificates for CRL information during server authentication, enter the following command:

host1/Admin(config-ssl-proxy)# no crl best-effort

Configuring Downloaded CRLs for Server Authentication

You can configure a CRL that the ACE downloads on the SSL proxy service for server authentication. If the service is not configured on a policy map or the policy map is not active, the ACE does not download the CRL. The ACE downloads the CRL under the following conditions:

When you first configure the CRL and apply it to an active Layer 4 policy map as an action. See the Cisco Application Control Engine Module SSL Configuration Guide for software version A2(1.0).

When you reload the ACE.

When the NextUpdate arrives, as provided within the CRL itself, the ACE reads this information and updates the CRL based on it. The ACE downloads the updated CRL upon the next server authentication request.

You can configure a maximum of four CRLs per context. After you configure the CRL, assign it to an SSL proxy service for server authentication (see the "Using CRLs for Server Authentication" section).

The ACE translates the hostnames within the CRLs to IP addresses using a Domain Name System (DNS) client that you configure. For details about configuring a DNS client, see the Cisco Application Control Engine Module SSL Configuration Guide for software version A2(1.0).

To configure a downloaded CRL, use the crypto crl command in configuration mode. The syntax of this command is as follows:

crypto crl crl_name url

The arguments are as follows:

crl_name—Name that you want to assign to the CRL. Enter an unquoted text string with a maximum of 64 alphanumeric characters.

url—URL where the ACE retrieves the CRL; the CRL distribution point (CDP). Enter the URL full path including the CRL filename in an unquoted text string with a maximum of 255 alphanumeric characters. Both HTTP and LDAP URLs are supported. Start the URL with the http:// prefix or the ldap:// prefix.

The ldap:/// prefix is not considered a valid LDAP CRL link in the CDP portion of the server certificate. Valid formats for LDAP URLs are as follows:

ldap://10.10.10.1:389/dc=cisco,dc=com?o=bu?certificateRevocationList

ldap://10.10.10.1/dc=cisco,dc=com?o=bu?certificateRevocationList

ldap://ldapsrv.cisco.com/dc=cisco,dc=com?o=bu?certificateRevocationList

ldap://ldapsrv.cisco.com:389/dc=cisco,dc=com?o=bu?certificateRevocationList

To use a question mark (?) character as part of the URL, press Ctrl-v before entering it. Otherwise the ACE interprets the question mark as a help command.

When attempting to download a CRL:

The ACE considers only the first four CDPs. From the CDPs obtained from certificate, the ACE only considers valid and complete CDPs for the downloading of the CRLs. If a CDP leads to the successful downloading of the CRL, ACE does not consider the subsequent CDPs for CRL downloads.

If none of the first four CDPs present in the certificate are valid to proceed with the downloading of the CRL, the ACE considers the certificate as revoked unless you configured the authentication-failure ignore command in parameter map SSL configuration mode.

If the ACE fails to download a CRL after trying four valid CDPs, the ACE aborts its initiated SSL connection unless you configured the authentication-failure ignore command in parameter map SSL configuration mode.

The ACE skips malformed CDPs and processes subsequent CDPs. To display CDP error statistics including the number of malformed CDPs, use the show crypto cdp-errors command.

For example, to configure a CRL that you want to name CRL1 from http://crl.verisign.com/class1.crl, enter the following command:

host1/Admin(config)# crypto crl CRL1 http://crl.verisign.com/class1.crl

To remove the CRL, enter the following command:

host1/Admin(config)# no crypto crl CRL1

Configuring Downloaded CRLs through LDAP for Client and Server Authentication

The A2(2.0) release supports CRL download through the LDAP protocol in client and server authentication. You can configure CRL downloads through LDAP in the following two ways:

The ACE can scan each uncached certificate for the CDP. If the CDP has an ldap:// based URL, it uses the URL to download the CRL to the ACE.

You can configure the ldap:// CDP on the ACE and the CRL can be downloaded manually for revocation check on the certificate.

To configure a downloaded CRL, use the crypto crl command in configuration mode. This command now supports an LDAP URL. The syntax of this command is as follows:

crypto crl crl_name url

The arguments are as follows:

crl_name—Name that you want to assign to the CRL. Enter an unquoted text string with a maximum of 64 alphanumeric characters.

url—URL where the ACE retrieves the CRL; the CRL distribution point (CDP). Enter the URL full path including the CRL filename in an unquoted text string with a maximum of 255 alphanumeric characters. Both HTTP and LDAP URLs are supported. Start the URL with the http:// prefix or the ldap:// prefix.

The ldap:/// prefix is not considered a valid LDAP CRL link in the CDP portion of the certificate. Valid formats for LDAP URLs in the certificates are as follows:

ldap://10.10.10.1:1202/dc=cisco,dc=com?bu?certificateRevocationList

ldap://10.10.10.1/dc=cisco,dc=com?bu?certificateRevocationList

ldap://ldap_crl_server1:1921/dc=cisco,dc=com?bu?certificateRevocationList

ldap://ldap_crl_server/dc=cisco,dc=com?bu?certificateRevocationList

To use a question mark (?) character as part of the URL, press Ctrl-v before entering it. Otherwise the ACE interprets the question mark as a help command.

Note that the hostname in ldap:// links are resolved using DNS configurations. LDAP uses TCP port 389. If the LDAP server that publishes the CRL listens on a non-standard LDAP port, then a non-standard LDAP port needs to be configured in the CDP.

For detailed CRL download statistics, see the "Displaying Detailed CRL-Downloading Statistics" section.

Figure 1 illustrates a sample configuration for CRL downloading through LDAP in client authentication.

Figure 1 CRL Download through the LDAP Protocol

The following example is the configuration of the authentication group with the root certificate that signed the client certificate:

crypto authgroup root_ca_pool
cert root-cert-2.cer 

The following example provides the configuration for the ldap:// based CDP URL:

crypto crl win2003crl1 
ldap://windows2003-srv.win2003.cisco.com/CN=root-ca(2),CN=windows2003-srv,CN=CDP,CN=Public
%20Key%20Services,CN=Services,CN=Configuration,DC=win2003,DC=cisco,DC=com?certificateRevoc
ationList?base?objectClass=cRLDistributionPoint 

access-list capture-acl line 8 extended permit tcp any any  
access-list permit-http line 8 extended permit tcp any any eq https 

The following example provides the DNS configuration for the ACE to successfully resolve the hostname in the ldap:// URL during the CRL download:

ip domain-lookup
ip domain-name win2003.cisco.com
ip name-server 100.1.1.147 

rserver host real1
ip address 100.1.1.122
inservice 

ssl-proxy service proxy
key proxy_key_1024.key
cert proxy_cert_1024.cer
authgroup root_ca_pool
crl win2003crl1 

serverfarm host sfarm1
rserver real1 80
inservice

class-map match-any ssl-terminate
3 match virtual-address 50.1.1.100 tcp eq https
class-map type http loadbalance match-all urlclass1
2 match http url .* 

policy-map type loadbalance first-match l7map
class urlclass1
serverfarm sfarm1 
policy-map multi-match p1
class ssl-terminate
loadbalance vip inservice
loadbalance policy l7map
loadbalance vip icmp-reply
ssl-proxy server proxy 

interface vlan 50
ip address 100.1.1.138 255.255.0.0
no shutdown

interface vlan 200
ip address 50.1.1.254 255.255.0.0
access-group input permit-http
service-policy input p1
no shutdown

Displaying Detailed CRL-Downloading Statistics

To display the detailed statistics for the downloading of a CRL including failure counters, use the show crypto crl name detail command. Table 5 describes the fields displayed by this command.

Table 5 Field Descriptions for the show crypto crl crl_name detail Command 

Field
Description

URL

URL where the ACE downloads the CRL.

Last Downloaded

Last time the ACE downloaded the CRL. If the CRL is configured on an SSL-proxy service on a policy map that is not active or the service is not associated with a policy map, the field displays the "not downloaded yet" message.

Total Number of Download Attempts

Number of times the ACE attempted to download the CRL.

Failed Download Attempts

Numbers of times that the ACE failed to download the CRL.

Successful Loads

Number of times that the ACE successfully loaded the CRL.

Failed Loads

Number of times that the ACE could not load the CRL because of a failure.

Hours since Last Load

Number of hours that elapsed since the ACE last successfully downloaded the CRL. If no successful download has occurred, this field displays NA, not applicable.

No IP Addr Resolutions

Number of times the DNS resolution for the server host address of CRL the failed.

Host Timeouts

Number of download retries to the CRL that had timed out.

Next Update Invalid

Number of times that the next update field of the CRL was invalid.

Next Update Expired

Number of times that the next update field of the CRL was expired.

Bad Signature

Number of times that the signature mismatch for the CRL was detected, with respect to the CA certificate configured for signature verification of the CRL.

CRL Found-Failed to load

Number of times that the ACE could not load the CRL because of the maximum size limitation of 10MB on ACE or the formatting of the CRL was not recognized. The ACE recognizes only DER and PEM encoded CRLs.

File Not Found

Number of times that the server responded that the CRL file was not found at the server.

Memory Outage failures

Number of times that the ACE failed to download the CRL because it temporarily could not provide memory to store the CRL data.

Cache Limit failures

Number of times that the ACE could not load the CRL because the CRL cache was exhausted.

Conn Failures

Number of times that the ACE failed to download the CRL because it could not establish a connection with the server or no server entity was listening on the destination system.

Internal Failures

Number of internal failures in the ACE that hampered downloading the CRL, for example, internal communication failures between components responsible for the downloading the CRL.

Not Eligible for download

Number of times that the CRL was found ineligible for downloading because the following conditions:

The downloading of the same CRL is in progress.

The CRL has already been loaded successfully earlier and has not expired yet.

HTTP Read Failures

Number of times that the ACE encountered an error when downloading the CRL because it could not read data on the connection established with server.

HTTP Write failures

Number of times that the ACE encountered an error when downloading the CRL because it could not write the CRL download request from the connection established with the server.


System Log Messages

Software version A2(2.0) introduces the following new or revised system log (syslog) messages.

New Syslog Messages

253011

Error Message    %ACE-2-253011: Crypto file storage failure: All certificates/keys were 
removed. Error: text_string

Explanation    A system failure deleted the SSL services internal database of certificates and keys. The text_string variable is either of the following:

Corrupted certificates/keys metadata found

Out of resources while trying to store certificates/keys metadata

Recommended Action    Contact Cisco TAC and send them the message output. Reimport the certificates and keys to maintain the integrity of the SSL services.

305009

Error Message    %ACE-6-305009: Built {dynamic|static} translation from interface_name 
[(acl-name)]:real_address to interface_name:mapped_address

Explanation    An address translation slot was created. The slot translates the source address from the local side to the global side. In reverse, the slot translates the destination address from the global side to the local side.

Recommended Action    None required.

305010

Error Message    %ACE-6-305010: Teardown {dynamic|static} translation from 
interface_name:real_address to interface_name:mapped_address duration time

Explanation    An address translation slot was deleted.

Recommended Action    None required.

305011

Error Message    %ACE-6-305011: Built {dynamic|static} {TCP|UDP|ICMP} translation from 
interface_name:real_address/real_port to interface_name:mapped_address/mapped_port

Explanation    A TCP, UDP, or ICMP address translation slot was created. The slot translates the source socket from the local side to the global side. In reverse, the slot translates the destination socket from the global side to the local side.

Recommended Action    None required.

305012

Error Message    %ACE-6-305012: Teardown {dynamic|static} {TCP|UDP|ICMP} translation 
from interface_name:real_address/{real_port|real_ICMP_ID}to 
interface_name:mapped_address/{mapped_port|mapped_ICMP_ID} duration time

Explanation    An address translation slot was deleted.

Recommended Action    None required.

Revised Syslog Messages

253003

Error Message    %ACE-6-253003: Certificate client_information is signed by an unknown 
CA

Explanation    This message is logged during the SSL handshake when a client attempts to connect with a certificate that was signed by an unknown CA (the certificate is not part of the authgroup for this VIP's SSL proxy). The client_information variable is the subject name of the client certificate.

Recommended Action    None required.

253004

Error Message    %ACE-6-253004: Certificate subject_of_certificate revoked, ssl-proxy: 
proxy_name, reason: reason

Explanation    This message is logged during the SSL handshake when client or server authentication is enabled. The ACE determines that the certificate has been revoked by the CA. The subject_of_certificate variable is the subject field of the certificate. The proxy_name is the name of the SSL proxy service. The reason is the reason for the revocation of the certificate and has one of the following messages:

revoked—The certificate is revoked by the CA.

no workable cdps in cert—The certificate does not have a workable CRL distribution point (CDP). A CDP indicates the location of the CRL in the form of a URL.

crl download failure—The download of the CRL failed.

Recommended Action    None required.

253006

Error Message    %ACE-6-253006: Error peer sent invalid or nonexistent certificate 
subject_of_peer_certificate, reason: reason

Explanation    This message is logged during the SSL handshake when client authentication is enabled. The ACE determines a certificate is invalid or nonexistent. The subject_of_peer_certificate variable is the subject field of the peer certificate. The reason variable is the reason for rejecting the certificate.

Recommended Action    None required.

Features in Software Version A2(1.1) through A2(1.3)

The following features were released in software version A2(1.1) through A2(1.3).

Configuring the Reverse IP Stickiness Feature

This section describes the reverse IP stickiness feature that is used primarily in firewall load balancing (FWLB) to ensure that applications with separate control and data channels use the same firewall for ingress and egress flows for a given connection. It contains the following subsections:

Overview of Reverse IP Stickiness

Configuration Requirements and Restrictions

Configuring Reverse IP Stickiness

Displaying Reverse IP Sticky Status and Statistics

Reverse IP Stickiness Configuration Examples

Overview of Reverse IP Stickiness

Reverse IP stickiness is an enhancement to regular stickiness and is used mainly in FWLB. It ensures that multiple distinct connections that are opened by hosts at both ends (client and server) are load-balanced and stuck to the same firewall. Reverse stickiness applies to such protocols as FTP, RTSP, SIP, and so on where there are separate control channels and data channels opened by the client and the server, respectively.

You configure reverse IP stickiness as an action under a Layer 7 load-balancing policy map by associating an existing IP address sticky group with the policy using the reverse-sticky command. Then you associate the Layer 7 policy map with a Layer 4 multi-match policy map and apply the Layer 4 policy map as a service policy on the ACE interface between the firewalls and the ACE. When incoming traffic matches the policy, the ACE verifies that a reverse IP sticky group is associated with the policy. If the association exists, the ACE creates a sticky entry in the sticky table that maps the opposite IP address (for example, the destination IP address if source IP sticky is configured) to the real server ID, which is the ID of the firewall. To obtain the real ID of the firewall, the ACE uses the encapsulation (encap) ID from the traffic coming from the firewall as a lookup key into the list of real servers in the server farm.


Note The ACE sticky table, which holds a maximum of 4 million entries, is shared across all sticky types, including reverse IP stickiness.


This section contains the following topics:

Symmetric Topology

Asymmetric Topology

Symmetric Topology

A typical firewall load-balancing topology (symmetric) includes two dedicated ACEs with the firewalls positioned between the ACEs. In this scenario, the ACEs are used exclusively for FWLB and simply forward traffic through their host interfaces in either direction. See Figure 2.

The hosts in either VLAN 31 or VLAN 21 can initiate the first connection and the hosts on both sides of the connection can "see" each other directly. Therefore, only catch-all VIPs (with an IP address of 0.0.0.0 and a netmask of 0.0.0.0) are configured on the ACE interfaces.

Figure 2 Typical Symmetric Firewall Load-Balancing Topology for Reverse IP Stickiness

For the network diagram shown in Figure 2, the following steps describe a possible connection scenario with reverse IP stickiness:


Step 1 Host A (a client) initiates an FTP control channel connection to the IP address of Host C (an FTP server).

Step 2 ACE 1 load balances the connection to one of the two firewalls (FW1 or FW2) in the FWS-OUT server farm. ACE 1 is configured with a source IP sticky group that is associated with a policy map, which is applied to interface VLAN 113. This configuration ensures that all connections coming from the same host (or directed to the same host) are load balanced to the same firewall. The ACE creates a sticky entry that maps the IP address of Host A to one of the firewalls.

Step 3 The firewall that receives the packets from ACE 1 forwards them to ACE 2.

Step 4 Assume that a sticky group that is based on the destination IP address is associated with a policy map and is applied to interface VLAN 21. The same sticky group is associated as a reverse sticky group with the policy that is applied to VLAN 111. When it receives the packets, ACE 2 creates a sticky entry in the sticky database based on the source IP address (because the sticky group is based on the destination IP address in this case), which maps the Host A IP address to the firewall in the FWS-IN server farm from which the traffic was received. Then, ACE 2 forwards the packets to the FTP server (Host C) in the server farm.

Step 5 If you have enabled the mac-sticky command on the VLAN 111 interface, ACE 2 forwards return traffic from the same connection to the same firewall from which the incoming traffic was received. The firewall routes the return traffic through ACE 1, which in turn forwards it to the MSFC and from there to the client.

Step 6 Now suppose that Host C (an FTP server) opens a new connection (for example, the corresponding FTP data channel of the previously opened FTP control channel) to the IP address of Host A. Because a sticky group based on destination IP is associated with the policy applied to interface VLAN 21, ACE 2 performs a sticky lookup and finds a valid sticky entry (the one created in Step 4) in the sticky database that allows ACE 2 to load balance the packets to the same firewall that the control connection traversed.

Step 7 The firewall routes the packets through ACE 1, which in turn forwards them to the MSFC and from there to the client (Host A).


Follow these guidelines and observations when you configure reverse IP stickiness:

When reverse IP sticky is enabled, the sticky entry is populated in one direction (for incoming traffic) and looked up in the opposite direction (for outgoing traffic), allowing traffic to flow through the same firewall in both directions.

The example that is described in the steps above is symmetric because it does not matter on which side of the connections that the clients and servers reside. Everything would work in a similar manner if Host C was a client opening the FTP control channel and Host A was a server opening the FTP data channel, assuming that a reverse sticky group was also configured on the ACE 1 VLAN 112 interface. To make reverse IP stickiness work symmetrically, you must apply a reverse sticky group to the ACE interfaces that are associated with the firewall server farm (in this example, VLAN 112 and VLAN 111) and apply the same sticky group as a regular sticky group to the ACE interfaces associated with the hosts (in this example, VLAN 113 and VLAN 21).

In this example, the assumption is to have a regular sticky group based on the source IP associated with the VLAN 113 interface of the ACE 1 module and another sticky group based on the destination IP associated with the VLAN 21 interface of the ACE 2 module (the reverse sticky groups on VLAN 112 and VLAN 111 would be based on the opposite IPs). Everything would work correctly if the regular sticky groups were reversed, that is, the sticky group on VLAN 113 was based on the destination IP and the one on VLAN 21 was based on the source IP, or if both regular sticky groups were based on both the source and the destination IP.

Asymmetric Topology

The following scenario is asymmetric because it cannot work equally in both directions as in the previous scenario. In this setup, one of the load balancers is unknown (Unknown LB) so that it is uncertain whether the load balancer supports reverse sticky. The clients must be on one side of the connection and the servers must be on the other side with the clients opening the first connection to the servers. See Figure 3. In this scenario, the ACE performs only FWLB and forwards traffic to the real servers in the server farm.

Figure 3 Asymmetric Firewall Load Balancing Topology for Reverse IP Stickiness

For the network diagram shown in Figure 3, the following steps describe the sequence of events for establishing a connection with reverse IP stickiness:


Step 1 A client initiates a connection (for example, an FTP control channel connection) to the IP address of one of the servers in the server farm.

Step 2 The Unknown LB load balances the connection to one of the two firewalls in the FWS-OUT server farm. The Unknown LB should, at a minimum, support load balancing based on the source or destination IP address hash predictor. These predictors ensure that all connections coming from the same client (or destined to the same server) are load balanced to the same firewall. Assume in this example that a predictor based on source IP hash is configured in the Unknown LB, so that all traffic coming from the same client will be directed to the same firewall.

Step 3 The firewall that receives the packet forwards it to the ACE.

Step 4 Assume that a sticky group that is based on the destination IP address is associated with a policy map that is applied to interface VLAN 21 using a service policy. The same sticky group is associated as a reverse sticky group with the policy that is applied to VLAN 111. When it receives the packets, the ACE creates a sticky entry in the sticky database based on the source IP address (because the sticky group is based on the destination IP in this case), which maps the Host A IP address to the firewall in the FWS-IN server farm from which the traffic was received. Then, the ACE forwards the packets to the FTP server (Host C) in the server farm.

Step 5 If you have enabled the mac-sticky command on VLAN 111, the ACE forwards the return traffic for the same connection to the same firewall from which the incoming traffic was received. The firewall routes the return traffic through the Unknown-LB, which in turn forwards it to the MSFC and then to the client.

Step 6 Now suppose that the FTP server opens a new connection (for example, the corresponding FTP data channel of the previously opened FTP control channel) to the IP address of the client. Because a sticky group based on the destination IP address is associated with the policy applied to interface VLAN 21, the ACE performs a sticky lookup and finds a valid sticky entry (the one created in Step 4) in the sticky database that allows the ACE to load balance the packets to the same firewall that the control connection traversed.

Step 7 The firewall routes the packet through the Unknown LB, which in turn forwards it to the MSFC and then to the client.


In this scenario, reverse sticky would also work properly under the following conditions:

The sticky group is associated with the policy map as a regular sticky group based on source the IP and applied to the VLAN 21 interface.

The sticky group is associated with the policy map as a reverse sticky group (based on the destination IP address) and applied to the VLAN 111 interface.

The Unknown LB has a predictor based on the hash of the destination IP.

For more information about configuring firewall load balancing, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

Configuration Requirements and Restrictions

Before attempting to configure reverse IP stickiness, be sure that you have met the following configuration requirements and restrictions:

A sticky group of type IP netmask based on source IP, destination IP, or both must be present in your configuration.

The sticky group cannot be a static sticky group.

Once you have associated reverse IP stickiness with a sticky group, you cannot change that sticky group to a static sticky group.

For firewall load balancing, configure the mac-sticky command on the ACE interface that is connected to the firewall.

Configuring Reverse IP Stickiness

To configure reverse IP stickiness, use the reverse-sticky command in policy map loadbalance class configuration mode. The syntax of this command is as follows:

reverse-sticky name

The name argument specifies the unique identifier of an existing IP address sticky group. Enter the name of an existing IP address sticky group as an unquoted text string with no spaces and a maximum of 64 alphanumeric characters.

For example, to configure reverse IP stickiness for a sticky group called DEST_IP_STICKY, enter the following sequence of commands:

host1/Admin(config)# sticky ip-netmask 255.255.255.255 address destination DEST_IP_STICKY
host1/Admin(config-sticky-ip)# serverfarm FWS-IN

host1/Admin(config)# policy-map type loadbalance first-match L7PMAP_TO_REALS
host1/Admin(config-pmap-lb)# class class-default
host1/Admin(config-pmap-lb-c)# forward
host1/Admin(config-pmap-lb-c)# reverse-sticky DEST_IP_STICKY

Displaying Reverse IP Sticky Status and Statistics

Use the following show commands to display the state of the reverse-sticky command and reverse sticky statistics:

show sticky database detail—Provides the reverse entry field that indicates the state (TRUE or FALSE) of reverse IP stickiness for each configured sticky group.

show stats sticky—Provides the Total active reverse sticky entries field that displays the total number of active reverse IP sticky entries in the sticky database.

show service-policy route detail—Provides the reverse sticky group field that displays the name of the sticky group configured for reverse IP stickiness.

Reverse IP Stickiness Configuration Examples

This section contains configuration examples that show how to configure reverse IP stickiness with a symmetric firewall load balancing configuration. These configuration examples correspond with the network diagram in Figure 2. The examples are as follows:

ACE 1 Configuration

ACE 2 Configuration

ACE 1 Configuration

access-list acl1 line 8 extended permit ip any any

rserver host FW1
  ip address 10.10.40.10
  inservice
rserver host FW2
  ip address 10.10.40.20
  inservice

serverfarm host FWS-OUT
  transparent
  rserver FW1
    inservice
  rserver FW2
    inservice

sticky ip-netmask 255.255.255.255 address source SOURCE_IP_STICKY
  serverfarm FWS-OUT

class-map match-all CATCH-ALL-VIP
  2 match virtual-address 0.0.0.0 0.0.0.0 any

policy-map type management first-match MGMT-POLICY
  class class-default
    permit

policy-map type loadbalance first-match LB_PMAP_TO_REALS
  class class-default
    sticky-serverfarm SOURCE_IP_STICKY
policy-map type loadbalance first-match ROUTE_PMAP
  class class-default
    forward
    reverse-sticky SOURCE_IP_STICKY

policy-map multi-match LB
  class CATCH-ALL-VIP
    loadbalance vip inservice
    loadbalance policy LB_PMAP_TO_REALS
policy-map multi-match ROUTE
  class CATCH-ALL-VIP
    loadbalance vip inservice
    loadbalance policy ROUTE_PMAP

service-policy input mgmt-policy

interface vlan 112
  description outside FW vlan
  bridge-group 15
  mac-sticky enable
  access-group input acl1
  service-policy input ROUTE
  no shutdown
interface vlan 113
  description client vlan
  bridge-group 15
  access-group input acl1
  service-policy input LB
  no shutdown

interface bvi 15
  ip address 10.10.40.2 255.255.255.0
  alias 10.10.40.3 255.255.255.0
  no shutdown

ip route 0.0.0.0 0.0.0.0 10.10.40.1

ACE 2 Configuration

access-list acl1 line 8 extended permit ip any any 

rserver host FW1
  ip address 10.10.50.10
  inservice
rserver host FW2
  ip address 10.10.50.20
  inservice

serverfarm host FWS-IN
  transparent
  rserver FW1
    inservice
  rserver FW2
    inservice

sticky ip-netmask 255.255.255.255 address destination DEST_IP_STICKY
  serverfarm FWS-IN

class-map match-all CATCH_ALL_VIP
  2 match virtual-address 0.0.0.0 0.0.0.0 any

policy-map type management first-match mgmt-policy
  class class-default
    permit

policy-map type loadbalance first-match L7PMAP_TO_FWS
  class class-default
    sticky-serverfarm DEST_IP_STICKY
policy-map type loadbalance first-match L7PMAP_TO_REALS
  class class-default
    forward
    reverse-sticky DEST_IP_STICKY

policy-map multi-match L4_TO_FWS
  class CATCH_ALL_VIP
    loadbalance vip inservice
    loadbalance policy L7PMAP_TO_FWS
policy-map multi-match L4_TO_REALS
  class CATCH_ALL_VIP
    loadbalance vip inservice
    loadbalance policy L7PMAP_TO_REALS
service-policy input mgmt-policy

interface vlan 21
  ip address 21.1.1.1 255.255.255.0
  access-group input acl1
  service-policy input L4_TO_FWS
  no shutdown
interface vlan 111
  description inside FW vlan
  ip address 10.10.50.1 255.255.255.0
  mac-sticky enable
  access-group input acl1
  service-policy input L4_TO_REALS
  no shutdown

Configuring the Switch Mode Feature

Use the switch mode feature to change the way that the ACE handles TCP connections that are not destined to a particular VIP and those connections that do not have any policies associated with their traffic. When you enable this feature, the ACE still creates connection objects for those TCP sessions that are not destined to the VIP. The ACE processes these connections as stateless connections, which means that they do not undergo any TCP normalization checks (for example, TCP window, TCP state, TCP sequence number, and other normalization checks).

The ACE also creates stateless connections for non-SYN TCP packets if they satisfy all other configured requirements, for example, ACLs and other policies. This process ensures that a long-lived persistent connection passes through the ACE successfully (even if it times out) by being reestablished by any incoming packet related to the connection.

By default, these stateless connections time out after 2 hours and 15 minutes unless you configure the timeout otherwise. When a stateless connection times out, the ACE does not send a TCP RST packet but instead closes the connection silently. Even though these connections are stateless, the TCP RST and FIN-ACK flags are honored and the connections are closed when the ACE sees these flags in the received packets.

To change the default timeout for these stateless connections, use the set timeout inactivity command in parameter map connection configuration mode. For details about this command, see theCisco Application Control Engine Module Security Configuration Guide.

The SYN cookie feature still operates normally for these stateless connections that are not destined to any VIP.

The default timeout value of 2 hours and 15 minutes is also applicable to the UDP connections that are not destined to any VIP.

To enable the switch mode feature, use the switch-mode command in configuration mode. The syntax of this command is as follows:

switch-mode

For example, to enable the switch mode feature, enter the following command:

host1/Admin(config)# switch-mode

To disable the switch mode feature, enter the following command:

host1/Admin(config)# no switch-mode

ACE Operating Considerations

This section provides the operating considerations for the ACE:

The ACE requires a route back to the client before it can forward a request to a server. If the route back to the client is not present, the ACE cannot establish a flow and drops the client request. Make sure that you configure the appropriate routing to the client network on the ACE VLAN where the client traffic enters the ACE module.

Software version A2(1.0) introduces hardware-assisted SSL (HTTPS) probes. For that reason, the ACE uses the all option for the default SSL version and uses the routing table (which may bypass the real server IP address) to direct HTTPS probes to their destination regardless of whether you specify the routed option in the ip address command. If you are using HTTPS probes in your A1(6.x) configuration with the default SSL version (SSLv3) or without the routed option, you may observe that your HTTPS probes behave differently with version A2(1.x) or higher. For more information about HTTPS probes, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

Additionally, hardware-assisted probes are subject to the same key-pair size limitations as SSL termination. The maximum size of a public key in a server SSL certificate that the ACE can process is 2048 bits. For more information about HTTPS probes, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

By design, if you set the maximum resources for sticky to unlimited using the limit-resource command, the ACE ignores the setting and sets the maximum value to equal-to-min. In addition, the maximum resource value for sticky in the show resource usage command output displays as 0. This behavior occurs because the ACE does not allow sticky resources to become oversubscribed as with other configurable resources. Instead, when the sticky resource usage reaches the minimum value, the ACE ages out older sticky entries in the sticky table and reuses them for new sticky entries.

In software version A2(1.2), the maximum number of match statements per ACE has been increased from 4,096 to 16,384.

The Total Conn-failures counter in the show rserver detail command displays the total number of connection attempts that failed to establish a connection to the real server.

For Layer 4 traffic with normalization on, the count increments if the three-way handshake fails to be established for either of the following reasons:

- An RST comes from the client or the server after a SYN-ACK.

- The server does not reply to a SYN. The connection times out.

For Layer 4 traffic with normalization off, the count does not increment.

For Layer 7 traffic (normalization is always on), the count increments if the three-way handshake fails to be established for either of the following reasons:

- An RST comes from the server after the front-end connection is established.

- The server does not reply to a SYN. The connection times out.

In software version A2(2.0), the ACE supports a maximum of 3,800 certificates and 3,800 key pairs.

In software version A2(2.0), the ACE now supports an SSL filename with a maximum of 39 characters.

When you downgrade the ACE software, the features and commands of the higher release are lost because they are not supported by the lower release.

Per CSCsz87533, the outbound UDP connection may timeout shortly after the ACE receives a RADIUS request, but before it gets the response for this request from the server. This situation can cause the ACE to improperly forward subsequent RADIUS traffic. If the server is not expected to initiate connections through the ACE, we recommend that you apply an inbound ACL on the server interface to block these connections.

In software version A2(2.2), the ACE introduces the STANDBY_WARM and WARM_COMPATIBLE redundancy states to handle any CLI incompatibility issue between peers during the upgrading and downgrading of the ACE software. When you upgrade or downgrade the ACE software in a redundant configuration with different software version, the STANDBY_WARM and WARM_COMPATIBLE states allow the configuration and state synchronization process to continue on a best-effort basis. This basis allows the active ACE to synchronize configuration and state information to the standby even though the standby may not recognize or understand the CLI commands or state information. These states allow the standby ACE to come up with best-effort support. In the STANDBY_WARM state, as with the STANDBY_HOT state, configuration mode is disabled on the standby ACE and configuration and state synchronization continues. A failover from the active to the standby based on priorities and preempt can still occur while the standby is in the STANDBY_WARM state.

When redundancy peers run on different version images, the SRG compatibility: field of the show ft peer detail command output displays WARM_COMPATIBLE instead of COMPATIBLE. When the peer is in the WARM_COMPATIBLE state, the FT groups on standby go to the STANDBY_WARM state instead of the STANDBY_HOT state. The following software version combinations indicate whether the SRG compatibility: field displays WARM_COMPATIBLE (WC) or COMPATIBLE (C):

Active ACE Software Version
Standby ACE Software Version
A2(1.3) or less
A2(1.4)
A2(1.5)
A2(1.6)
A2(2.0)
A2(2.1)
A2(2.2)
A2(1.3) or less

C

C

C

C

C

C

C

A2(1.4)

C

C

C

WC

C

C

WC

A2(1.5)

C

C

C

WC

C

C

WC

A2(1.6)

C

WC

WC

C

C

WC

WC

A2(2.0)

C

C

C

C

C

C

C

A2(2.1)

C

C

C

WC

C

C

WC

A2(2.2)

C

WC

WC

WC

C

WC

C


Software Version A2(2.2) Resolved Caveats and Open Caveats

This release note includes resolved and open defects that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(2.2):

Software Version A2(2.2) Resolved Caveats

Software Version A2(2.2) Open Caveats

Software Version A2(2.2) Resolved Caveats

The following resolved caveats apply to software version A2(2.2):

CSCsu88684, CSCsq27062—When a large number of Layer 2 connected real servers are in the ARP FAILED state and each real server is associated with probes, the ACE becomes unresponsive, displays the following messages on the console, and eventually reboots:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

Workaround: None.

CSCsx68671—With generic protocol parsing, payload sticky and UDP fast-age traffic, Layer 7 UDP connections may cause a memory leak in the ACE module dataplane. Workaround: None.

CSCta20756, CSCsx15558—When the ACE has over 120,000 concurrent SSL connections, it displays SSL connection rate denies, FastQ transmit back pressure, and SSL RX back pressure. Eventually, the ACE becomes unresponsive. Workaround: None.

CSCta97335—When you configure the ACE with multiple contexts, DHCP, and a VLAN shared with the Admin context, the DHCP is not supported in a user-configured context. Workaround: None.

CSCtb05686—When you configure multiple service policies under one interface and then delete a policy, Layer 7 connections reset in the other service policies. Workaround: None.

CSCtb15617—The ACE release note should include information about the required supervisor engine Cisco IOS software and hardware revisions. Workaround: None.

Software Version A2(2.2) Open Caveats

The following open caveats apply to software version A2(2.2):

CSCsr01570, CSCsy90965—The Set-Cookie: length is null. Changing the default class map from a sticky-server farm to none does not eliminate a cookie insertion. Workaround: Remove and then enter the class class-default command.

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv54222—When an HTTP client sends pipelined requests, if the next request comes in the middle of the server response, the HTTP connection becomes unresponsive and data is missing on the web page. Workaround: Configure a connection parameter map with the set tcp wan-optimization rtt 0 command.

CSCsv92321, CSCsx25981—The ACE module reboots unexpectedly and writes a core file to the disk. Workaround: None.

CSCsx05150—When using 2048-bit certificate and key pairs with block and export ciphers, a rehandshake may lead to stuck connections. Workaround: Either use nonblock and nonexport ciphers or use certificate and key pairs that are less than 2048 bits.

CSCsx19525—When you configure 1,000 SSL VIPs on the ACE and then you change the configuration on those VIPs, a buffer leak occurs as displayed by the show np 1 me-stats command "-scommon" output and traffic conditions. Workaround: Reboot the ACE and do not make configuration changes that affect those VIPs.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: None.

CSCsx81743—HSRP or other multicast control packets might be either lost for up to 10 seconds toward the CPU or flooded in case of a link flap, as observed in the following conditions:

The Catalyst 6500 series switch is running Cisco IOS release 12.2(33)SXH3a or 12.2(33)SXI.

The port channel spans multiple modules. This condition has been seen in a combination of WS-X6708-10GE and supervisor engine EtherChannel or WS-X6708-10GE and WS-X6708-10GE EtherChannel.

The Supervisor Engine 720 and Supervisor Engine 4 is in the Catalyst 6500 series switch chassis.

The port that is flapping is not a port on the supervisor.

The ACE is load balancing traffic in the chassis.

Workaround: None.

CSCsx93137 and CSCsx93995—When you enter one of the following commands in any context but do not complete entering the remote host password when prompted, the ACE waits for your input:

crypto import ftp | sftp | {bulk ftp}

crypto export ftp | sftp

Then, if you enter one of the following commands, the session may appear to be in an unresponsive state:

crypto delete

crypto export

crypto generate csr

crypto generate key

crypto import

crypto verify

show crypto authgroup

show crypto certificate

show crypto chaingroup

show crypto files

show crypto key

After a while, the command aborts with a "SSL PKI subsystem is busy. Please try again later" message. Reissuing the command results in the same behavior.

Workaround: Enter the remote host password as requested by the associated crypto import | export command. If the problem persists, clear the relevant sessions by executing one of the following commands:

clear users

clear telnet session_ID

clear ssh session_ID

You can execute those commands if you have the appropriate privileges (for example, Admin). For details about role-based access control (RBAC) and user roles, see the Cisco Application Control Engine Module Virtualization Configuration Guide.

CSCsy29181—If either of the DP processors is at MAXCONN, the ACE should show MAXCONN in the show commands. However, the ACE waits until both DP processors are at MAXCONN. This issue occurs when the cde-same-port-hash is configured. Workaround: None.

CSCsy65650—When the ACE reports the termination of TCP flows, it may display incorrect values for the duration and amount of data transferred. This issue occurs with HTTP and connections that are terminated with TCP RST. Workaround: None. If accounting is needed and relies on this log, use another method.

CSCsy88379—The TAC diagnostic script showtech generates large output due to the show xlate command. Workaround: None.

CSCsy98701—The standby ACE generates a load-balancing core file when you configure two ACEs as FT pairs that are replicating sticky entries and you enter certain show commands on the active/master ACE. Workaround: None.

CSCsz10107—When you configure preempt and the Catalyst 6500 series switch with an active ACE module is reloaded, the ACE may not correctly replicate connections when it reboots and becomes active again. Some connections may get dropped. Workaround: None. This issue does not occur when reloading only the ACE or if preempt is not configured.

CSCsz14634—The ACE has problems when you copy large configurations from TFTP to the running-configuration and use the snmp-server community command to add the public group Network-Monitor to a context when the command was not in the original configuration. Workaround: None.

CSCsz18739—The ACE reloads when running software version A2(1.4) and RADIUS AAA is configured. Workaround: None.

CSCsz19849—You cannot import an ACE VIP in WAF. Importing works in software version A2(1.2) and in A2(1.3). Workaround: None.

CSCsz28035—Accessing the qnx shell from the physical console port of either NP on an ACE puts you in a shell. If you type exit, the NP console hangs and becomes inaccessible. Workaround: None.

CSCsz31739—When the VIP is out of service and loadbalance icmp-reply is not configured, the virtual server entry still exists in the ARP cache. The ACE will respond to ARP requests sent for this VIP. Workaround: None.

CSCsz34933—The ACE may send a reset with the sequence number zero for a probe configured with the connection term forced command. Workaround: Use the graceful termination no connection term command.

CSCsz40699—When you use the SLB-Admin, Server-Appln-Maintenance, or a custom role with a create feature server farm rule, you cannot bring real servers in or out of service under the server farm. Workaround: None. There are currently no workarounds using these specific roles. However, you can complete these tasks using the Admin role.

CSCsz49088—When you monitor the ACE CPU, you can only monitor it using an Admin role. The show processes cpu command is available only in the Admin role. The Network-Monitor role, which should have access to all show commands is unable to access the show processes cpu command. Configuring a new role on the ACE does not allow you to monitor the system feature. Therefore, only Admin users are able to run this command. Workaround: Run the show processes cpu command in an Admin role.

CSCsz86630—DNS inspection may not work after you upgrade from software version A2(1.1) to a higher release. The problem occurs only for a percentage of responses and it builds over the time. The following errors appear in the output of the show np me-stats -sfixup command in the higher release:

+[Hash miss errors]

+[NAT app fixup response error]

Workaround: Disable DNS inspection and configure more aggressive timeouts (for example, 4 seconds) for UDP and port 53.

CSCta03825—When the UDP booster is configured, the ACE does not forward every first packet from a new client's DNS request to a real server on each network processor (NP). Two packets (one for each NP) are dropped for each session. Workaround: Disable the UDP booster.

CSCta29049—When the UDP booster is enabled, the ACE drops the UDP packets that originate from the server. Workaround: Disable the UDP booster.

CSCta83978—If you download an unusually large number of best-effort CRLs from a server, the SSL process on the control plane may become unresponsive. Workaround: Do not use best-effort CRLs.

CSCta92673—When SSL traffic is flowing and you reconfigure SSL proxies that contain authgroups, the ACE may leak memory in the control plane. The memory leak is directly proportional to the number of reconfigurations that you perform. Workaround: Avoid reconfiguring an SSL proxy when an authgroup is applied to the proxy.

CSCta92891—If you change the load-balance predictor from least conns to hash url with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCta93957—If you upgrade a redundant ACE pair to software version A2(2.1), downgrade the standby to software version A2(1.4), and allow the pair to synchronize configurations, and then upgrade the standby again to A2(2.1), the standby ACE does not lock configuration mode, allowing you to make configuration mode changes. Workaround: Enable a bulk synchronization by entering the no ft auto-sync command followed by the ft auto-sync command on the active ACE.

CSCtb03834, CSCtb47541—When you configure the failaction reassign command in a server farm and all the real servers in the server farm are down, the ACE becomes unresponsive to most CLI commands and its CPU spikes up to 100 percent by the cfgmgr process. Workaround: Use the no failaction command to disable failaction reassign in the server farm.

CSCtb07772—When the ACE is reproxying, it drops server packets larger than the server advertised maximum segment size (MSS) which leads to the stalling and eventual timeout of the connection. Workaround: Configure a parameter map with the exceed-mss allow command.

CSCtb08318—When you configure the snmp-server unmask-community command in a non-Admin context on the active ACE, incremental synchronization does not synchronize this command on the standby ACE. Workaround: Perform bulk synchronization to the standby ACE. You can execute the no ft auto-sync running-config and ft auto-sync running-config commands on the active ACE whenever you are configuring or unconfiguring the snmp-server unmask-community command in a non-Admin context.

CSCtb13426—After the ACE has run for a long time without a reboot or there is a lot of communication between the supervisor engine and the ACE, when you enter the show scp stats command, the TX bytes field displays a negative byte count in its output. Workaround: None.

CSCtb13438—When you enter the supervisor no power enable module slot_number command for the slot number of the standby ACE, the standby ACE asserts itself to be the active ACE before the shutdown and both ACEs become active. Workaround: None.

CSCtb15183—When you configure the ACE with an access list and then perform multiple dynamic configurations and the use of the resequence option on it, duplicate access-list line numbers may occur on the ACE, further resequence commands fail, and you can not add an object. Workaround: Reboot the ACE to clear this condition.

CSCtb16605—When you add the cookie secondary command to a sticky group after you assigned the group to a policy and an interface, this command has no effect. Workaround: Remove the policy and reconfigure it.

CSCtb23312—The ACE becomes unresponsive when its uptime reaches approximately 485 days. Workaround: Gracefully reboot the ACE before its uptime reaches 480 days.

CSCtb23798—If you configure a BVI interface and a VLAN interface in two different contexts with the same ID and apply a global policy in the context with the BVI, the configuration may fail with either of the following errors:

Error: Global Policy applied, conflicts with VIP, NAT or Interface IP in shared 
interface!

Error: Cannot overlap vip or NAT or interface address configured in a shared 
interface!

Workaround: None.

CSCtb25491—After modifying an access list and then resequencing it in quick succession, the following error message appears in the syslog file:

WARNING: Unknown error while processing access-group. Incomplete rule is currently 
applied on interface vlanXXXX. 

Workaround: Manually roll back to a previous access rule configuration on the interface. Do not issue resequence commands in quick succession. After you execute a command, reenter it with a different line number.

CSCtb27018—When you configure the ACE for SIP UDP, the ACE does not accept the SIP UDP probes requests because the source port of the 200 OK message is different than the destination port of the OPTIONS method. Workaround: None.

CSCtb29571—After you repeatedly configure and unconfigure DHCP in Admin and user contexts, the DHCP relay service may restart. Workaround: None.

CSCtb30337—In a configuration with two gateways for the same network and asymmetric traffic, the ACE may not handle the connection properly if the source MAC address changes in the middle of connection. Workaround: None.

CSCtb34660—When a client sends large HTTP POST requests, the ACE advertises the incorrect value for the window size when sending the response page. Workaround: Set the buffer share to 64K bytes unless the ACE starts advertising a window size greater than 64K bytes.

CSCtb34696—When a large POST request is sent to the ACE VIP address with a default window size, the ACE does not acknowledge the bytes and retransmits them in another frame as a result of a misassignment in a previous GET request. Workaround: Set the buffer share to 64K bytes.

CSCtb35900—When all of the ports for the first IP address in the NAT pool are used up, NAT pool exhaustion occurs and ACE-wide problems occur. Workaround: Configure a single NAT pool range, for example, nat-pool 5 10.147.2.11 10.147.2.14 netmask 255.255.255.255 pat.

CSCtb38297—When you configure the weighted leastconn configuration on the ACE, the ACE sends a majority of the traffic to a few of the real servers in a server farm and very little traffic to the other real servers. When the real servers are in a failed state (PROBE_FAILED) and configured with custom weights, a configuration download occurs. Workaround: Perform one of the following:

Change any configuration on the affected server farm when all the real servers are operational. For example, enter the no inservice and inservice commands of any real server in the server farm.

Remove the weight configuration.

Remove the probe configuration and then make a configuration change when all real servers are operational. Readd the probe configuration after 30 seconds.

CSCtb39310— When you configure the ACE with leastconn predictors using weight buckets and the ACE processes load balancing requests, the ACE reboots. Workaround: None.

CSCtb39697—The NAT Pool Alloc [fail] counters increment on the standby ACE but the counters on the active ACE do not. Workaround: None.

CSCtb48429—When repeatedly logging into and out of the ACE, a memory leak occurs. Workaround: None.

CSCtb49907—When the ACE fails and the standby ACE becomes active, a gratuitous ARP on the standby ACE in bridge mode does not update the ARP table causing a probe failure. After the ARP entry times out, the standby ACE recovers. Workaround: None.

Software Version A2(2.1) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

This release note includes resolved and open defects that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats, command changes, and syslog messages in software version A2(2.1):

Software Version A2(2.1) Resolved Caveats

Software Version A2(2.1) Open Caveats

Command Changes in Software Version A2(2.1)

System Log Messages

Software Version A2(2.1) Resolved Caveats


Note This software release includes resolved caveats that were merged from software versions A2(1.4), A2(1.4a), and A2(1.5). For details about those resolved caveats, see the A2(1.x) release note at the following URL: http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3.00_A2/release/note/racea2_x.html.


The following resolved caveats apply to software version A2(2.1):

CSCsh04655—When you use the Generic Protocol parser to load balance some types of TCP traffic, connections may hang and no outbound leg is established if fewer than the configured max-parse-length number of bytes are sent by the client.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl75662—You may observe that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time that the probe is sent even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe was not sent; the initial large time interval has to expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from the real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform either of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, and then log in again to access a different context than the original context with the configured packet capture function.

CSCsm89594, CSCsr14898—XML output for the show serverfarm detail command is not valid XML. If the server farm does not have a configured probe, the generated XML output still contains a close tag </sf_probes> and does not have an open tag <sf_probes>. Workaround: Configure a probe in the server farm. If a probe is configured on the server farm, then there should be both an open tag and a close tag present in the XML output. If a probe is not configured on the server farm, then neither tag should be present.

CSCso60304—When an invalid XML attribute is sent to the ACE, it does not respond as expected. Instead, the ACE displays a 500 Internal Server Error message. No negative impact to the ACE is observed. Workaround: None.

CSCso80478—When you perform multiple parallel SNMP walks that last 30 seconds or longer on an ACE in a redundant configuration, you may observe response timeouts on both the active and the standby ACEs. You may also observe this behavior in multiple contexts. This behavior does not occur with SNMP walks of shorter durations. Workaround: None.

CSCso81785—If you are using TACACS+ and the Cisco Access Control Server (ACS) with an RSA authentication manager, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another network access server (NAS) to set your PIN.

CSCso81811—If you are using TACACS+ and the Cisco ACS with an RSA authentication manager and your account is in next token mode, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another NAS to enter the next token code and make your account accessible again.

CSCso82971—If you are using a TACACS+ server that is an RSA server with TACACS+ continue authentication, authentication may fail to the configured server, but you still can log in using local authentication.

Use one of the following workarounds:

Use the Cisco ACS instead of the RSA server.

Do not configure local as the secondary authentication method.

CSCso85639—If you configure the passdetect interval command value for less than 30 seconds, the ACE sends overlapping probes that use additional management connections (resources). Workaround: Increase the passdetect interval command value to 45 seconds.

CSCso86485—When a client-side VLAN interface is brought up and down an excessive number of times on the active ACE under a light traffic load, the standby ACE may generate a core dump. Workaround: None.

CSCso95457—When you enter the clear conn all command, the ACE sends an RST to close the connection only to the server and purges both the inbound and outbound connection entries from its connection database. As a result, the client connection is left open and any further requests arriving on that connection are not serviced. Workaround: None.

CSCso95620—With long-lived HTTP, SSL, FTP and UDP traffic on the ACE, you may observe a memory loss of approximately 333 KB in the ACE during an EtherChannel link (FT port channel) failure and recovery on the Catalyst 6500 series switch. Workaround: None.

CSCsq87162—SSL transactions may not complete when the server-conn reuse command is enabled. Workaround: Disable the server-conn reuse command.

CSCsq99448—When you upgrade the ACE from version A1(6.3a) to A2(1.1), you might experience unresponsiveness in the outbound connection manager (OCM) because of the deletion of an improper internal message. Workaround: None.

CSCsr09129—When you configure SIP load balancing with inspection enabled, the ACE should open a pinhole to the address in the Via header for the server response. However, the server responses remain in the data channel. Workaround: None.

CSCsr18029—The ACE may reload after an SNMP query. Workaround: None.

CSCsr62027—When TCP normalization is disabled, the ACE places replicated TCP connections in the INIT state on the standby ACE. After the normal embryonic connection timeout occurs, the ACE removes the replicated connections from the standby. Workaround: Do not disable normalization.

CSCsu49899—When an HTTP virtual server that performs Layer 7 inspection shares the same virtual IP addresses as other servers, the ACE responds to SYN requests whether or not the Layer 7 virtual server is up or down. The ACE completes the three-way handshake before sending an RST. Workaround: Make sure that HTTP Layer 7 virtual servers have unique virtual IP addresses or all of them use the same VIP to ensure the other protocols do not get spoofed unnecessarily.

CSCsu55180, CSCsv02360—When you configure the ACE with SSL termination and server connection reuse, and a client makes an HTTPS request to the VIP address, some connections fail if the client MTU is low (for example, an MTU of 576). Workaround: None.

CSCsu60137—When the ACE issues a POST request, an SSL bad-record MAC error occurs with Firefox Version 2 and 3. The same POST request works with Microsoft IE. Workaround: None.

CSCsu67523 and CSCsu67556—Upgrading the ACE software to version A2(1.1a) causes the ACE to reboot and generate a core dump. Workaround: None.

CSCsu67539—When you upgrade the ACE software to version A2(1.1), the ACE reboots and generates a core dump. Workaround: None.

CSCsu68314—When the ACE becomes unresponsive and generates a core dump, the core-dump file contains three different types of files. These files should be separate files. Workaround: Use the file command to uncompress the core-dump files.

CSCsu68366—The ACE reboots and generates a qnx_2_mecore_log.999.tar.gz core-dump file. Workaround: None.

CSCsu86606—When you reboot the ACE and as it powers up, the Catalyst 6500 series switch disables the ACE and displays the following log messages:

Oct  1 07:43:25.710 EDT: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off 
(Reset)
Oct  1 07:43:41.611 EDT: %OIR-SP-6-PWRFAILURE: Module 1 is being disabled due to power 
convertor failure 0x1 

Workaround: None.

CSCsu95356—When you configure the ACE with the predictor least conn command, the real server does not get the expected number of connections. Workaround: Remove the real server from the server farm and readd it.

CSCsu95887—After the active ACE module completes configuration synchronization, it generates a core dump. Workaround: None.

CSCsu96977—When you configure more than 640 action lists and enter the do show action_list command with the Tab or ? key for help, the ACE becomes unresponsive. Workaround: None.

CSCsv02224—When you configure and remove an SSL-proxy service after you configure and remove multiple class maps under a policy map, the following error appears on the console:

Error: Called API encountered error appears console.

The ACE rejects the ssl-proxy command and the command does not appear in the configuration. Workaround: None.

CSCsv04319—If you create a TACACS+ server with a numeric key, the ACE sends a warning about the key; however, it does not create the server. The message should be an error and not a warning. Workaround: Use a key that is not entirely numeric.

CSCsv04848—When you configure RADIUS on the ACE and a user logs off, the RADIUS client sends an accounting stop message to the server for that user but the ACE does not immediately delete all connections for that user. If the source IP address for the user is immediately reassigned to another user, the new user could open a new connection before the old connections from the previous user times out. The result is that the ACE incorrectly forwards the new connections and does not load balance the packets. Workaround: Set the UDP connection timer to a smaller number (for example, 10 seconds).

CSCsv10547—The config-register setting does not synchronize after an ACE module boots. The config-register setting synchronizes only when you configure it with ACE modules in active or standby mode. Workaround: None.

CSCsv31476—When the ACE generates a core-dump file for the kernel or Virtual Shell (VSH) applications, the file does not contain the code-train version information. Workaround: None.

CSCsv35373—Failaction reassignment does not work with real servers on different VLANs. Workaround: None.

CSCsv40516, CSCsr22703, CSCsu67574—When you upgrade the ACE software to version A2(1.0a), the ACE reboots and generates a core dump. Workaround: None.

CSCsv41126, CSCsu80235—When you configure stickiness on a context and the sticky database lookup is 8,192 over the maximum threshold, the ACE drops connections causing the users to experience resets or their pages do not load properly. The Drop Max Remote Stky counter displayed by the show np [1 | 2] me-stats -slb command continues to increase. Workaround: Force a failover to the backup ACE and reboot the module that had the problem.

CSCsv47724—The heartbeats on fault-tolerant (FT) ACE modules occasionally miss due to late TCP timers. The ACEs increment the Heartbeats Missed counter on the standby ACE and the Unidirectional HB's Received counter on the active ACE. Workaround: None.

CSCsv48498—When you enable FTP inspection and disable normalization on the client-side interface, the ACE inserts the TCP Option Timestamp in packets to the client and the FTP server, even if both the client and the server are not using this option. Workaround: Enable normalization or disable FTP inspection.

CSCsv49606—When you configure stickiness on the ACE, the ACE becomes unresponsive. Workaround: None.

CSCsv52331—The ACE becomes unresponsive due to an SRAM parity error. Workaround: None.

CSCsv52478—When you reboot the Catalyst 6500 series chassis, the ACE may reboot as Active. Workaround: None.

CSCsv52942—When the server farm that has no backup, goes to inactive state after all the real servers go to the MAXCONNS state, the real servers may not accept connections even though they are out of maximum connections. Workaround: Configure a backup to the server farm.

CSCsv53187—The ACE generates an NP ha_hb_g_ns core dump during a standard operation. Workaround: None.

CSCsv53620—When you add an SSL proxy class to a policy map, the following error occurs:

Error: Called API encountered error

Workaround: Remove the class from the policy map and then readd it.

CSCsv65178—When you specify TCP as the protocol in a class map configured for DNS traffic, the ACE allows the configuration and DNS inspection fails. Workaround: Specify UDP as the protocol in a class map configured for DNS traffic.

CSCsv69769—When you configure an expect regex value, the ACE allows a space in the quoted name of the value. Workaround: Do not use a space. Instead, use a search character (.*) or allow the variable to be on a long string input.

CSCsv74527, CSCsw82768—When DNS traffic runs consistently at more than 10000 CPS, proxy entries are leaked on the standby ACE in a HA environment after approximately two hours. Proxy entries are leaked and are not cleared on the standby ACE due to connection validation errors. Workaround: None.

CSCsv95254, CSCsv53112—When an IP address conflict occurs on a bridged VLAN, the ARP manager may become unresponsive causing the ACE to generate a core dump. Workaround: Resolve the IP conflict in your configuration.

CSCsv98101—Although console and remote login access has failed to the ACE, traffic is still passing. Workaround: Reboot the ACE to clear this condition.

CSCsx14648, CSCsx08589—After the ACE takes a long time to boot with some errors on the console or terminal, the Admin user behaves as a network-monitor user. After another reboot, the ACE loads and the Admin user has Admin privileges, but the SSL-proxy configuration in the Admin context has lost certificates. The Admin context includes several VIPs with the SSL-proxy configuration and the configuration includes several contexts. Workaround: Define the VIPs in a context other than the Admin context.

CSCsw28313—If one client sends multiple, consecutive DHCP requests to the ACE, the ACE may become unresponsive and generate a core dump file. Workaround: Block the DHCP requests by configuring an access list.

CSCsw81300—When you configure the ACE with the combination of HTTP inspection and an HTTP load-balancing policy map with only a class-default class, server-connection reuse does not allow traffic. Workaround: Change the class map in the HTTP load-balance policy map from a class-default class map to a type HTTP load-balance class map.

CSCsw97987—When you configure multiple class maps to a multi-match policy map and you send traffic to a class map, if you delete and readd all of the other class maps, the traffic destined for the remaining class map gets a hit when you try to readd it to the same policy map. Workaround: In a multi-match policy map with more then one class map, do not delete and readd all class maps except the class map where you are sending the traffic.

CSCsw98274—When you add and remove the class map along with the SSL proxy from a multi-match policy map multiple times, if you attempt to add a class map and then try to apply an SSL proxy, the "Error: Called API encountered error" message occurs and the proxy is not applied to the class map. Workaround: Do not add and remove the class map from a multi-match policy map too quickly. If this situation continues, reboot the ACE.

CSCsx14648—Crypto files may be deleted if high loads are created on the control plane, for example, by copying and pasting a very large configuration. The control plane must be heavily loaded for this issue to occur. Workaround: Copy and paste large configurations in small segments, giving each segment time to load before moving to the next segment.

CSCsx39224—When you configure a sticky server farm as part of a policy map and the real servers are brought out of service making the server farm inactive, the backup server farm does not take the connections after the primary server farm becomes inactive. Workaround: Configure the server farm as part of the policy-map instead of the sticky server farm.

CSCsx47594—When an SSL server does not use an RSA certificate and the ACE does not determine that the certificate is not RSA, the ACE becomes unresponsive when there is SSL back-end traffic with HTTPS probes. Workaround: Make sure that the SSL server uses an RSA certificate.

CSCsx63328, CSCsx13274—When the ACE SSL is at its peak performance, a leaked SSL context state occurs that cannot be detected with show commands. Workaround: None.

CSCsx72444—If you configure system logging over TCP to send messages to a server and the server closes the connection because of a failure or a restart, the ACE close its own socket and displays the following error message:

Monitor logging: enabled (level - information)
Logging to 192.168.1.11 tcp/5140
(socket created but failed to connect)

After the ACE closes the socket, it never tries to reopen it and no more messages are sent. Workaround: Remove and readd the syslog host command configuration or use a syslog over UDP.

CSCsx81954—If an HTTP request spans multiple packets, it is possible that the ACE will discard the second packet, forcing the client to retransmit. The HTTP request must be large enough that it is sent in more than one packet and the request is not the first one on a persistent connection. The discarded packet causes a retransmission by the client and the ACE does not drop packets after the retransmission. Workaround: None.

CSCsx97484—When the ACE reboots with the primary server farm out of service, traffic does not switch to the backup server farm. Workaround: Configure one real server under the primary server that could trigger the failover again.

CSCsy29490, CSCsv83236—When you configure the ACE with a sticky cookie and enable persistence rebalance, the show serverfarm command displays connection entries after traffic has stopped. Also, the connection entries do not clear correctly. Workaround: Disable persistence rebalance or use another sticky type (for example, IP sticky).

CSCsy34814—The syslog message 305010 includes the duration of the xlate translation. However, this duration is always equal to the xlate idle timeout. Workaround: Use the timestamps in the creation and the tear down of the xlate connections to calculate the xlate duration.

CSCsy55274—When the ACE is running software version A2(2.0) with application inspection configured, both network processors may generate core dump files. This issue may occur when the inspection configuration is in an error-handling scenario with a missing NULL pointer check. Workaround: None.

CSCsz34011—After a series of reboots, both ACE modules lose their context configurations. If the active ACE halts and reloads, after it reboots, it will read the first half of the startup-config file, establish FT with the standby ACE (the new active), and synchronize the configuration to obtain the rest of the configurations from the other ACE. If the other ACE stops functioning, the active ACE will not have obtained the rest of the configurations, including context configurations. Context configurations may be lost, although they still exist in the startup-config file. Workaround: None.

CSCsz32455—When you enter the show tech-support command, it may fail with an error during the execution of the show acl-merge merge vlan commands. Workaround: Enter the commands in the show tech-support command manually.

CSCsz69431—When the ACE is configured for redundancy and Route Health Injection (RHI) and the FT VLAN goes up and down, the standby ACE may transition to a nonredundant state and advertise its routes using RHI with the real interface IP address. Workaround: Configure an FT query VLAN with a management policy and ACL that allow ICMP (pings).

CSCsz69433—The FT VLAN may transition incorrectly to a nonredundant state if the interface goes up and down. When the FT interface correctly transitions out of this nonredundant state, any RHI routes are not withdrawn. Workaround: Configure an FT query VLAN with a management policy and an ACL that allow ICMP (pings).

CSCsz73222—After you apply a configuration where the logging server address does not match the network address of any configured interface, the ACE may become unresponsive and generate a network processor crash file that indicates an SRAM parity error. Workaround: Disable logging by entering the no logging enable command or configure a server on a network that is local to the ACE.

CSCsz77633—When the ACE is receiving Layer 7 traffic, it may discard Layer 4 sticky connection requests on the same or on a different context because the ACE may incorrectly reset the connection after traffic is sent for some duration. You should not encounter this issue with only Layer 4 traffic or only Layer 7 traffic. The issue is seen only with the combination of the two types of traffic. Workaround: None.

CSCta01759—When an SSL certificate with a nonconforming serial number length is presented to the ACE as part of the authentication mechanism, the ACE becomes unresponsive. Typically, CAs do not issue certificates with a nonconforming serial number length. Workaround: Use only conforming-length (up to a maximum of 20 bytes per RFC3280) serial numbers in SSL certificates.

CSCta05557—If you dump verbose queue outputs using either the ucdump command on the network processor console or at the CLI by using the show np 1 | 2 me-stats -q queue_name -vvv command, the network processor may become unresponsive and unusable. This issue occurs randomly depending on the content of the message. Specifically, the problem was seen when the ACE dumped the verbose queue elements for the lbrx queue. However, it can happen to a few other queues as well. Workaround: None.

CSCta14111—The show service-policy command may not display all configured policies. The command output has a limited size. If you exceed that size because of a large number of class maps and match statements, the remaining information may not appear in the output. Workaround: None.

CSCta15251—If you change the load-balancing predictor in a server farm to one of the hash predictors while traffic is flowing and with two real servers that are configured as backup servers for each other (cyclic backup servers), the ACE load-balancing queues eventually becomes full and the ACE becomes unresponsive. Workaround: None.

CSCta15196—The show service-policy detail command may display invalid port numbers if the associated VIP has a configured port range. Workaround: None.

CSCta26489—A user with a custom role that includes the rule number permit modify feature real server command cannot change the real server configuration even though the real server is defined as an object in this domain. When you try to configure the real server, you may see the message "Error: cannot create new object; user has modify permissions only." This problem has occurred in A2(2.x) software, but not in A2(1.x) software. Workaround: Add the rule number permit create feature rserver command to the user role.

CSCta33566—When the set tcp timeout embryonic command is configured, the ACE may not send RSTs at the time specified by the command. Retransmitted SYNs from the clients are not received by the server because the retransmits are causing the embryonic connections to be reset. Workaround: None.

CSCta38648—The ACE reports a loss of particle buffers when you reconfigure a large number of VIPs at the same time. The buffers are lost because they cannot find a place in the same system pool intended to hold these buffers. Workaround: None.

CSCta42712—If a real server is down, configuring a passdetect interval that is less than 30 seconds can cause overlapping probes, which can lead to resource issues. This problem occurs because the default half-open timeout for the TCP probe traffic is configured to be 30 seconds and cannot be changed. Workaround: Configure a passdetect interval that is greater than or equal to 30 seconds.

CSCta45580—When a large number of VIPs (greater than 50) use the same SSL proxy with a certificate revocation list (CRL) applied and the CRL server is down when the ACE attempts to download the CRL for the first time, the download fails as expected. When the CRL server comes back up and the CRL is applied again, the ACE may not attempt to download the CRL again. Workaround: Unconfigure and reconfigure the CRL.

CSCta53777—When SSL traffic that requires client authentication enters the ACE, it may begin leaking memory. If the real servers are brought down at the same time, the rate of the memory leak increases until the ACE may eventually become unresponsive. Workaround: Reload the ACE to reclaim the occupied memory and restart the system.

Software Version A2(2.1) Open Caveats

The following open caveats apply to software version A2(2.1):

CSCse12120—When you press Ctrl-D and attempt to log in to the ACE with a valid username and password using the session command through EOBC from the supervisor engine, the login attempt fails. Workaround: Press Ctrl-D twice to access the switch login, and then log in.

CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, and then try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in an RSA format instead of an RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.

CSCsm93110—When you configure Microsoft Internet Information Services (IIS) version 5.0 to accept client certificates, SSL initiation through the ACE may fail. Workaround: Upgrade to IIS version 6.0.

CSCso33506—In a redundant configuration with the FT pair running mismatched code (A1(x) and A2(x)) and mismatched licenses, if the active ACE is rebooted, the Admin context comes up, but, in user contexts, the running-config file is blank. This behavior occurs only when there is both a license and a code mismatch. Workaround: Resolve one of the mismatches and reload the ACEs or enter the copy start run command in each user context.

CSCso55790—While trying to copy core dump files from the core: directory to an FTP server, the copy operation fails and the following error message is displayed:

local: /TN-COREFILE/core.618: Permission denied

Workaround: Copy the files from debug or from the console after you modify the permissions using debug.

CSCso76159—When you dynamically modify a service policy to use an HTTP parameter map with the header modify per request command, the ACE does not insert a header into every GET request for existing long-lived persistent flows. However, the ACE does insert a header into every GET request for new flows. Workaround: None.

CSCso82657—While moving a VLAN from a Cisco Firewall Services Module (FWSM) to an ACE or from an ACE to an FWSM, IP routing is not updated on the ACE to reflect the change. This behavior occurs when you are making a change to the svclc commands and the shut and no shut commands on the ACE interfaces. Workaround: None.

CSCsq03035—The ACE was configured with an idle timeout of 0 (never time out), while TCP and UDP traffic was sent and left in an idle state over an extended period of time. The idle timeout was then changed from infinite to 60 seconds. The UDP traffic was immediately cleared, while the TCP traffic was not. After waiting more than 15 minutes, the idle TCP flows still had not been cleared. Workaround: None.

CSCsq64401—If you configure the switch-mode command in an ACTIVE user context in a redundant configuration, the command is not synchronized to the STANDBY_HOT user context on the other ACE. This problem occurs only in a redundant configuration where an ACE has its Admin context in the STANDBY_HOT state and a user context in the ACTIVE state.

There are two possible workarounds for this behavior as follows:

Never allow a user context to be in the ACTIVE state on the standby ACE.

Reload the ACE that has its user context in the STANDBY_HOT state.

CSCsr01570, CSCsy90965—The Set-Cookie: length is null. Changing the default class-map from a sticky-serverfarm to none does not eliminate a cookie insertion. Workaround: Remove and then enter the class class-default command.

CSCsu88684, CSCsq27062—When a large number of Layer 2 connected real servers are in the ARP FAILED state and each real server is associated with probes, the ACE becomes unresponsive, displays the following messages on the console, and eventually reboots:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

The ACE reboots after the messages are displayed. Workaround: None.

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv31046—When you configure the least-connections predictor on the ACE, the ACE may not sustain 160,000 CPS traffic. Workaround: None.

CSCsv54222—When an HTTP client sends pipelined requests, if the next request comes in the middle of the server response, the HTTP connection becomes unresponsive and data is missing on the web page. Workaround: Configure a connection parameter-map with the set tcp wan-optimization rtt 0 command.

CSCsv92321, CSCsx25981—The ACE module reboots unexpectedly and writes a core file to disk. Workaround: None.

CSCsw40764—When the ACE executes the no access-list command to delete an ACL configured with 64,000 entries, an API timeout occurs. Workaround: Do not delete all of these entries from an ACL at one time. Delete the entries from an ACL one at a time or in small chunks.

CSCsw51821—When you enable RTSP inspection on the ACE and the server sends the next request without responding to the previous client request, a static parse error occurs and the packet is dropped. Also if you configure RTSP inspection on the ACE, the ACE resets the connection. Workaround: Make sure that the server responds to every client request with the proper return code (for example, 200 OK) before sending the next request.

CSCsx05150—When using 2048-bit certificate and key pairs with block and export ciphers, a rehandshake may lead to stuck connections. Workaround: Either use nonblock and nonexport ciphers or use certificate and key pairs that are less than 2048 bits.

CSCsx13147—When you import a number of SSL PKI key or certificate files into a context by using the crypto import command, if you remove the context without first removing the files through the crypto delete command, the ACE may not import additional SSL PKI key or certificate files. The failure is due to a lack of resources or during a subsequent file import process, some or all of the previously imported key or certificate files may be forcibly removed from some or all contexts. Workaround: Use the crypto delete command to explicitly delete the SSL PKI key or certificate files from the contexts before removing the context. Try rebooting the ACE if this problem has already happened.

CSCsx19525—When you configure 1,000 SSL VIPs on the ACE and then you change the configuration on those VIPs, a buffer leak occurs as displayed by the show np 1 me-stats command "-scommon" output and traffic conditions. Workaround: Reboot the ACE and do not make configuration changes that affects those VIPs.

CSCsx28656—When you create a large configuration consisting of interfaces and ACLs in a redundant configuration, if you remove a context from the active ACE, the context is not removed from the standby and the standby ACE transitions to the Hot state even though configuration synchronization failed. Workaround: Disable redundancy. Remove the configuration manually from the standby ACE and then reenable redundancy.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx38885—When the ACE contains a large configuration, if you quickly add and remove multiple class maps under a Layer 7 policy map, API timeout errors occur. Workaround: Do not add and remove class maps under a Layer 7 policy map in quick succession.

CSCsx52128—When you copy a large configuration with many ACLs to the running-config file and perform other configuration changes continuously, the aclmerged process does not get the CPU and also the configurations result in API errors. Workaround: When you copy a large configuration with many ACLs to the running-config file, wait approximately 2 minutes for it to complete. Do not perform any configuration changes at that time.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: None.

CSCsx80970—When you configure a multi-match policy map with more than one class map, if you perform an inspect policy change in a class map, the traffic to other class maps may be hit. Workaround: Do not make any inspect changes on the multi-match policy map when traffic is running.

CSCsx93137 and CSCsx93995—When you enter one of the following commands in any context but do not complete entering the remote host password when prompted, the ACE waits for your input:

crypto import ftp | sftp | {bulk ftp}

crypto export ftp | sftp

Then, if you enter one of the following commands, the session may appear to be in an unresponsive state:

crypto delete

crypto export

crypto generate csr

crypto generate key

crypto import

crypto verify

show crypto authgroup

show crypto certificate

show crypto chaingroup

show crypto files

show crypto key

After a while, the command aborts with a "SSL PKI subsystem is busy. Please try again later" message. Reissuing the command results in the same behavior.

Workaround: Enter the remote host password as requested by the associated crypto import | export command. If the problem persists, clear the relevant sessions by executing one of the following commands:

clear users

clear telnet session_ID

clear ssh session_ID

You can execute those commands if you have the appropriate privileges (for example, Admin). For details about role-based access control (RBAC) and user roles, see the Cisco Application Control Engine Module Virtualization Configuration Guide.

CSCsy04371—When a server farm with no backup transitions to the Inactive state after all the real servers transition to the MAXCONNS state, if the real servers transition out of the MAXCONNS state, they may not accept connections. Workaround: Configure a backup to the server farm.

CSCsy23268—The ACE may send probe traffic with the source IP address of the alias IP address instead of the local interface IP address. This issue occurs on the active ACE only. Workaround: None.

CSCsy29181—If either of the DP processors is at MAXCONN, the ACE should show MAXCONN in the show commands. However, the ACE waits until both DP processors are at MAXCONN. This issue occurs when the cde-same-port-hash is configured. Workaround: None.

CSCsy65650—When the ACE reports the termination of TCP flows, it may display incorrect values for the duration and amount of data transferred. This issue occurs with HTTP and connections that are terminated with TCP RST. Workaround: None. If accounting is needed and relies on this log, use another method.

CSCsy88379—The TAC diagnostic script showtech generates large output due to the show xlate command. Workaround: None.

CSCsy98701—The standby ACE generates a load-balancing core file when you configure two ACEs as FT pairs that are replicating sticky entries and you enter certain show commands on the active/master ACE. Workaround: None.

CSCsz10107—When you configure preempt and the Catalyst 6500 with an active ACE module is reloaded, the ACE may not correctly replicate connections when it reboots and becomes active again. Some connections may get dropped. Workaround: None. This issue does not occur when reloading only the ACE or if preempt is not configured.

CSCsz14634—The ACE has issues when you copy large configurations from TFTP to the running-configuration and use the snmp-server community command to add the public group Network-Monitor to a context when the command was not in the original configuration. Workaround: None.

CSCsz18739—The ACE reloads when running software version A2(1.4) and RADIUS AAA is configured. Workaround: None.

CSCsz19849—You cannot import an ACE VIP in WAF. Importing works in software version A2(1.2) and in A2(1.3). Workaround: None.

CSCsz28035—Accessing the qnx shell from the physical console port of either NP on an ACE puts you in a shell. If you type exit, the NP console hangs and becomes inaccessible. Workaround: None.

CSCsz31739—When the VIP is out of service and loadbalance icmp-reply is not configured, the virtual server entry still exists in the ARP cache. The ACE will respond to ARP requests sent for this VIP. Workaround: None.

CSCsz34933—The ACE may send a reset with the sequence number zero for a probe configured with the connection term forced command. Workaround: Use the graceful termination no connection term command.

CSCsz40699—When you use the SLB-Admin, Server-Appln-Maintenance, or a custom role with a create feature server farm rule, you cannot bring real servers in or out of service under the server farm. Workaround: None. There are currently no workarounds using these specific roles. However, you can complete these tasks using the Admin role.

CSCsz49088—When you monitor the ACE CPU, you can only monitor it using an Admin role. The show processes cpu command is available only in the Admin role. The Network-Monitor role, which should have access to all show commands is unable to access the show processes cpu command. Configuring a new role on the ACE does not allow you to monitor the system feature. Therefore, only Admin users are able to run this command. Workaround: Run the show processes cpu command in an Admin role.

CSCta20756, CSCsx15558—When the ACE has over 120,000 concurrent SSL connections, it displays SSL connection rate denies, FastQ transmit back pressure, and SSL RX back pressure. Eventually, the ACE becomes unresponsive. Workaround: None.

CSCta83978—If you download an unusually large number of best-effort CRLs from a server, the SSL process on the control plane may become unresponsive. Workaround: Do not use best-effort CRLs.

CSCta92673—When SSL traffic is flowing and you reconfigure SSL proxies that contain authgroups, the ACE may leak memory in the control plane. The memory leak is directly proportional to the number of reconfigurations that you perform. Workaround: Avoiding reconfiguring an SSL proxy when an authgroup is applied to the proxy.

CSCta92891—If you change the load-balance predictor from least conns to hash url with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCta93957—If you upgrade a redundant ACE pair to software version A2(2.1), downgrade the standby to software version A2(1.4) and allow the pair to synchronize configurations, and then upgrade the standby again to A2(2.1), the standby ACE does not lock configuration mode, allowing you to make configuration mode changes. Workaround: Enable a bulk synchronization by entering the no ft auto-sync command followed by the ft auto-sync command on the active ACE.

CSCtb02056—When you configure the ACE with SSL certificates and keys in multiple contexts, the output of the show crypto certificate all command may become corrupted. Workaround: Use the show crypto certificate cert_name command instead of the show crypto certificate all command.

Command Changes in Software Version A2(2.1)

Table 6 lists the commands and options that have been changed in software version A2(2.1).

Table 6 CLI Commands Changed in Version A2(2.1)  

Mode
Command and Syntax
Description

Exec

crypto crlparams crl_name cacert ca_cert_filename

no crypto crlparams crl_name

Configures signature verification on a CRL to determine that it is from a trusted certificate authority (CA). The arguments are as follows:

crl_name— Name of an existing CRL.

ca_cert_filename— Name of the CA certificate file used for signature verification.

Use the no version of this command to remove signature verification from the CRL.

Exec

show acl-merge {acls internal vlan [vlan_id] {in | out} [summary]} | {match internal vlan [vlan_id] {in | out} ip_address1 ip_address2 protocol src_port dest_port} | {merged-list internal vlan [vlan_id] {in | out} [non-redundant | summary]}

The new internal vlan keyword displays the ACL merge information for VLAN 1.

Exec

show conn [{address ip_address1 [ip_address2] netmask mask [detail]}
| count | detail | {port number1 [number2] [detail]} | {protocol {tcp | udp} [detail]} | {rserver rs_name [port_number serverfarm sfarm_name1 | serverfarm sfarm_name1] [detail]} | {serverfarm sfarm_name2 [detail]}]

The detail option has been added for a specified address, port, protocol, real server, or server farm. This option displays additional information for the connection including idle time, elapsed time, byte count, packet count, and, if applicable, the state of the connection in the reuse pool.

Exec

show crypto cdp-errors

Per CSCsz83339, the output for this command now includes the Best Effort CDP Errors Ignored field. This field displays the number of times that the ACE ignored CDP errors in the presented SSL certificate and thereby allowed the SSL connection. This field is related to the new cdp-errors ignore command in parameter map SSL configuration mode.

Exec

show crypto crl name detail

The new detail keyword displays additional statistics for CRL download failures. For information on the fields for this command, see the "Displaying Detailed CRL-Downloading Statistics" section.

Exec

show ft config-error [context_name]

In a redundant configuration, the new config-error keyword displays the commands that fail on the standby ACE during bulk synchronization. If all commands succeed on the standby ACE, the command displays the following message:

No bulk config apply errors

In the Admin context, the optional context_name argument is the name of a user context. If you do not enter the argument, the command uses the Admin context. In a user context, this argument is not available.

Exec

show parameter-map [name]

Per CSCsx75858, this command now displays the urlcookie-start field. This field displays one of the following:

The start string of the secondary cookie or the none setting configured by the set secondary-cookie-start command in parameter map HTTP configuration mode.

The default string of ?.

Exec

show serverfarm [name] [detail]

The fields displayed by this command now include the real server description field as defined by the description command in server farm host real server configuration mode.

Exec

show stats http

The TCP fin/rst msgs sent, Bounced fin/rst msgs sent, SSL fin/rst msgs sent fields have been expanded to the following fields:

TCP fin msgs sent

TCP rst msgs sent

Bounced fin msgs sent

Bounced rst msgs sent

SSL fin msgs sent

SSL rst msgs sent

Exec

show sticky cookie-insert group sticky_group_name

The new show sticky cookie-insert command displays information that correlates the inserted cookie, the sticky entry, and the final destination for the cookie insert configuration.The output for this command includes the following fields:

Cookie—Cookie-insert hash string for each real server in the associated server farm.

HashKey—64-bit hash value associated with the cookie.

rserver-instance—String containing the server-farm name, real-server name, and real-server port in the following format:

server_farm_name/real_server_name:rserver_port

Exec

show sticky database static | i never

The "| i never" modifier filters the show sticky database static command for the "never" time-to-expire flag.

Exec

show sticky database static http-cookie cookie_value

This command no longer displays the hash key.

Exec

show tech-support

Per CSCsx33405, this command no longer displays the following:

All show acl-merge acls vlan command output

All show acl-merge merge-list vlan number out command output

It also now displays a maximum of four VLANs.

Configuration

snmp-server unmask-community

no snmp-server unmask-community

The unmask-community keyword allows you to unmask the snmpCommunityName and snmpCommunitySecurityName OIDs of the SNMP-COMMUNITY-MIB. By default, they are masked. Use the no form of the command to mask them.

Configuration

username name1 ...

The name1 argument now supports the following non-alphanumeric characters:

- _ @ \

This argument does not support the following characters:

$ / ; ! #

Note Per CSCsy95433, the "." character is not supported on the local database but a username with this character is authenticated when it is configured on an ACS server.

Previously, this argument supported only alphanumeric characters.

Class map configuration

[line_number] match ...

The line_number option now is an integer from 1 to 255. Previously, this option was an integer from 2 to 255.

Object group configuration

udp operator radius-auth ...

Per CSCsr94846, the radius keyword is deprecated and is now radius-auth for Remote Authentication Dial-in User Service (port 1812).

Parameter map HTTP configuration

persistence-rebalance strict

Per CSCsy21634, the new strict option allows you to configure the ACE to load balance each subsequent GET request on the same TCP connection independently. For more information on this command, see the "Configuring Persistence with Load Balancing on Each HTTP Request" section.

Parameter map HTTP configuration

set secondary-cookie-start {none | text}

no set secondary-cookie-start

Per CSCsx75858, this new command either defines the ASCII-character string at the start of a secondary cookie in a URL or ignores any start string of a secondary cookie in the URL and considers the secondary cookie part of the URL.

The keyword and argument for this command are as follows:

none—The secondary cookie start is not configured or the ACE ignores any start string of a secondary cookie in the URL and considers the secondary cookie as part of the URL.

When you configure the none keyword to consider the entire URL query string as part of a URL, the commands that rely on the URL query, such as the match cookie secondary and predictor hash cookie secondary commands, do not work. Do not configure these commands under the same real server.

text—The start string of the secondary cookie. Enter a maximum of two characters. The default is ?.

Use the no form of this command to reset the start string to the default of ?.

Parameter map SSL configuration

cdp-errors ignore

no cdp-errors ignore

Per CSCsz83339, the new cdp-errors ignore command configures an SSL parameter map that ignores CDP errors when the crl best-effort command is configured for client or server certificate revocation checks. For more information on this command, see the "Configuring the ACE to Ignore Authentication Failures Due to CDP Errors" section.

Server farm and Serverfarm redirect

predictor hash cookie secondary cookie_name

The new secondary keyword selects the server by using the hash value based on the specified name in the cookie name in the URL query string, not the cookie header.

For the cookie_name argument, enter a cookie name as an unquoted text string with no spaces and a maximum of 64 alphanumeric characters.

For example, consider the following request:

GET /index.html?TEST=test123
Cookie: TEST=456

If you configure the predictor hash cookie secondary TEST command, it selects the server using the hash value based on test123. If you configure the predictor hash cookie TEST command, it selects the server using the hash value based on test456.

This option allows the ACE to correctly load balance in cases when the query string identifies the actual resource, instead of the URL. In the following example, if the ACE hashes on the URL, it would load balance on the same real server:

http://youtube.com/watch?v=C16mk4OfcuM
http://youtube.com/watch?v=cJ3jPzs2NLk

server farm host real server configuration

description text

The new description command allows you to provide a description for the real server in a server farm. Enter an unquoted text string with a maximum of 240 alphanumeric characters. If the text string includes spaces, enclose the string in quotes.


Displaying Detailed CRL-Downloading Statistics

To display the detailed statistics for the downloading of a CRL including failure counters, use the show crypto crl name detail command. Table 5 describes the fields displayed by this command.

Table 7 Field Descriptions for the show crypto crl crl_name detail Command 

Field
Description

URL

URL where the ACE downloads the CRL.

Last Downloaded

Last time the ACE downloaded the CRL. If the CRL is configured on an SSL-proxy service on a policy map that is not active or the service is not associated with a policy map, the field displays the "not downloaded yet" message.

Total Number of Download Attempts

Number of times the ACE attempted to download the CRL.

Failed Download Attempts

Numbers of times that the ACE failed to download the CRL.

Successful Loads

Number of times that the ACE successfully loaded the CRL.

Failed Loads

Number of times that the ACE could not load the CRL because of a failure.

Hours since Last Load

Number of hours that elapsed since the ACE last successfully downloaded the CRL. If no successful download has occurred, this field displays NA, not applicable.

No IP Addr Resolutions

Number of times the DNS resolution for the server host address of CRL the failed.

Host Timeouts

Number of download retries to the CRL that had timed out.

Next Update Invalid

Number of times that the next update field of the CRL was invalid.

Next Update Expired

Number of times that the next update field of the CRL was expired.

Bad Signature

Number of times that the signature mismatch for the CRL was detected, with respect to the CA certificate configured for signature verification of the CRL.

CRL Found-Failed to load

Number of times that the ACE could not load the CRL because of the maximum size limitation of 10MB on ACE or the formatting of the CRL was not recognized. The ACE recognizes only DER and PEM encoded CRLs.

File Not Found

Number of times that the server responded that the CRL file was not found at the server.

Memory Outage failures

Number of times that the ACE failed to download the CRL because it temporarily could not provide memory to store the CRL data.

Cache Limit failures

Number of times that the ACE could not load the CRL because the CRL cache was exhausted.

Conn Failures

Number of times that the ACE failed to download the CRL because it could not establish a connection with the server or no server entity was listening on the destination system.

Internal Failures

Number of internal failures in the ACE that hampered downloading the CRL, for example, internal communication failures between components responsible for the downloading the CRL.

Not Eligible for download

Number of times that the CRL was found ineligible for downloading because the following conditions:

The downloading of the same CRL is in progress.

The CRL has already been loaded successfully earlier and has not expired yet.

HTTP Read Failures

Number of times that the ACE encountered an error when downloading the CRL because it could not read data on the connection established with server.

HTTP Write failures

Number of times that the ACE encountered an error when downloading the CRL because it could not write the CRL download request from the connection established with the server.


System Log Messages

Software version A2(2.1) introduces the following new or revised system log (syslog) messages.

New Syslog Message

253011

Error Message    %ACE-6-253011: The CRL crl_Name may not be from a trusted source. 
Signature mismatch detected for CRL. 

Explanation    When the ACE performs signature verification on a CRL with a CA certificate configured with the crypto crlparams command, it detects a signature mismatch. Either the CRL (crl_name) download failed or the CRL has been removed from the ACE.

Recommended Action    Verify the CRL configuration for the crypto crlparams command.

Revised Syslog Messages

253004

Error Message    %ACE-6-253004: Certificate subject_of_certificate revoked, ssl-proxy: 
proxy_name, reason: reason

Explanation    This message is logged during the SSL handshake when client authentication is enabled. The ACE determines that the client certificate has been revoked by the CA. The subject_of_certificate variable is the subject field of the certificate. The proxy_name is the name of the SSL proxy service. The reason is the reason for the revocation of the certificate and has one of the following messages:

revoked—The certificate is revoked by the CA.

no workable cdps in cert—The certificate does not have a workable CRL distribution point (CDP). A CDP indicates the location of the CRL in the form of a URL.

crl download failure—The download of the CRL failed.

Recommended Action    None required.

253008

Error Message    %ACE-6-253008: CRL crl_name could not be retrieved, reason: reason

Explanation    This message is logged when the ACE failed to retrieve a CRL. If you define CRL checking for SSL client authentication, the ACE periodically retrieves a CRL. Due to a variety of reasons, these attempts can occasionally fail. The crl_name variable is the name of the CRL as defined by the crypto crl command. The reason variable is the reason for the CRL download failure. and can be one of the following messages:

DNS error

host conn timeout

memory outage

crl max size limit violation

crl cache full

crl data/file not found

invalid format of data

crl signature mismatch

next update field erroneous

next update field expired

internal error

not okay to download

http connection error

http file read error

http request writing error

ldap bind error

ldap search error

Recommended Action    Check to see if there is a network connectivity problem or if the server location of the CRL has changed.

253012 (formerly 253011)

Error Message    %ACE-2-253012: Crypto file storage failure: All certificates/keys were 
removed. Error: text_string

Explanation    A system failure deleted the SSL services internal database of certificates and keys. The text_string variable is either of the following:

Corrupted certificates/keys metadata found

Out of resources while trying to store certificates/keys metadata

Recommended Action    Contact Cisco TAC and send them the message output. Reimport the certificates and keys to maintain the integrity of the SSL services.

305010

Error Message    %ACE-6-305010: Teardown {dynamic|static} translation from 
interface_name:real_address to interface_name:mapped_address duration time

Per CSCsy34814, the duration time variable now displays the total duration time of the Xlate entry; the time that the entry was created until it expired. Previously, the duration time variable displayed the Xlate idle timeout. The duration time variable applies to dynamic NAT or PAT only.

305012

Error Message    %ACE-6-305012: Teardown {dynamic|static} {TCP|UDP|ICMP} translation 
from interface_name:real_address/{real_port|real_ICMP_ID}to 
interface_name:mapped_address/{mapped_port|mapped_ICMP_ID} duration time

Per CSCsy34814, the duration time variable now displays the total duration time, which is the time that the entry was created until it expired. Previously, the duration time variable displayed the idle timeout. The duration time variable applies to dynamic NAT or PAT only.

441001

Error Message    %ACE-5-441001: Serverfarm (name) failed over to backupServerfarm 
(backup_name) in policy_map (lb_Policy_Map). Number of failovers = count1, number 
of times back in service = count2 

Explanation    A serverfarm failover event has occurred. The name variable is the name of the serverfarm. The backup_name is the name of the backup serverfarm. The lb_Policy_Map is the name of the load-balancing policy map. The count1 variable is the number of times that the primary serverfarm failed over to the backup serverfarm. The count2 variable is the number of times that the primary serverfarm returned to service.

Recommended Action    None required.

441002

Error Message    %ACE-5-441002: Serverfarm (name) is now back in service in policy_map 
(lb_Policy_Map). Number of failovers = count1, number of times back in service = 
count2 

Explanation    A serverfarm in service event has occurred. The name variable is the name of the serverfarm. The lb_Policy_Map is the name of the load-balancing policy map. The count1 variable is the number of times that the primary serverfarm failed over to the backup serverfarm. The count2 variable is the number of times that the primary serverfarm returned to service.

Recommended Action    None required.

Software Version A2(2.0) Resolved and Open Caveats

This release note includes resolved and open defects that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(2.0):

Software Version A2(2.0) Resolved Caveats

Software Version A2(2.0) Open Caveats

Software Version A2(2.0) Resolved Caveats

The following resolved caveats apply to software version A2(2.0):

CSCsh02677—When you configure global service policies, the ACE downloads the rules in the class map to all the interfaces in the context. When the global policy has more than 10 class maps and the context has more than five interfaces, the process responsible for downloading these rules uses most of the CPU resources. Workaround: Perform either of the following:

If only a few interfaces in the context require the policy, apply it locally on these interfaces instead of globally.

If all the interfaces require the policy, then divide the global policy into multiple policies (one for each interface) with each policy having its own distinct class map and policy map. Then apply these policies locally to the interfaces.

CSCsh93373—When you remove multiple ACEs from an ACL that has been invoked inside a class map, the merge list is built and destroyed for each deletion and for each feature. If the downloaded ACL is large, it causes the CPU utilization of the aclmerge process to go high and other configuration and exec commands time out. Workaround: Remove the policy from the interface, make the changes, and then reapply the policy to the interface.

If you are making changes only to the ACL, do the following:

a. Remove the ACL from the class map with the no match access-list command.

b. Make the changes.

c. Readd the match access-list command to the class map.

CSCsi69881—When the ACE is configured with persistent rebalance and multiple class maps on the same policy map, unrelated changes performed under the policy map cause the Layer 7 connection to reset. Workaround: None.

CSCsk15979—When the timeouts for idle connections are set to infinite, the ACE clears the TCP connections after a configuration change occurs to the UDP class map. Both the UDP and TCP class maps share the same VIP and both have the idle timer set to infinite. A series of connections are made and are allowed to sit idle for hours. When a change is made to the UDP class map to time out these connections in 60 seconds, the ACE clears both the TCP and UDP connections. Workaround: None.

CSCsk78825—When you remove a NAT pool from an existing large configuration, entering the show commands causes API timeouts and the console to become unresponsive.Workaround: None.

CSCsm02293—For interface with multiple service policies, making configuration changes to an inspection service policy can cause the reset of existing connections on another policy. Workaround: None.

CSCsq14440—The aclmerged process in the ACE may not complete or may exceed the available system resources. With very large configurations where there are many ACLs, NAT statements, and class maps, the processing of these elements can require a significant amount of time and internal resources. In some cases, the processing (as displayed by the show proc cpu | include aclmerged command) may become unresponsive and never complete. In other cases, the processing may complete, but the output could exceed the resources available on the ACE, which may cause the ACE to not function properly.

Workaround and recovery: Currently, there is no method to predict the aclmerged response. However, in most cases, the commands eventually complete and the ACE continues to function properly. The suggested workaround is to allow aclmerged to complete without any intervention, assuming that there is no external impact to traffic. If the process does not complete or if there is a significant disruption to traffic flow, then reboot your ACE. If you enter the write memory command prior to the reboot, then the ACE attempts to come up in the post-change configuration. This may allow the desired configuration to be applied properly after the reboot. If you do not enter the write memory command before rebooting the ACE, then the ACE should reload and continue to operate in the same manner as before the change.

CSCsr22521—When you enter the show service command on the active ACE and enter the show running-config interface | be command on the standby ACE, an "Error: API call timed out" error message occurs. Workaround: None.

CSCsr72591—When you need to import many SSL keys and certificates, it may take a long time (approximately 30 minutes to import 1000 keys and certificates). You must import them one at a time; there is no bulk import feature available. Workaround: None. See the "Bulk Importing of SSL Certificates and Key Pair Files" section.

CSCsu42225—When you configure the ACE with a Layer 4 load-balancing policy map and it receives a series of UDP requests with a payload of 3,200 bytes that spans three nonfragmented packets, the ACE drops two packets from the first request. For subsequent requests, the ACE load balances all packets successfully. Workaround: None.

CSCsv31394, CSCsm46044—When you modify the policy-map configuration on an interface, the ACE occasionally records a service-policy download error. Workaround: None.

CSCsv32122—When you configure approximately 8,000 match source-address statements, you can see traffic drop for 10 to 20 seconds with a lockup of the console or terminal. Workaround: None.

CSCsv33051—When you configure RADIUS load balancing and create a RADIUS-attribute sticky group with the sticky radius framed-ip command, if the Framed-IP-Address is reused and load balanced to a different rserver, the ACE may not update the sticky entry. Workaround: Configure the RADIUS client to issue Framed-IP-Addresses and include them in the RADIUS access request messages or configure separate Framed-IP-Address pools for each RADIUS real server.

CSCsv52288—This enhancement allows the ACE to support 16,384 match source-address statements entries. The previous limit was 8,192.

CSCsv52887—The ACE may experience a short lockup period of the console or terminal when you modify match source-address entries in a configuration with a large number of match source-address statements under a high traffic load. Workaround: None.

CSCsv56901—When you enable client authentication and a CRL on the ACE, the CRL applied to the SSL proxy under traffic could cause a memory leak. Workaround: None.

CSCsv56991—After removing the real server configuration on a server farm and reconfiguring a real server with the same configuration, the connections may not get replicated. After one failover, both active and standby ACEs are synchronized. But after another failover, the standby ACE is not synchronized with the active ACE.

CSCsv59066—When using KAL-AP to report the VIP address status, all VIPs with the same addresses report a load of 255 if one is out of service. Workaround: Do not use KAL-AP to monitor multiple VIPs with the same IP addresses.

CSCsv61295—When you configure the ACE with SIP inspection, when the SIP message contains the letters "tel" before the sip: information, packet drops occur. Workaround: None.

CSCsv89746— In the ACE 2(1.2) release, the logging rate-limit command adds an extra "1" in the running configuration which causes the command to function incorrectly. Workaround: Do not use the logging rate-limit command.

CSCsv94341— When you configure a class-default class map in a RADIUS policy map, RADIUS accounting on or off packets are dropped. This behavior occurs due to an incorrect check on the empty rule list. Workaround: Configure a class map other than the default traffic class in the RADIUS policy map.

CSCsw14149—Applying an ACL to an interface and then expanding it to 24,000 access control elements takes over 9 hours. Workaround: None.

CSCsw14181— When you apply an ACL with 24,000 access control elements to an interface and then remove it, an "Error: Called API timed out" message occurs. Workaround: Remove the ACL from the interface and then remove the ACL.

CSCsw18441—When you enable the access-list debug errors, the ACL Hit count statistics are displayed. Workaround: None.

CSCsw18450—In a large configuration, deleting a class map from a policy that is applied on an interface takes 10 minutes for one class map. The show commands return with failures and the aclmerge process takes more than 90 percent of the CPU for more than 10 minutes. Workaround: None.

CSCsw18452—Adding a service policy under an interface takes more than 12 hours. The show commands return with an "API call timed out" error and the aclmerge process takes more than 90 percent of the CPU for a long time. Workaround: None.

CSCsw20096—Configuring the logging level does not work for some syslogs. The running-config shows the updated value, but the actual syslog generation is based on the default level. Workaround: Set the logging levels for the console and buffering based on the default levels of these syslogs.

CSCsw28726—When the ACE is configured to insert the client source IP address into the back-end HTTP header, it may intermittently insert an incorrect source IP address in the HTTP header. Workaround: None.

CSCsw29087—On rare occasions, the ACE drops RADIUS load-balanced traffic due to buffer exhaustion. Layer-4 load-balanced traffic is unaffected. The ACE will be out of buffers on at least one network processor, as indicated when the following counter increments:

show np [1|2] me-stats -socm
Drop [out of connections]:                    96023             0

Workaround: Reboot the ACE.

CSCsw35807—The crypto import command may fail but may not report an error because the ACE is running out of space on the secure storage area of the flash memory. For example, the crypto import terminal command may not report an error implying that it was successful, while the crypto import sftp command may report the following message:

Successfully imported file from remote server

Key pairs and certificates imported under these circumstances may or may not appear in the show crypto files command output, may not be usable, and may disappear on the first ACE reboot. Rebooting the ACE makes the failure of the crypto import commands obvious.

Workaround: Avoid importing files other than valid SSL key pairs and certificates. Avoid importing large numbers of excessively large key pairs and certificates. Avoid exceeding the maximum supported number of key pairs and certificates.

CSCsw35954—If the ACE runs out of space on the secure storage area of the flash memory during the execution of the crypto import command before it is rebooted, it may keep repeatedly rebooting until you power it off or replace its flash storage card. Workaround: Reboot the ACE with a software version that contains the A2(2.0) fix for this issue. If necessary, reboot the ACE again and boot it back with the original image. If booting the ACE with a newer image is not an option, perform the following steps:

a. Reboot the ACE with different software versions until you find one that successfully boots and allows you to log in to the ACE.

b. Delete all crypto files using the crypto delete command.

c. Reboot ACE again and boot it back with the original image.

CSCsw41402—Applying a packet capture may render the context unusable as the merged list on the incoming interface is deleted. Workaround: None.

CSCsw49482—When you or users enter a long-running SSL PKI command (for example, the crypto generate or show crypto command) in the same or in a different context, if you press Ctrl-c to abort it, the session may become unresponsive and various SSL PKI commands may fail sporadically with surprising error messages or with an "Error: API timed out" message. Previously-imported key or certificate files may disappear without any indication. Workaround: Do not abort long-running SSL PKI commands by pressing Ctrl-c, or wait at least three minutes after aborting one of these commands using Ctrl-c before entering another SSL PKI command.

CSCsw52831 —If a RADIUS packet is the second packet on a UDP connection and it is received shortly after the first RADIUS packet on the connection, it may be dropped. Workaround: None.

CSCsw63921—When you configure the ACE with a Layer 7 rule and persistence rebalance, it does not load balance a large Post packet correctly. The ACE sends half of the data to one server and the second half to another server within the default class. The show http stats command displays static parse errors. Workaround: Remove the persistent rebalance configuration.

CSCsw69707—In earlier ACE releases, the set tcp buffer-share command was configurable only for TCP connections. This command now applies to UDP connections. However, the CLI remains unchanged.

CSCsw71243—When you enter one of the following commands while another one from this list is executing in the same or different context, the commands may spontaneously fail and may report an unrelated error or no error at all:

crypto delete

crypto export

crypto generate csr

crypto generate key

crypto import

crypto verify

show crypto authgroup

show crypto certificate

show crypto chaingroup

show crypto files

show crypto key

This problem may cause all of the key pair or certificate files that were previously imported or user generated to disappear from the ACE either immediately or after the first reboot. Workaround: Do not enter more than one command from the above list simultaneously on the ACE, not just context-wide.

CSCsw75536—The ACE may stop splicing TCP sequence numbers between the front-end and back-end connections of a load-balanced connection. Initially the connection may operate with several successful HTTP transactions. However, the connection may eventually fail due to the ACE sending the TCP sequence numbers from the front-end connection to the back-end real server. Workaround: None.

CSCsw77807—SIP probes with random Call-IDs and From-Tags in the SIP options may fail with the Cisco Session Border Controller (SBC). The SBC responds with a SIP "482 Loop Detected" message because the same Call-Id and From-Tag are used in all requests. Workaround: Do not use SIP probes with Cisco SBC.

CSCsw83500—The show conn protocol tcp | inc CLSRST command displays a large number of connections. Workaround: Enter the clear flow command for all flows in the CLSRST state to free the buffers.

CSCsw99769—Under some conditions with the A2(1.2) and A2(1.3) releases, when some QNX processes (such as ssl_Hs) receive an abort signal, the ACE may not create a set of core files and does not reboot. Instead, the ACE may become unresponsive and the core files may be incomplete or nonexistent. The behavior is different between NP1 and NP2. Workaround: Manually reboot the ACE.

CSCsx01630—If the ACE is configured with multiple service policies for an interface, deleting a service policy can cause Layer 7 connections to reset in other service policies. Workaround: None.

CSCsx03110—When a service policy id applied globally on several interfaces and the ssl-proxy command is applied to the policy, the traffic is not SSL offloaded and is forwarded as if there is no SSL proxy configured. Workaround: Apply the service policy locally on the relevant interfaces.

CSCsx11078— Whenever you import PKI key or certificate files and then delete them, a memory leak occurs on the ACE. Workaround: None.

CSCsx11478—When a large number of certificates and keys are imported to the ACE, SSL-related configurations are lost after the ACE reboots. The ACE displays the following error messages:

`crypto crl test1 http://10.7.107.68/crl1`  *** Context 0: cmd exec error ***  

Workaround: None.

CSCsx19410—When a service policy is applied globally and SSL traffic is load balanced through an interface, if you remove another interface, the active interface drops the SSL traffic and no load balancing is performed. Workaround: Apply the service policy locally on the relevant interface. Then, reconfigure the interface that was removed.

CSCsx25224—During a large VIP test, ipcp stall messages occur continuously with traffic. These messages cause the IPCP Q to stall for the cfgmgr. The cfgmgr enters the Suspend state and all of the tasks related to cfgmgr will become unresponsive.Workaround: None.

CSCsx26856—The ACE becomes unresponsive when reconfigurations are performed on a large number of VIPs (up to 1,500). Workaround: Avoid frequent reconfigurations of a large number of VIPs. Do not use CRLs with VIPs.

CSCsx33084—When you configure the ACE with a front-end SSL termination proxy that includes client authentication and session reuse, refreshing a page in a Firefox, Netscape, or Opera browser does not work and the page fails to load in the browser.

CSCsx33515—The ACE becomes unresponsive when you apply a chain group to a large number of VIPs (up to 1,500) and then change the chain group. Workaround: None.

CSCsx61234—When the ACE imports a certificate or key pair with a filename of 40 characters and a passphrase of any valid length, it is not displayed through the show crypto files command and is not usable. Workaround: Reduce the filename to 39 characters and the problem does not occur.

CSCsy03713—When the ACE reboots and the primary server farm is out of service, traffic does not switch to the backup server farm. Workaround: Configure a real server under the primary server farm that could trigger the failover again.

Software Version A2(2.0) Open Caveats

The following open caveats apply to software version A2(2.0):

CSCsj68643—The following log messages may appear sporadically in the ACE log:

"can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg"

"can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg"

These messages do not impact the operation of the ACE. The messages may be caused by more than one device that is accessing the ACE context through XML. Workaround: None.

CSCsj74250—When you configure the TACACS+ server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, then there is a key mismatch when the ACE attempts to authenticate a user. Workaround: Paste the properly encrypted key into the running-configuration file.

CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, then subsequently try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in RSA format instead of RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.

CSCsl21191—When you enter the show module command on the supervisor engine for a running ACE, the command output may fail to display the software version information from the ACE. When this behavior occurs, the command output displays similarly to the following example output:

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
 4  0018.b9a6.88fc to 0018.b9a6.8903   1.1   8.6(0.252-En 8.6(0.252-En Ok  

This behavior rarely occurs, but once it does, the behavior will continue every time that you enter the show module command. The ACE continues to forward traffic normally. This is a display problem only. Workaround: Reboot the ACE.

CSCsl46334—When a high rate of Layer 7 load-balanced traffic is flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse is configured, the ACE may start dropping traffic after a few hours. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health checks. Workaround: None.

CSCsl75662—You may observe that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time that the probe is sent even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from the real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform either of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, and then log in again to access a different context than the original context with the configured packet capture function.

CSCsm93110—When you configure Microsoft Internet Information Services (IIS) version 5.0 to accept client certificates, SSL initiation through the ACE may fail. Workaround: Upgrade to IIS version 6.0.

CSCso33506—In a redundant configuration with the FT pair running mismatched code (A1(x) and A2(x)) and mismatched licenses, if the active ACE is rebooted, the Admin context comes up, but, in user contexts, the running-config file is blank. This behavior occurs only when there is both a license and a code mismatch. Workaround: Resolve one of the mismatches and reload the ACEs or enter the copy start run command in each user context.

CSCso38853—After four consecutive Route Processor Redundancy (RPR) failovers in the Catalyst 6500 series switch, the primary and standby ACEs may enter the Active-Active state. This state is not resolved until you reload the primary ACE. Workaround: None.

CSCso55790—While trying to copy core dump files from the core: directory to an FTP server, the copy operation failed with the following permission denied error message:

local: /TN-COREFILE/core.618: Permission denied

Workaround: Copy the files from debug or from the console after you modify the permissions using debug.

CSCso60304—When an invalid XML attribute is sent to the ACE, it does not respond as expected. Instead, the ACE displays a 500 Internal Server Error message. No negative impact to the ACE is observed. Workaround: None.

CSCso76159—When you dynamically modify a service policy to use an HTTP parameter map with the header modify per request command, the ACE does not insert a header into every GET request for existing long-lived persistent flows. However, the ACE does insert a header into every GET request for new flows. Workaround: None.

CSCso80478—When you perform multiple parallel SNMP walks that last 30 seconds or longer on an ACE in a redundant configuration, you may observe response timeouts on both the active and the standby ACEs. You may also observe this behavior in multiple contexts. This behavior does not occur with SNMP walks of shorter durations. Workaround: None.

CSCso81785—If you are using TACACS+ and the Cisco Access Control Server (ACS) with an RSA authentication manager, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another network access server (NAS) to set your PIN.

CSCso81811—If you are using TACACS+ and the Cisco ACS with an RSA authentication manager and your account is in next token mode, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another NAS to enter the next token code and make your account accessible again.

CSCso82657—While moving a VLAN from a Cisco Firewall Services Module (FWSM) to an ACE or from an ACE to an FWSM, IP routing is not updated on the ACE to reflect the change. You can observe this behavior when making a change to the svclc commands and the shut and no shut commands on interfaces on the ACE. Workaround: None.

CSCso82971—If you are using a TACACS+ server that is an RSA server with TACACS+ continue authentication, authentication may fail to the configured server, but you still can log in using local authentication.

Use one of the following workarounds:

Use the Cisco ACS instead of the RSA server.

Do not configure local as the secondary authentication method.

CSCso85639—If you configure the passdetect interval command value for less than 30 seconds, the ACE sends overlapping probes that use additional management connections (resources). Workaround: Increase the passdetect interval command value to 45 seconds.

CSCso86485—When a client-side VLAN interface is brought up and down an excessive number of times on the active ACE under a light traffic load, the standby ACE may generate a core dump. Workaround: None.

CSCso95457—When you enter the clear conn all command, the ACE sends an RST to close the connection only to the server and purges both the inbound and outbound connection entries from its connection database. As a result, the client connection is left open and any further requests arriving on that connection are not serviced. Workaround: None.

CSCso95620—With long-lived HTTP, SSL, FTP and UDP traffic on the ACE, you may observe a memory loss of approximately 333 KB in the ACE during an EtherChannel link (FT port channel) failure and recovery on the Catalyst 6500 series switch. Workaround: None.

CSCsq03035—The ACE was configured with an idle timeout of 0 (never time out), while TCP and UDP traffic was sent and left in an idle state over an extended period of time. The idle timeout was then changed from infinite to 60 seconds. The UDP traffic was immediately cleared, while the TCP traffic was not. After waiting more than 15 minutes, the idle TCP flows still had not been cleared. Workaround: None.

CSCsq23701—After an FT VLAN failure, which resulted in an Active/Active FT state, has been resolved, the ACE with the higher priority should take over as the active ACE (even though the preempt command is disabled) through the election process, but did not. Workaround: Enter the preempt command.

CSCsq27062—After toggling the state of the FT port channel in the Catalyst 6500 series switch 110 times, the primary ACE module generated a core dump and reloaded. Workaround: None.

CSCsq64401—If you configure the switch-mode command in an ACTIVE user context in a redundant configuration, the command is not synchronized to the STANDBY_HOT user context on the other ACE. This behavior occurs only in a redundant configuration where an ACE has its Admin context in the STANDBY_HOT state and a user context in the ACTIVE state.

There are two possible workarounds for this behavior as follows:

Never allow a user context to be in the ACTIVE state on the standby ACE.

Reload the ACE that has its user context in the STANDBY_HOT state.

CSCsq87162—SSL transactions may not complete when the server-conn reuse command is enabled. Workaround: Disable the server-conn reuse command.

CSCsq99448—When you upgrade the ACE from version A1(6.3a) to A2(1.1), you might experience unresponsiveness in the outbound connection manager (OCM) because of the deletion of an improper internal message. Workaround: None.

CSCsr09129—When you configure SIP load balancing with inspection enabled, the ACE should open a pinhole to the address in the Via header for the server response. However, the server responses remain in the data channel. Workaround: None.

CSCsr14898—XML output for the show serverfarm detail command is not valid XML. If the server farm does not have a configured probe, the generated XML output still contains a close tag </sf_probes> and does not have an open tag <sf_probes>. Workaround: Configure a probe in the server farm. If a probe is configured on the server farm, then there should be both an open tag and a close tag present in the XML output. If a probe is not configured on the server farm, then neither tag should be present.

CSCsr18029—The ACE may reload after an SNMP query. Workaround: None.

CSCsr22703—The ACE became unresponsive and generated a core dump while it was executing an OS kernel function. This behavior appears to have been a one-time event. Workaround: None.

CSCsr62027—When TCP normalization is disabled, the ACE places replicated TCP connections in the INIT state on the standby ACE. After the normal embryonic connection timeout occurs, the ACE removes the replicated connections from the standby. Workaround: Do not disable normalization.

CSCsu49899—When an HTTP virtual server that performs Layer 7 inspection shares the same virtual IP addresses as other servers, the ACE responds to SYN requests whether or not the Layer 7 virtual server is up or down. The ACE completes the three-way handshake before sending an RST. Workaround: Make sure that HTTP Layer 7 virtual servers have unique virtual IP addresses or all of them use the same one to ensure the other protocols do not get spoofed unnecessarily.

CSCsu60137—When the ACE issues a POST request, an SSL bad-record MAC error occurs with Firefox Version 2 and 3. The same POST request works with Microsoft IE. Workaround: None.

CSCsu67523 and CSCsu67556—Upgrading the ACE software to version A2(1.1a) causes the ACE to reboot and generate a core dump. Workaround: None.

CSCsu67539—When you upgrade the ACE software to version A2(1.1), the ACE reboots and generates a core dump. Workaround: None.

CSCsu67574—When you upgrade the ACE software to version A2(1.0a), the ACE reboots and generates a core dump. Workaround: None.

CSCsu68314—When the ACE becomes unresponsive and generates a core dump, the core-dump file contains three different types of files. These files should be separate files. Workaround: Use the file command to uncompress the core-dump files.

CSCsu68366—The ACE reboots and generates a qnx_2_mecore_log.999.tar.gz core-dump file. Workaround: None.

CSCsu80235—When you configure stickiness on a context and the sticky database lookup is 8,192 over the maximum threshold, the ACE drops connections causing the users to experience resets or their pages do not load properly. The Drop Max Remote Stky counter displayed by the show np [1 | 2] me-stats -slb command continues to increase. Workaround: Force a failover to the backup ACE and reboot the module that had the problem.

CSCsu86606—When you reboot the ACE and as it powers up, the Catalyst 6500 series switch disables the ACE and displays the following log messages:

Oct  1 07:43:25.710 EDT: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off 
(Reset)
Oct  1 07:43:41.611 EDT: %OIR-SP-6-PWRFAILURE: Module 1 is being disabled due to power 
convertor failure 0x1 

Workaround: None.

CSCsu88684—When a large number of Layer 2 connected real servers are in the ARP FAILED state and each real server is associated with probes, the ACE becomes unresponsive, displays the following messages on the console, and eventually reboots:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

Then the ACE reboots. Workaround: None.

CSCsu95356—When you configure the ACE with the predictor least conn command, the real server does not get the expected number of connections. Workaround: Remove the real server from the server farm and readd it.

CSCsu95887—After the active ACE module completes configuration synchronization, it generates a core dump. Workaround: None.

CSCsu96977—When you configure more than 640 action lists and enter the do show action_list command with the Tab or ? key for help, the ACE becomes unresponsive. Workaround: None.

CSCsv02224—When you configure and remove an SSL-proxy service after you configure and remove multiple class maps under a policy map, the following error appears on the console:

Error: Called API encountered error appears console.

The ACE rejects the ssl-proxy command and the command does not appear in the configuration. Workaround: None.

CSCsv02360—When you configure the ACE with SSL termination and server connection reuse, and a client makes an HTTPS request to the VIP address, some connections fail if the client MTU is low (for example, an MTU of 576). Workaround: None.

CSCsv04319—If you create a TACACS+ server with a numeric key, the ACE sends a warning about the key; however, it does not create the server. The message should be an error and not a warning. Workaround: Use a key that is not entirely numeric.

CSCsv04848—When you configure RADIUS on the ACE and a user logs off, the RADIUS client sends an accounting stop message to the server for that user but the ACE does not immediately delete all connections for that user. If the source IP address for the user is immediately reassigned to another user, the new user could open a new connection before the old connections from previous user times out. The result is that the ACE incorrectly forwards the new connections and does not load balance the packets. Workaround: Set the UDP connection timer to a smaller number (for example, 10 seconds).

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv10547—The config-register setting does not synchronize after an ACE module boots. The config-register setting synchronizes only when you configure it with ACE modules in active or standby mode. Workaround: None.

CSCsv31046—When you configure the least-connections predictor on the ACE, the ACE may not sustain 160,000 CPS traffic. Workaround: None.

CSCsv31476—When the ACE generates a core-dump file for the kernel or Virtual Shell (VSH) applications, the file does not contain the code-train version information. Workaround: None.

CSCsv47724—The heartbeats on fault-tolerant (FT) ACE modules occasionally miss due to late TCP timers. The FT ACEs increment the Heartbeats Missed counter on the standby ACE and the Unidirectional HB's Received counter on the active ACE. Workaround: None.

CSCsv48498—When you enable FTP inspection and disable normalization on the client-side interface, the ACE inserts the TCP Option Timestamp in packets to the client and the FTP server, even if both the client and the server are not using this option. Workaround: Enable normalization or disable FTP inspection.

CSCsv49518—The ACE becomes unresponsive due to the ICM being stuck at 100 percent in the proxy_connection_stack_lock state. Workaround: None.

CSCsv49606—When you configure stickiness on the ACE, the ACE becomes unresponsive. Workaround: None.

CSCsv52331—The ACE becomes unresponsive due to an SRAM parity error. Workaround: None.

CSCsv52478—When you reboot the Catalyst 6500 series chassis, the ACE may reboot as Active. Workaround: None.

CSCsv53112—When you enter the show xlate command, the ACE may generate a core dump. Workaround: None.

CSCsv53187—The ACE generates an NP ha_hb_g_ns core dump during a standard operation. Workaround: None.

CSCsv53620—When you add an SSL proxy class to a policy map, the following error occurs:

Error: Called API encountered error

Workaround: Remove the class from the policy map and then readd it.

CSCsv65178—When you specify TCP as the protocol in a class map configured for DNS traffic, the ACE allows the configuration and DNS inspection fails. Workaround: Specify UDP as the protocol in a class map configured for DNS traffic.

CSCsv69769—When you configure an expect regex value, the ACE allows a space in the quoted name of the value. Workaround: Do not use a space. Instead, use a search character (.*) or allow the variable to be on a long string input.

CSCsv95254—When an IP address conflict occurs on a bridged VLAN, the ARP manager may become unresponsive causing the ACE to generate a core dump. Workaround: None.

CSCsw40764—When the ACE executes the no access-list command to delete an ACL configured with 64,000 entries, API timeout occurs. Workaround: Do not delete all of these entries from an ACL at one time. Delete the entries from an ACL one at a time or in small chunks.

CSCsw81300—When you configure the ACE with the combination of HTTP inspection, HTTP load-balance policy map with only a class-default class, server-connection reuse does not allow traffic. Workaround: Change the class map in the HTTP load-balance policy map from a class-default class map to a type HTTP load-balance class map.

CSCsw82768—When the ACE runs end-to-end SSL traffic at a rate of 1,000 to 2,000 TPS, proxy entries may leak on the standby ACE. Workaround: None.

CSCsw97987—When you configure multiple class maps to a multi-match policy map and you send traffic to a class map, if you delete and readd all of the other class maps, the traffic destined for the remaining class map gets a hit when you try to readd it to the same policy map. Workaround: In a multi-match policy map with more then one class map, do not delete and readd all class maps except the one where you are sending the traffic.

CSCsw98274—When you add and remove the class map along with the SSL proxy from a multi-match policy map multiple times, if you attempt to add a class map and then try to apply an SSL proxy, the "Error: Called API encountered error" message occurs and the proxy is not applied to the class map. Workaround: Do not add and remove the class map from a multi-match policy map too quickly. If this situation continues, reboot the ACE.

CSCsx05150—When using 2048-bit certificate and key pairs with block and export ciphers, a rehandshake may lead to stuck connections. Workaround: Either use nonblock and nonexport ciphers or use certificate and key pairs that are less than 2048 bits.

CSCsx08589—After the ACE takes a long time to boot with some errors on the console or terminal, the Admin user behaves as a network-monitor user. After another reboot, the ACE loads and the Admin user has Admin privileges but the SSL-proxy configuration in the Admin context has lost certificates. The Admin context includes several VIPs with the SSL-proxy configuration and the configuration includes several contexts. Workaround: Define the VIPs in a context other than the Admin context.

CSCsx11453—When you remove and apply a service policy several times on the client VLAN while traffic is running on the ACE, the ACE becomes unresponsive. Workaround: Do not change the service policy while traffic is running on the ACE.

CSCsx13061—When you perform a checkpoint rollback in a specific order or execute a match and no match statement under a class map, ACL memory is leaked and some entries configured in the ACL are not removed from the interface. Workaround: Remove the interface and readd it.

CSCsx13147—When you import a number of SSL PKI key or certificate files into a context by using the crypto import command, if you remove the context without first removing the files through the crypto delete command, the ACE may not import additional SSL PKI key or certificate files stating that the failure is due to a lack of resources or, during a subsequent files import process, some or all of the previously-imported key or certificate files may be forcibly removed from some or all contexts. These symptoms disappear after you reboot the ACE. Workaround: Use the crypto delete command to explicitly delete the SSL PKI key or certificate files from contexts before removing the context. Rebooting the ACE also alleviates this problem if it has already happened.

CSCsx13274—When the ACE SSL is at peak performance, a leaked SSL context state occurs that cannot be detected with show commands. Workaround: None.

CSCsx19525—When you configure 1,000 SSL VIPs on the ACE and then you change the configuration on those VIPs, a buffer leak occurs as displayed by the show np 1 me-stats command "-scommon" output and traffic conditions. Workaround: Reboot the ACE and do not make configuration changes that affects those VIPs.

CSCsx24893—When you update a 1,500 VIPs, a change in one context affects traffic in another context. Workaround: None.

CSCsx27063—When you apply rules with more than 100,000 elements on an interface, the show acl-merge and show np 1 commands show that the rules are still applied after crossing the 100,000 limit per interface. Workaround: None.

CSCsx28587—When the maximum aclmerge instance limit of 8191 is reached and then freed, ACL merge will not occur. Also, after reaching the maximum limit of instances, if you remove the outbound ACL from the interface, the policy action nodes are not released. Workaround: None.

CSCsx28656—When you create a large configuration consisting of interfaces and ACLs in a redundant configuration, if you remove a context from the active ACE, the context is not removed from the standby and the standby ACE transitions to the Hot state even though config-sync failed. Workaround: Place redundancy out of service. Remove the configuration manually from the standby ACE and place redundancy in service.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx38885—When the ACE contains a large configuration, if you quickly add and remove multiple class maps under a Layer 7 policy map, API timeout errors occur. Workaround: Do not add and remove class maps under a Layer 7 policy map in quick succession.

CSCsx41858—When you configure redundancy on the ACE and it reboots, IP connectivity to and from the ACE fails. For example, if you Telnet or ping to or from the ACE, it fails. All the interface are down for the following reason:

VLAN not assigned from the supervisor

Workaround: Reconfigure the VLANs and the svclc module number vlan-group number command on the Supervisor module.

CSCsx47594—When an SSL server does not use an RSA certificate and the ACE does not determine that the certificate is not RSA, the ACE becomes unresponsive under SSL backend traffic including the HTTPS probes. Workaround: Make sure that the SSL server uses an RSA certificate.

CSCsx52128—When you copy a large configuration with a lot of ACLs to the running-config file and perform other configuration changes continuously, the aclmerged process does not get the CPU and also the configurations result in API errors. Workaround: When you copy a large configuration with a lot of ACLs to the running-config file, wait approximately 2 minutes for it to complete. Do not perform any configuration changes at that time.

CSCsx55228—When you remove an entry with an object group from an ACL which is associated as global access group and then readd it, merge errors occur and nonallowed traffic goes through the ACE. Workaround: Unconfigure and then reconfigure the access group.

CSCsx62330—When you configure one or more contexts with an SSL configuration and HTTPS probes, if you import 2,000 or more certificates and keys and then reboot the ACE, the probes fail. The problem does not occur if you do not reboot the ACE after the configuration. Workaround: If possible, reduce the number of certificates and keys to below 2,000 and then reboot the ACE.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: None.

CSCsx80970—When you configure a multi-match policy map with more than one class map, if you perform an inspect policy change in a class map, the traffic to other class maps may be hit. Workaround: Do not make any inspect changes on the multi-match policy map when traffic is running.

CSCsx93137 and CSCsx93995—When you enter one of the following commands in any context but do not complete entering the remote host password when prompted, the ACE waits for your input:

crypto import ftp | sftp | {bulk ftp}

crypto export ftp | sftp

Then, if you enter one of the following commands, the session may appear to be in an unresponsive state:

crypto delete

crypto export

crypto generate csr

crypto generate key

crypto import

crypto verify

show crypto authgroup

show crypto certificate

show crypto chaingroup

show crypto files

show crypto key

After a while, the command aborts with a "SSL PKI subsystem is busy. Please try again later" message. Reissuing the command results in the same behavior.

Workaround: Enter the remote host password as requested by the associated crypto import | export command. If the problem persists, clear the relevant sessions by executing one of the following commands:

clear users

clear telnet session_ID

clear ssh session_ID

You can execute those command if you have the appropriate privileges (for example, Admin). For details about role-based access control (RBAC) and user roles, see the Cisco Application Control Engine Module Virtualization Configuration Guide.

CSCsy04371—When a server farm with no backup transitions to the Inactive state after all the real servers transition to the MAXCONNS state, if the real servers transitions out of the MAXCONNS state, they may not accept connections. Workaround: Configure a backup to the server farm.

Command Changes from Software Version A2(1.1) to A2(2.0)

Table 8 lists the commands and options that have been changed from software version A2(1.1) to A2(2.0).

Table 8 CLI Commands Changed from Version A2(1.1) to A2(2.0)  

Mode
Command and Syntax
Description

Exec

crypto delete

crypto export

crypto generate csr

crypto generate key

crypto import

crypto verify

The crypto commands are now disabled by default for the network-monitor role.

Note that the ACE does not execute any crypto commands or the following show crypto commands in parallel:

show crypto authgroup

show crypto certificate

show crypto chaingroup

show crypto files

show crypto key

When you enter one of these commands while another is executing, the ACE blocks the command from executing until the active command is finished. If you enter more than one command while another is executing, the order they are processed is undefined.

If the blocked command times out before it executes, the following message appears:

SSL PKI subsystem is busy. Please try again later

You can reenter the command at a later time.

You can press Ctrl-C to cancel a blocked command. When a crypto command is executing, pressing Ctrl-C may not cancel it.

Exec

crypto import [non-exportable] bulk sftp [passphrase passphrase] ip_addr username remote_path

The crypto import command has been expanded to include a bulk keyword and its options and arguments. For more information on this command, see the "Bulk Importing of SSL Certificates and Key Pair Files" section.

Exec

crypto import [non-exportable] {{ftp | sftp} [passphrase passphrase] ip_addr username remote_filename local_filename} | {tftp [passphrase passphrase] ip_addr remote_filename local_filename} | terminal local_filename [passphrase passphrase]

The local_filename and passphrase arguments now support a maximum of 39 characters.

Exec

ft swtichover

The ft command is now disabled by default for the network-monitor role.

Exec

show connection serverfarm name detail

The new detail option displays detailed information for the server farm connection including idle time, elapsed time, byte count, packet count, and state of the connection in the reuse pool.

Exec

show crypto authgroup

show crypto certificate

show crypto chaingroup

show crypto files

show crypto key

Note that the ACE does not execute any crypto commands or these show crypto commands in parallel. When you enter one of these commands while another is executing, the ACE blocks the command from executing until the active command is finished. If you enter more than one command while another is executing, the order they are processed is undefined.

If the blocked command times out before it executes, the following message appears:

SSL PKI subsystem is busy. Please try again later

You can reenter the command at a later time.

You can press Ctrl-C to cancel a blocked command. When one of these commands is executing, pressing Ctrl-C may not cancel it.

Exec

show crypto cdp-errors

The new cdp-errors keyword displays the statistics for discrepancies in CRL Distribution Points (CDPs) for the certificates on the ACE. A CDP indicates the location of the CRL in the form of a URL. CDP parsing in the certificate occurs only when best effort CRL is in use.

The output for this command includes the following fields:

Incomplete—Number of times that the CDPs are missing information required to download the CRLs, for example, host, file name or base information.

Unrecognized Transports—Number of times that the ACE does not recognize or support the transport mechanism in the CDP for the CRL.

Malformed—Number of times that the CDPs are malformed with erroneous information, for example, specifying an incorrect attribute or base information. This counter also includes CDPs with URL lengths exceeding the ACE limit of 255 characters; a truncated URL could point to the wrong CRL.

Missing from cert—Number of times that the CDPs are missing from the certificate.

Exec

show crypto crl name detail

The new detail keyword displays additional statistics for CRL download failures. For information on the fields for this command, see the "Displaying Detailed CRL-Downloading Statistics" section.

Exec

show crypto crl best-effort

The new best effort keyword displays summarized information for all best-effort CRLS on the ACE (a maximum of 16 CRLs). The output for this command includes the following fields:

Best-Effort CRL—Identifier to distinguish each best-effort CRL present at this time. At another time, the identifier can vary for the same CRL.

CRL Distribution Point—URL of the CDP. The ACE displays the first 255 characters of the URL.

CRL Downloaded—Whether the CRL is downloaded on the ACE, Yes or No.

CRL Issuer Name—Name of the CRL issuer. The ACE displays the first 255 characters of the name.

Last Update—Contents of the Last Update field extracted from the CRL. The ACE displays the first 64 characters in the field

Next Update—Contents of the Next Update field extracted from the CRL. The ACE displays the first 64 characters in the field.

If no best-effort CRL exists on the ACE, the ACE displays the following message:

No best effort crl present in the system

Exec

show ft group detail

When the redundant ACEs have incompatible CLI images during an upgrade or downgrade, now the Running cfg sync status and Startup cfg sync status fields display the following message:

Config sync disables when peer is not fully CLI 
compatible

Previously, these fields displayed the following message:

Config sync disabled when peer is of lower version

Exec

show kalap udp load {all | vip tag name}

The new vip tag keyword displays the latest load information for the specified VIP tag name.

The all keyword now displays information for all VIP tags. For more information on this command, see the "Displaying the Load Information for a VIP KAL-AP Tag" section.

Exec

show service-policy [policy_name [class-map class_name]] [detail | summary | url-summary]

Added the optional class-map class_name, summary, and url-summary options to this existing command. You can now specify summary statistics for server load-balancing policies. In addition, you can specify detailed or summary statistics for a particular policy with all its associated class maps or a particular class map associated with a particular policy.

The output of the summary option in tabular format includes the following fields:

Service-policy—Unique identifier of the policy map.

Class—Name of the class map associated with the policy map.

VIP—Virtual IP address specified in the class map.

Protocol—Protocol specified in the class map.

Port—Port specified in the class map.

VLAN—VLAN ID of the interface to which the policy map has been applied.

State—Operational state of the VIP. Possible states are IN-SRVC (in service) and OUT-SRVC (out of service).

Curr Conns—Number of active connections to the VIP.

Hit Count—Total number of requests for the VIP.

Dropped Conns—Number of requests for the VIP that were dropped.

For information on the url-summary option, see the "Displaying the Layer 7 Match HTTP URL Statement Hit Counts Feature" section.

Exec

show stats crypto client | server

The SSL CRL download failed field has been removed.

Exec

show stats kalap [all]

The new optional all keyword in the admin context displays the total number of KAL-AP statistics for all contexts. These statistics are followed by the statistics for the admin context and then all other contexts.

The show stats kalap command includes two new fields:

Total requests dropped due to queue overflow—Number of requests that the ACE drops when the KAL-AP request queue is full. The ACE has a maximum KAL-AP request queue size of 1024 requests.

Total queries successfully received—Number of queries that the ACE received from the GSS. A request from the GSS may contain between 1 to 60 queries.

Class map

[line_number] match virtual-address address {[mask] | any | {tcp | udp {any | eq port_number | range port1 port2}} | protocol_number}

Previously, the ACE allowed you to configure a class-map VIP address that overlaps with an ACE interface IP address. The ACE no longer allows this configuration and displays the following warning:

Error: Entered VIP address is not the first address in 
the VIP range

Class map HTTP inspection

[line_number] match request-method {ext method

Added the following match statement HTTP inspection extension methods:

bcopy

bdelete

bmove

bpropfind

bproppatch

poll

notify

search

subscribe

unsubscribe

x-ms-emumatts

Configuration

crypto crl name url

You can now configure a CRL that the ACE downloads on the SSL proxy service for server authentication. It also supports LDAP for CRL downloads. For information on this command, see the "Configuring Downloaded CRLs for Server Authentication" section.

For client authentication, the url argument support LDAP URLs. For more information, see the "Configuring Downloaded CRLs through LDAP for Client and Server Authentication" section.

Configuration

domain name

The name argument is now a maximum of 76 characters. Formerly, it was a maximum of 64 characters.

Parameter map connection

set tcp buffer-share

Per CSCsw69707, you can now configure this command for UDP connections. Previously, buffer share was configurable only for TCP connections.

Policy map class

kal-ap-tag tag_name

Associates a KAL-AP tag with a VIP address in the class map. For information on this command, see the "Associating a KAL-AP Tag to a VIP Class Map" section.

Policy map HTTP inspection

match request-method ext method

Added the following inline match HTTP inspection extension methods:

bcopy

bdelete

bmove

bpropfind

bproppatch

poll

notify

search

subscribe

unsubscribe

x-ms-emumatts

Role

rule number {permit | deny} {create | modify | debug | monitor} [feature changeto-command | exec-commands]

Previously, you could not configure user-defined roles to use the changeto command. The new changeto-command option allows a user-defined role to use the changeto command. Also, users retain their privileges when accessing different contexts. By default, this command is disabled for user-defined roles.

Previously, the ACE enabled Exec mode commands for user-defined roles. The new exec-commands option allows a user-defined role to use the capture, clear, debug, delete, gunzip, mkdir, move, rmdir, set, setup, system, tac-pac, untar, write, and undebug commands. By default, these commands are now disabled for user-defined roles.

SSL parameter map configuration

authentication-failure ignore

Allows the SSL connection even if the authentication fails. Possible reasons for the authentication failure include:

Certificate has expired

Certificate is not yet valid

Certificate has been revoked

General failure of receiving the certificate

This command and the failure reasons apply to both server certificates and client certificates.

SSL parameter map configuration

expired-crl reject

This command now configures the ACE to reject a server certificate when the CRL in use has expired. For information on this command, see the "Rejecting Server Certificates Because of Expired CRL" section.

SSL proxy configuration

crl crl_name | best-effort

This command now allows you to determine which CRL information to use for server authentication. For more information, see the "Using CRLs for Server Authentication" section.


Available ACE Licenses

By default, the ACE supports virtualization with one Admin context and five user contexts, 4 gigabits per second (Gbps) module bandwidth, and 1,000 SSL transactions per second (TPS). You can increase the number of default user contexts, module bandwidth, and SSL TPS by purchasing the following licenses:

ACE-VIRT-020—20 virtual contexts

ACE-VIRT-050—50 virtual contexts

ACE-VIRT-100—100 virtual contexts

ACE-VIRT-250—250 virtual contexts

ACE-08G-LIC—8 Gbps bandwidth

If you purchase an ACE with a bandwidth of 4 Gbps, you can upgrade the module bandwidth to 8 Gbps by using the ACE-UPG1-LIC license.

ACE-16G-LIC—16 Gbps bandwidth (ACE20-MOD-K9 module only)

If you purchase an ACE with a bandwidth of 8 Gbps, you can upgrade the module bandwidth to 16 Gbps by using the ACE-UPG2-LIC license (ACE20-MOD-K9 module only).

ACE-SSL-5K-K9—SSL with 5,000 TPS

ACE-SSL-10K-K9—SSL with 10,000 TPS

ACE-SSL-15K-K9—SSL with 15,000 TPS

You can upgrade virtualization in increments, provided that you do not exceed the limits of the ACE (a maximum of 250 contexts), by using the following licenses:

ACE-VIRT-UP1—Upgrades 20 to 50 contexts

ACE-VIRT-UP2—Upgrades 50 to 100 contexts

ACE-VIRT-UP3—Upgrades 100 to 250 contexts

You can upgrade SSL in 5,000 TPS increments up to a maximum of 15,000 TPS by using the following SSL upgrade licenses:

ACE-SSL-UP1-K9—Upgrades SSL from 5,000 TPS to 10,000 TPS (3.0(0)A1(3) or later)

ACE-SSL-UP2-K9—Upgrades SSL from 10,000 TPS to 15,000 TPS (3.0(0)A1(3) or later)

You can also obtain an ACE demo license for each type of virtualization, bandwidth, or SSL TPS license, including upgrade increments for contexts. You can get a demo license that is valid between 30 and 90 days. At the end of this period, you will need to update the demo license with a permanent license to continue to use the ACE software. To view the expiration of the demo license, use the show license usage command in Exec mode. If you need to replace the ACE module, you can copy and install the licenses onto the replacement module.


Note You can access the license and show license commands only in the Admin context. You must have the Admin role in the Admin context to perform the tasks of installing, removing, and updating the license.


Ordering an Upgrade License and Generating a License Key

This section describes the process to order an upgrade license and to generate a license key for your ACE. To order an upgrade license, perform the following steps:


Step 1 Order one of the licenses from the list in the "Available ACE Licenses" section using any of the available Cisco ordering tools on Cisco.com.

Step 2 When you receive the Software License Claim Certificate from Cisco, follow the instructions that direct you to the cisco.com website. As a registered user of cisco.com, go to this URL:

http://www.cisco.com/go/license

Step 3 Enter the Product Authorization Key (PAK) number found on the license certificate as your proof of purchase.

Step 4 Provide all the requested information to generate a license key.

Step 5 After the system generates the license key, you will receive a license key e-mail with an attached license file and installation instructions. Save the license key e-mail in a safe place in case you need it in the future (for example, to transfer the license to another ACE).


For information about installing and managing ACE licenses, refer to Chapter 3, Managing ACE Software Licenses, in the Cisco Application Control Engine Module Administration Guide.

Upgrading Your ACE Software

For complete instructions on how to upgrade your ACE software, see the Cisco Application Control Engine Module Administration Guide.


Note To upgrade your ACE software to version A2(1.0) or higher, your ACE must be running software version 3.0(0)A1(5a) or higher.


An incompatibility exists between certain ACE software versions in the 3.0(0)A1.6.3x and A2.1x release trains. In a redundant configuration, the FT ACE pairs will not recognize each other and will report the following status as part of the show ft peer detail command output:

SRG Compatibility: INCOMPATIBLE

The following software version combinations that are indicated with an "x" are incompatible:

A1(6.3x) Release
A2(1.0)
A2(1.0a)
A2(1.1)
A2(1.1a)
A2(1.2)
A2(1.3)
A2(2.0)

3.0(0)A1(6.3b)

x

 

x

x

     

3.0(0)A1(6.3c)

x

x

x

x

     

Before you upgrade your ACE software, be sure that your ACE configurations meet the upgrade prerequisites in the following sections:

Changing the Admin Password

Changing the www User Password

Checking Your Configuration for FT Priority and Preempt

Creating a Checkpoint

Updating Your Application Protocol Inspection Configurations

Changing the Admin Password

Before you upgrade to software version A2(1.0) or higher, you must change the default Admin password, if you have not already done so. Otherwise, after you upgrade the ACE software, you will be able to log in to the ACE only through the console port or through the supervisor engine of the Catalyst 6500 series switch or the Cisco 7600 series router. For details about changing the Admin password, see the Cisco Application Control Engine Module Administration Guide.

Changing the www User Password

Before you upgrade to software version A2(1.0) or higher, you must change the default www user password if you have not already done so. Otherwise, after you upgrade the ACE software, the www user will be disabled and you will not be able to use Extensible Markup Language (XML) to remotely configure an ACE until you change the default www user password. For details about changing the www user password, see the Cisco Application Control Engine Module Administration Guide.

Checking Your Configuration for FT Priority and Preempt

If you want the currently active ACE to remain active after the software upgrade, be sure that the active ACE has a higher priority than the standby (peer) ACE and that the preempt command is configured. To check the redundant configuration of your ACEs, use the show running-config ft command. The preempt command is enabled by default and does not appear in the running-config file.

Creating a Checkpoint

We strongly recommend that you create a checkpoint in the running-configuration file of each context in your ACE. A checkpoint creates a snapshot of your configuration that you can later roll back to in case a problem occurs with an upgrade and you want to downgrade the software to a previous release. Use the checkpoint create command in Exec mode in each context for which you want to create a configuration checkpoint and name the checkpoint. For details about creating a checkpoint and rolling back a configuration, see Cisco Application Control Engine Module Administration Guide. For information about downgrading your ACE, see the "Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration" section.

Updating Your Application Protocol Inspection Configurations

Because the ACE version A2(1.0) or higher software has stricter error checks for application protocol inspection configurations than A1(x) software versions, be sure that your inspection configurations meet the guidelines that follow. The error checking process in A2(1.0) or higher software denies misconfigurations in inspection classifications (class maps) and displays error messages. If such misconfigurations exist in your startup- or running-configuration file before you load the A2(1.0) or higher software, the standby ACE in a redundant configuration may boot up to the STANDBY_COLD state. For information about redundancy states, see the Cisco Application Control Engine Module Administration Guide.

If the class map for the inspection traffic is generic (match . . . any or class-default is configured) so that noninspection traffic is also matched, the ACE displays an error message and does not accept the inspection configuration. For example:

switch/Admin(config)# class-map match-all TCP_ANY
switch/Admin(config-cmap)# match port tcp any
switch/Admin(config)# policy-map multi-match FTP_POLICY
switch/Admin(config-pmap)# class TCP_ANY 
switch/Admin(config-pmap-c)# inspect ftp
Error: This class doesn't have tcp protocol and a specific port

The following examples show some of the generic class-map match statements and an ACL that are not allowed in A2(1.0) or higher inspection configurations:

match port tcp any

match port udp any

match port tcp range 0 65535

match port udp range 0 65535

match virtual-address 192.168.12.15 255.255.255.0 any

match virtual-address 192.168.12.15 255.255.255.0 tcp any

access-list acl1 line 10 extended permit ip any any

For application protocol inspection, the class map must have a specific protocol (related to the inspection type) configured and a specific port or range of port numbers.

For HTTP, FTP, RTSP, Skinny, and ILS protocol inspection, the class map must have TCP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port tcp eq www

For SIP protocol inspection, the class map must have TCP or UDP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port tcp eq 124

or

host1/Admin(config-cmap)# match port udp eq 135

For DNS inspection, the class map must have UDP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port udp eq domain

For ICMP protocol inspection, the class map must have ICMP as the configured protocol. For example, enter the following commands:

host1/Admin(config)# access-list ACL1 extended permit icmp 192.168.12.15 255.255.255.0 
192.168.16.25 255.255.255.0 echo

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match access-list ACL1

Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration

If you need to downgrade your ACE software from version A2(1.0) or higher to an earlier version, use the procedure that follows. You can downgrade your ACE from software version A2(1.0) or higher to 3.0(0)A1(6.1) or higher. Downgrading your ACE software to a software version below 3.0(0)A1(6.1) is not supported and not recommended. We recommend that you downgrade to the highest 3.0(0)A1(6.x) software version that is available. This procedure assumes that your ACEs are configured as redundant peers to ensure that there is no disruption to existing connections during the downgrade process. In the following procedure, the active ACE is referred to as ACE-1 and the standby ACE is referred to as ACE-2.

This section contains the following topics:

Before You Begin

Downgrade Procedure

Before You Begin

Before you downgrade your ACE software, ensure that the following conditions exist:

Identical versions of 3.0(0)A1(6.x) software images reside in the image: directory of both ACEs.

The active ACE has a higher priority than the standby ACE and preempt is enabled on the FT group if you want the active ACE to remain active after the downgrade procedure.

Downgrade Procedure

To downgrade your A2(1.0) or higher software in a redundant configuration, perform the following steps:


Step 1 If you have created checkpoints in your 3.0(0)A1(6.x) running-configuration files (highly recommended), roll back the configuration in each context on each ACE to the check-pointed configuration. For example:

host1/Admin# checkpoint rollback CHECKPOINT_ADMIN
host1/Admin# changeto C1
host1/C1# checkpoint rollback CHECKPOINT_C1

Do the same on the other ACE. For information about creating checkpoints and rolling back configurations, see Chapter 4, Managing the ACE Software.

Step 2 Configure ACE-1 to automatically boot from the 3.0(0)A1(6.x) image. To set the boot variable and configuration register to 1, use the boot system image: and config-register commands in configuration mode. For example, enter the following command:

host1/Admin# config
host1/Admin(config)# boot system image:c6ace-t1k9-mzg.3.0.0_A1_6_3.bin
host1/Admin(config)# config-register 1
host1/Admin(config)# exit
host1/Admin# 

You can set up to two images through the boot system command. If the first image fails, the ACE tries to boot from the second image.


Note Use the no boot system image: command to remove the configured A2(1.x) or higher boot variable.


Step 3 Verify that the boot variable was synchronized to ACE-2 by entering the following command on ACE-2:

host1/Admin# show bootvar
BOOT variable = "disk0:c6ace-t1k9-mzg.3.0.0_A1_6_3.bin"
Configuration register is 0x1
host1/Admin#

Step 4 Use the show ft group detail command to verify the state of each module. Upgrade the ACE that has its Admin context in the STANDBY_HOT state (ACE-2) first by entering the reload command.When ACE-2 loads the startup-configuration file, you may observe a few errors if you did not roll back the configuration to a checkpoint. These errors are harmless and occur because the 3.0(0)A1(6.x) software does not recognize the A2(1.x) or higher commands in the startup-configuration file. After ACE-2 boots up, it may take a few minutes to reach the STANDBY_HOT state again. At this time, configuration synchronization is disabled, but the connections through ACE-1 are still being replicated to ACE-2.

host1/Admin# reload
This command will reboot the system
Save configurations for all the contexts. Save? [yes/no]: [yes]

Step 5 Perform a graceful failover of all contexts from ACE-1 to ACE-2 by entering the ft switchover all command in Exec mode on ACE-1. ACE-2 becomes the new active ACE and assumes mastership of all active connections with no interruption to existing connections.

host1/Admin# ft switchover all

Step 6 Reload ACE-1 with the same 3.0(0)A1(6.x) software version as ACE-2. Again, you may observe a few errors as ACE-1 loads the startup-configuration file.

host1/Admin# reload

After ACE-1 boots up, it assumes the role of standby and enters the STANDBY_HOT state (this may take several minutes). You can verify the states of both ACEs by entering the show ft group detail command in Exec mode. Because both ACE-1 and ACE-2 are running the same version of software now, configuration mode is enabled. The configuration is synchronized from ACE 2 (currently active) to ACE-1. If ACE-1 is configured with a higher priority and preempt is configured on the FT group, ACE-1 reasserts mastership after it has received all configuration and state information from ACE-2, making ACE-2 the new standby. ACE-1 becomes the active ACE once again.

Step 7 Perform manual cleanup in the running-configuration files of both ACEs to remove unnecessary version A2(1.0) or higher configuration elements. For example, you may need to remove a service policy from an interface that was part of the version A2(1.x) or higher configuration that is no longer needed in version 3.0(0)A1(6.x).

Step 8 Enter the write memory all command in both ACEs to save the running-configuration files in all configured contexts to their respective startup-configuration files. This action will eliminate future errors when the ACEs reload their startup-configuration files.


ACE Documentation Set

In addition to this document, the ACE documentation set includes the following publications:

Document Title
Description

Cisco Application Control Engine Module Hardware Installation Note

This guide provides information for installing the ACE into the Catalyst 6500 series switch and the Cisco 7600 series router.

Cisco Application Control Engine Module Getting Started Guide

This guide describes how to perform the initial setup and configuration tasks for the ACE.

Cisco Application Control Engine Module Administration Guide

This guide describes how to perform administration tasks on the ACE, including initial setup, establish remote access, configure class maps and policy maps, manage the ACE software, configure SNMP, define system message logging, configure redundancy, and upgrade your ACE software.

Cisco Application Control Engine Module Virtualization Configuration Guide

This guide provides instructions on how to operate your ACE in a single-context or in multiple-contexts. Multiple-contexts use the concept of virtualization to partition your ACE into multiple virtual devices or contexts.

Cisco Application Control Engine Module Routing and Bridging Configuration Guide

This guide provides instructions for configuring the routing and bridging features of the ACE. This guide provides a routing overview and describes how to perform ACE configuration tasks, including:

Configuring VLANs

Configuring routing

Configuring bridging

Configuring Address Resolution Protocol (ARP)

Configuring Dynamic Host Configuration Protocol (DHCP)

Cisco Application Control Engine Module Server Load-Balancing Configuration Guide

This guide describes how to perform ACE server load-balancing configuration tasks, including:

Server health monitoring

Real servers and server farms

Stickiness

Class maps and policy maps to load-balance traffic to real servers in server farms

Firewall load balancing

TCL scripts

Cisco Application Control Engine Module Security Configuration Guide

This guide describes how to perform ACE security configuration tasks, including:

Security access control lists (ACLs)

User authentication and accounting using a TACACS+, RADIUS, or LDAP server

Application protocol and HTTP deep packet inspection

TCP/IP normalization and termination parameters

Network address translation (NAT)

Cisco Application Control Engine Module SSL Configuration Guide

This guide describes how to perform ACE SSL configuration tasks, including:

SSL certificates and keys

SSL initiation

SSL termination

End-to-end SSL

Cisco Application Control Engine Module System Message Guide

Describes how to configure system message logging on the ACE. This guide lists and describes the system log messages generated by the ACE.

Cisco Application Control Engine Module Command Reference

This reference provides an alphabetical list of all command line interface (CLI) commands including syntax, options, and related commands.

Cisco CSM-to-ACE Conversion Tool User Guide

Describes how to use the CSM-to-ACE conversion tool to migrate Cisco Content Switching Module (CSM) running-configuration or startup-configuration files to the ACE.

Cisco CSS-to-ACE Conversion Tool User Guide

Describes how to use the CSS-to-ACE conversion tool to migrate Cisco Content Services Switches (CSS) running-configuration or startup-configuration files to the ACE.

Cisco Application Control Engine (ACE) Module Troubleshooting Guide, Release A2(x)

Describes the procedures and methodology in wiki format to troubleshoot the most common problems that you may encounter during the operation of your ACE.


Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS Version 2.0.