Guest

Cisco Services Modules

Release Note vA2(1.x), Cisco ACE Application Control Engine Module

  • Viewing Options

  • PDF (1.2 MB)
  • Feedback
Release Note for the Cisco Application Control Engine Module

Table Of Contents

Release Note for the Cisco Application Control Engine Module

Contents

Supervisor Engine and Cisco IOS Support for the ACE Module

Virtual Switching System Support

ACE Module Troubleshooting Wiki

New Software Feature in Version A2(1.5)

Using the "\xST" Metacharacter in Regular Expressions for Layer 4 Generic Data Parsing

Overview

"\xST" Metacharacter Regex Usage Considerations

Configuration Examples

New Software Features in Version A2(1.1)

Configuring the Reverse IP Stickiness Feature

Overview of Reverse IP Stickiness

Configuration Requirements and Restrictions

Configuring Reverse IP Stickiness

Displaying Reverse IP Sticky Status and Statistics

Reverse IP Stickiness Configuration Examples

Configuring the Switch Mode Feature

New Software Features in Version A2(1.0)

Available ACE Licenses

Ordering an Upgrade License and Generating a License Key

Upgrading Your ACE Software

Changing the Admin Password

Changing the www User Password

Checking Your Configuration for FT Priority and Preempt

Creating a Checkpoint

Updating Your Application Protocol Inspection Configurations

Downgrading Your ACE Software from Version A2(1.0) to 3.0(0)A1(6.x) in a Redundant Configuration

Before You Begin

Downgrade Procedure

ACE Operating Considerations

ACE Documentation Set

Software Version A2(1.6a) Resolved Caveats and Open Caveats

Software Version A2(1.6a) Resolved Caveats

Software Version A2(1.6a) Open Caveats

Software Version A2(1.6) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.6) Resolved Caveats

Software Version A2(1.6) Open Caveats

Software Version A2(1.6) Command Changes

Software Version A2(1.5a) Resolved Caveats and Open Caveats

Software Version A2(1.5a) Resolved Caveats

Software Version A2(1.5a) Open Caveats

Software Version A2(1.5) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.5) Resolved Caveats

Software Version A2(1.5) Open Caveats

Software Version A2(1.5) Command Changes

Revised System Log Messages

253004

441001

441002

Software Version A2(1.4a) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.4a) Resolved Caveats

Software Version A2(1.4a) Open Caveats

Software Version A2(1.4a) Command Changes

Software Version A2(1.4) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.4) Resolved Caveats

Software Version A2(1.4) Open Caveats

Software Version A2(1.4) Command Changes

Displaying Detailed CRL-Downloading Statistics

System Log Messages

New syslog Message

Revised syslog Message

Software Version A2(1.3) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.3) Resolved Caveats

Software Version A2(1.3) Open Caveats

Software Version A2(1.3) Command Changes

Software Version A2(1.2) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.2) Resolved Caveats

Software Version A2(1.2) Open Caveats

Software Version A2(1.2) Command Changes

Software Version A2(1.1a) Resolved Caveats and Open Caveats

Software Version A2(1.1a) Resolved Caveats

Software Version A2(1.1a) Open Caveats

Software Version A2(1.1) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.1) Resolved Caveats

Software Version A2(1.1) Open Caveats

Software Version A2(1.1) Command Changes

Software Version A2(1.0a) Resolved and Open Caveats

Software Version A2(1.0a) Resolved Caveats

Software Version A2(1.0a) Open Caveats

Software Version A2(1.0) Resolved Caveats and Open Caveats

Software Version A2(1.0) Resolved Caveats

Software Version A2(1.0) Open Caveats

Obtaining Documentation and Submitting a Service Request


Release Note for the Cisco Application Control Engine Module


Release: October 20, 2009
Updated: March 25, 2010


Note The most current Cisco documentation for released products is available on Cisco.com.


Contents

This release note applies to the following software versions for the Cisco Application Control Engine (ACE) Module, models ACE10 (ACE10-6500-K9) and ACE20 (ACE20-MOD-K9):

A2(1.6a)

A2(1.6)

A2(1.5a)

A2(1.5)

A2(1.4a)

A2(1.4)

A2(1.3)

A2(1.2)

A2(1.1a)

A2(1.1)

A2(1.0a)

A2(1.0)

For information on the ACE module features and configuration details, see the ACE documentation located at:

http://www.cisco.com/en/US/products/ps6906/tsd_products_support_model_home.html

This release note contains the following sections:

Supervisor Engine and Cisco IOS Support for the ACE Module

Virtual Switching System Support

ACE Module Troubleshooting Wiki

New Software Feature in Version A2(1.5)

New Software Features in Version A2(1.1)

New Software Features in Version A2(1.0)

Available ACE Licenses

Ordering an Upgrade License and Generating a License Key

Upgrading Your ACE Software

Downgrading Your ACE Software from Version A2(1.0) to 3.0(0)A1(6.x) in a Redundant Configuration

ACE Operating Considerations

ACE Documentation Set

Software Version A2(1.6a) Resolved Caveats and Open Caveats

Software Version A2(1.6) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.5a) Resolved Caveats and Open Caveats

Software Version A2(1.5) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.4a) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.4) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.3) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.2) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.1a) Resolved Caveats and Open Caveats

Software Version A2(1.1) Resolved Caveats, Open Caveats, and Command Changes

Software Version A2(1.0) Resolved Caveats and Open Caveats

Obtaining Documentation and Submitting a Service Request

Supervisor Engine and Cisco IOS Support for the ACE Module

Table 1 and Table 2 summarize the supervisor engine model and Cisco IOS version support for the ACE module in the Catalyst 6500 series switch and the Cisco 7600 series router, respectively.

Table 1 Supervisor Engine and IOS Support for the ACE Module in a Catalyst 6500 Series Switch with a Multilayer Switch Feature Card (MSFC3)

Supervisor Engine Model
Minimum Required IOS Version
Other IOS Version Support

WS-SUP720

12.2(18)SXF4 (or later)

12.2(33)SXH (or later), 12.2(33)SXI1 (or later)

WS-SUP720-3B

WS-SUP720-3BXL

VS-S720-10G-3C

12.2(33)SXH (or later)

VS-S720-10G-3CXL

1 Minimum required IOS version for VSS support. See the Virtual Switching System Support section.


Table 2 Supervisor Engine, Route Switch Processor (RSP), and Cisco IOS Support for the ACE Module in a Cisco 7600 Series Router with an MSFC3

Supervisor Engine or RSP
Minimum Required IOS Version
Other IOS Version Support

WS-SUP720

12.2(18)SXF4 (or later)

12.2(33) SRB (or later)

Not supported: 12.2(33)SXH1

WS-SUP720-3B

WS-SUP720-3BXL

RSP720

12.2(33)SRC (or later)

None

1 Cisco IOS release 12.2(33)SXH runs only on the Catalyst 6500 series switch. Therefore, the Supervisor 720-10GE engines are not supported in the Cisco 7600 series router.


For more information about Cisco IOS releases, see the Release Notes for Cisco IOS Release 12.2SXF and Rebuilds and the Release Notes for Cisco IOS Release 12.2(33)SXH and Later Releases.

Virtual Switching System Support

The ACE10 and the ACE20 running ACE software version A2(1.2) or later and installed in a Catalyst 6500 series switch running IOS software version 12.2(33)SXI or later support the Virtual Switching System (VSS). VSS is a system virtualization technology that allows the pooling of multiple Catalyst 6500 switches into a single virtual switch for increased operational efficiency by simplifying the network. Inter-chassis Supervisor switchover (SSO) boosts non-stop communication. For more information about VSS, see the Cisco IOS Version 12.2(33)SXI Configuration Guide.

ACE Module Troubleshooting Wiki

The ACE documentation set now includes the ACE Module Troubleshooting Wiki. This wiki is a collaborative site that describes the basic procedures and methodology to assist you in troubleshooting the most common problems that you may encounter while you are operating your ACE.

As a registered user of Cisco.com, we strongly encourage you to add content to this site in the form of troubleshooting tips, procedures, or even entire sections. When you add content to the site, you should adhere to the format that has been established for the wiki. To access the ACE Module Troubleshooting Wiki on Cisco DocWiki, click the following URL:

http://docwiki.cisco.com/wiki/Cisco_Application_Control_Engine_(ACE)_Module_Troubleshooting_ Guide,_Release_A2(x)

New Software Feature in Version A2(1.5)

The A2(1.5) software maintenance release provides the following new feature.

Using the "\xST" Metacharacter in Regular Expressions for Layer 4 Generic Data Parsing

This section describes the use of the new "\xST" metacharacter added in software version A2(1.5) for regular expressions that are used as part of Layer 4 generic data parsing.

It includes the following topics:

Overview

"\xST" Metacharacter Regex Usage Considerations

Configuration Examples

Overview

The "\xST" (STop) metacharacter is now available in software version A2(1.5) for all regular expressions (regexes) that are supported by the ACE. This new metacharacter has been provided for specific use cases that utilize the maximum parse length to terminate parsing. However, the "\xST" metacharacter is specifically designed for use by applications that involve the generic data parsing of a Layer 4 payload.

If you intend to use the "\xST" metacharacter for regex matches on packets from protocols, we recommend that you use this metacharacter only for the following protocols in the generic data parsing of a Layer 4 payload:

SSL session-ID stickiness—To perform sticky hashing on the initial packets in an SSL handshake, allowing the ACE to stick the same client to the same SSL server based on the SSL session ID.

Financial Information eXchange (FIX) type `A' Logon message—To define load-balancing criteria while setting up the outbound path of a connection.

In earlier releases of the ACE software, without the ability to include the "\xST" metacharacter in regexes, there are certain SSL session-id and FIX packets that may get stuck in the ACE HTTP engine and eventually time out the connection. The inclusion of the "\xST" metacharacter will now aid the ACE in properly load-balancing SSL session-id and FIX packets.

The "\xST" metacharacter has been added to software version A2(1.5) per CSCsh04655.

"\xST" Metacharacter Regex Usage Considerations

The new "\xST" metacharacter has the following usage guidelines related to its inclusion in regex matching:

If the input matches a regex pattern that includes the "\xST" metacharacter, the regex engine will halt upon finding the character directly next to the '\xST' in the regex string (2nd '\x01' in the match statement).

No additional input data will be considered by the ACE once the matching pattern is seen which may affect other regexes that are configured elsewhere in the policy. In this case, the "\xST" metacharacter should be used only once in the policy.

The "\xST" metacharacter should only be used at the end of a regex pattern and not at the beginning. In this case, the ACE will display the "Error: Invalid regular expression" error message.

The "\xST" metacharacter should not be added directly after a * wildcard match. For example, "abc.*\xST" would not be a recommended regex.

Configuration Examples

The following configuration examples show the use of the "\xST" metacharacter in two very specific regexes:

SSL session-ID Stickiness Configuration Example

parameter-map type generic SESSID-PARAM

set max-parse-length 76

sticky layer4-payload SESSID-STICKY

serverfarm SF1

response sticky

layer4-payload offset 43 length 32 begin-pattern "(\x20|\x00\xST)"

FIX Protocol Configuration Example

sticky layer4-payload FIX-STICKY
  serverfarm FIX-SF1
  layer4-payload begin-pattern "\x0149=" end-pattern "\x01"

class-map type generic match-all FIX-CM
  2 match layer4-payload regex ".*\x0110=...\x01\xST"

New Software Features in Version A2(1.1)

The A2(1.1) software maintenance release provides the following two new features:

Configuring the Reverse IP Stickiness Feature

Configuring the Switch Mode Feature

Configuring the Reverse IP Stickiness Feature

This section describes the reverse IP stickiness feature that is used primarily in firewall load balancing (FWLB) to ensure that applications with separate control and data channels use the same firewall for ingress and egress flows for a given connection. It contains the following subsections:

Overview of Reverse IP Stickiness

Configuration Requirements and Restrictions

Configuring Reverse IP Stickiness

Displaying Reverse IP Sticky Status and Statistics

Reverse IP Stickiness Configuration Examples

Overview of Reverse IP Stickiness

Reverse IP stickiness is an enhancement to regular stickiness and is used mainly in FWLB. It ensures that multiple distinct connections that are opened by hosts at both ends (client and server) are load-balanced and stuck to the same firewall. Reverse stickiness applies to such protocols as FTP, RTSP, SIP, and so on where there are separate control channels and data channels opened by the client and the server, respectively.

You configure reverse IP stickiness as an action under a Layer 7 load-balancing policy map by associating an existing IP address sticky group with the policy using the reverse-sticky command. Then you associate the Layer 7 policy map with a Layer 4 multi-match policy map and apply the Layer 4 policy map as a service policy on the ACE interface between the firewalls and the ACE. When incoming traffic matches the policy, the ACE verifies that a reverse IP sticky group is associated with the policy. If the association exists, the ACE creates a sticky entry in the sticky table that maps the opposite IP address (for example, the destination IP address if source IP sticky is configured) to the real server ID, which is the ID of the firewall. To obtain the real ID of the firewall, the ACE uses the encapsulation (encap) ID from the traffic coming from the firewall as a lookup key into the list of real servers in the server farm.


Note The ACE sticky table, which holds a maximum of 4 million entries, is shared across all sticky types, including reverse IP stickiness.


This section contains the following topics:

Symmetric Topology

Asymmetric Topology

Symmetric Topology

A typical firewall load-balancing topology (symmetric) includes two dedicated ACEs with the firewalls positioned between the ACEs. In this scenario, the ACEs are used exclusively for FWLB and simply forward traffic through their host interfaces in either direction. See Figure 1.

The hosts in either VLAN 31 or VLAN 21 can initiate the first connection and the hosts on both sides of the connection can "see" each other directly. Therefore, only catch-all VIPs (with an IP address of 0.0.0.0 and a netmask of 0.0.0.0) are configured on the ACE interfaces.

Figure 1 Typical Symmetric Firewall Load-Balancing Topology for Reverse IP Stickiness

For the network diagram shown in Figure 1, the following steps describe a possible connection scenario with reverse IP stickiness:


Step 1 Host A (a client) initiates an FTP control channel connection to the IP address of Host C (an FTP server).

Step 2 ACE 1 load balances the connection to one of the two firewalls (FW1 or FW2) in the FWS-OUT server farm. ACE 1 is configured with a source IP sticky group that is associated with a policy map, which is applied to interface VLAN 113. This configuration ensures that all connections coming from the same host (or directed to the same host) are load balanced to the same firewall. The ACE creates a sticky entry that maps the IP address of Host A to one of the firewalls.

Step 3 The firewall that receives the packets from ACE 1 forwards them to ACE 2.

Step 4 Assume that a sticky group that is based on the destination IP address is associated with a policy map and is applied to interface VLAN 21. The same sticky group is associated as a reverse sticky group with the policy that is applied to VLAN 111. When it receives the packets, ACE 2 creates a sticky entry in the sticky database based on the source IP address (because the sticky group is based on the destination IP address in this case), which maps the Host A IP address to the firewall in the FWS-IN server farm from which the traffic was received. Then, ACE 2 forwards the packets to the FTP server (Host C) in the server farm.

Step 5 If you have enabled the mac-sticky command on the VLAN 111 interface, ACE 2 forwards return traffic from the same connection to the same firewall from which the incoming traffic was received. The firewall routes the return traffic through ACE 1, which in turn forwards it to the MSFC and from there to the client.

Step 6 Now suppose that Host C (an FTP server) opens a new connection (for example, the corresponding FTP data channel of the previously opened FTP control channel) to the IP address of Host A. Because a sticky group based on destination IP is associated with the policy applied to interface VLAN 21, ACE 2 performs a sticky lookup and finds a valid sticky entry (the one created in Step 4) in the sticky database that allows ACE 2 to load balance the packets to the same firewall that the control connection traversed.

Step 7 The firewall routes the packets through ACE 1, which in turn forwards them to the MSFC and from there to the client (Host A).


Follow these guidelines and observations when you configure reverse IP stickiness:

When reverse IP sticky is enabled, the sticky entry is populated in one direction (for incoming traffic) and looked up in the opposite direction (for outgoing traffic), allowing traffic to flow through the same firewall in both directions.

The example that is described in the steps above is symmetric because it does not matter on which side of the connections that the clients and servers reside. Everything would work in a similar manner if Host C was a client opening the FTP control channel and Host A was a server opening the FTP data channel, assuming that a reverse sticky group was also configured on the ACE 1 VLAN 112 interface. To make reverse IP stickiness work symmetrically, you must apply a reverse sticky group to the ACE interfaces that are associated with the firewall server farm (in this example, VLAN 112 and VLAN 111) and apply the same sticky group as a regular sticky group to the ACE interfaces associated with the hosts (in this example, VLAN 113 and VLAN 21).

In this example, the assumption is to have a regular sticky group based on the source IP associated with the VLAN 113 interface of the ACE 1 module and another sticky group based on the destination IP associated with the VLAN 21 interface of the ACE 2 module (the reverse sticky groups on VLAN 112 and VLAN 111 would be based on the opposite IPs). Everything would work correctly if the regular sticky groups were reversed, that is, the sticky group on VLAN 113 was based on the destination IP and the one on VLAN 21 was based on the source IP, or if both regular sticky groups were based on both the source and the destination IP.

Asymmetric Topology

The following scenario is asymmetric because it cannot work equally in both directions as in the previous scenario. In this setup, one of the load balancers is unknown (Unknown LB) so that it is uncertain whether the load balancer supports reverse sticky. The clients must be on one side of the connection and the servers must be on the other side with the clients opening the first connection to the servers. See Figure 2. In this scenario, the ACE performs only FWLB and forwards traffic to the real servers in the server farm.

Figure 2 Asymmetric Firewall Load Balancing Topology for Reverse IP Stickiness

For the network diagram shown in Figure 2, the following steps describe the sequence of events for establishing a connection with reverse IP stickiness:


Step 1 A client initiates a connection (for example, an FTP control channel connection) to the IP address of one of the servers in the server farm.

Step 2 The Unknown LB load balances the connection to one of the two firewalls in the FWS-OUT server farm. The Unknown LB should, at a minimum, support load balancing based on the source or destination IP address hash predictor. These predictors ensure that all connections coming from the same client (or destined to the same server) are load balanced to the same firewall. Assume in this example that a predictor based on source IP hash is configured in the Unknown LB, so that all traffic coming from the same client will be directed to the same firewall.

Step 3 The firewall that receives the packet forwards it to the ACE.

Step 4 Assume that a sticky group that is based on the destination IP address is associated with a policy map that is applied to interface VLAN 21 using a service policy. The same sticky group is associated as a reverse sticky group with the policy that is applied to VLAN 111. When it receives the packets, the ACE creates a sticky entry in the sticky database based on the source IP address (because the sticky group is based on the destination IP in this case), which maps the Host A IP address to the firewall in the FWS-IN server farm from which the traffic was received. Then, the ACE forwards the packets to the FTP server (Host C) in the server farm.

Step 5 If you have enabled the mac-sticky command on VLAN 111, the ACE forwards the return traffic for the same connection to the same firewall from which the incoming traffic was received. The firewall routes the return traffic through the Unknown-LB, which in turn forwards it to the MSFC and then to the client.

Step 6 Now suppose that the FTP server opens a new connection (for example, the corresponding FTP data channel of the previously opened FTP control channel) to the IP address of the client. Because a sticky group based on the destination IP address is associated with the policy applied to interface VLAN 21, the ACE performs a sticky lookup and finds a valid sticky entry (the one created in Step 4) in the sticky database that allows the ACE to load balance the packets to the same firewall that the control connection traversed.

Step 7 The firewall routes the packet through the Unknown LB, which in turn forwards it to the MSFC and then to the client.


In this scenario, reverse sticky would also work properly under the following conditions:

The sticky group is associated with the policy map as a regular sticky group based on source the IP and applied to the VLAN 21 interface.

The sticky group is associated with the policy map as a reverse sticky group (based on the destination IP address) and applied to the VLAN 111 interface.

The Unknown LB has a predictor based on the hash of the destination IP.

For more information about configuring firewall load balancing, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

Configuration Requirements and Restrictions

Before attempting to configure reverse IP stickiness, be sure that you have met the following configuration requirements and restrictions:

A sticky group of type IP netmask based on source IP, destination IP, or both must be present in your configuration.

The sticky group cannot be a static sticky group.

Once you have associated reverse IP stickiness with a sticky group, you cannot change that sticky group to a static sticky group.

For firewall load balancing, configure the mac-sticky command on the ACE interface that is connected to the firewall.

Configuring Reverse IP Stickiness

To configure reverse IP stickiness, use the reverse-sticky command in policy map loadbalance class configuration mode. The syntax of this command is as follows:

reverse-sticky name

The name argument specifies the unique identifier of an existing IP address sticky group. Enter the name of an existing IP address sticky group as an unquoted text string with no spaces and a maximum of 64 alphanumeric characters.

For example, to configure reverse IP stickiness for a sticky group called DEST_IP_STICKY, enter the following sequence of commands:

host1/Admin(config)# sticky ip-netmask 255.255.255.255 address destination DEST_IP_STICKY
host1/Admin(config-sticky-ip)# serverfarm FWS-IN

host1/Admin(config)# policy-map type loadbalance first-match L7PMAP_TO_REALS
host1/Admin(config-pmap-lb)# class class-default
host1/Admin(config-pmap-lb-c)# forward
host1/Admin(config-pmap-lb-c)# reverse-sticky DEST_IP_STICKY

Displaying Reverse IP Sticky Status and Statistics

Use the following show commands to display the state of the reverse-sticky command and reverse sticky statistics:

show sticky database detail—Provides the reverse entry field that indicates the state (TRUE or FALSE) of reverse IP stickiness for each configured sticky group.

show stats sticky—Provides the Total active reverse sticky entries field that displays the total number of active reverse IP sticky entries in the sticky database.

show service-policy route detail—Provides the reverse sticky group field that displays the name of the sticky group configured for reverse IP stickiness.

Reverse IP Stickiness Configuration Examples

This section contains configuration examples that show how to configure reverse IP stickiness with a symmetric firewall load balancing configuration. These configuration examples correspond with the network diagram in Figure 1. The examples are as follows:

ACE 1 Configuration

ACE 2 Configuration

ACE 1 Configuration

access-list acl1 line 8 extended permit ip any any

rserver host FW1
  ip address 10.10.40.10
  inservice
rserver host FW2
  ip address 10.10.40.20
  inservice

serverfarm host FWS-OUT
  transparent
  rserver FW1
    inservice
  rserver FW2
    inservice

sticky ip-netmask 255.255.255.255 address source SOURCE_IP_STICKY
  serverfarm FWS-OUT

class-map match-all CATCH-ALL-VIP
  2 match virtual-address 0.0.0.0 0.0.0.0 any

policy-map type management first-match MGMT-POLICY
  class class-default
    permit

policy-map type loadbalance first-match LB_PMAP_TO_REALS
  class class-default
    sticky-serverfarm SOURCE_IP_STICKY
policy-map type loadbalance first-match ROUTE_PMAP
  class class-default
    forward
    reverse-sticky SOURCE_IP_STICKY

policy-map multi-match LB
  class CATCH-ALL-VIP
    loadbalance vip inservice
    loadbalance policy LB_PMAP_TO_REALS
policy-map multi-match ROUTE
  class CATCH-ALL-VIP
    loadbalance vip inservice
    loadbalance policy ROUTE_PMAP

service-policy input mgmt-policy

interface vlan 112
  description outside FW vlan
  bridge-group 15
  mac-sticky enable
  access-group input acl1
  service-policy input ROUTE
  no shutdown
interface vlan 113
  description client vlan
  bridge-group 15
  access-group input acl1
  service-policy input LB
  no shutdown

interface bvi 15
  ip address 10.10.40.2 255.255.255.0
  alias 10.10.40.3 255.255.255.0
  no shutdown

ip route 0.0.0.0 0.0.0.0 10.10.40.1

ACE 2 Configuration

access-list acl1 line 8 extended permit ip any any 

rserver host FW1
  ip address 10.10.50.10
  inservice
rserver host FW2
  ip address 10.10.50.20
  inservice

serverfarm host FWS-IN
  transparent
  rserver FW1
    inservice
  rserver FW2
    inservice

sticky ip-netmask 255.255.255.255 address destination DEST_IP_STICKY
  serverfarm FWS-IN

class-map match-all CATCH_ALL_VIP
  2 match virtual-address 0.0.0.0 0.0.0.0 any

policy-map type management first-match mgmt-policy
  class class-default
    permit

policy-map type loadbalance first-match L7PMAP_TO_FWS
  class class-default
    sticky-serverfarm DEST_IP_STICKY
policy-map type loadbalance first-match L7PMAP_TO_REALS
  class class-default
    forward
    reverse-sticky DEST_IP_STICKY

policy-map multi-match L4_TO_FWS
  class CATCH_ALL_VIP
    loadbalance vip inservice
    loadbalance policy L7PMAP_TO_FWS
policy-map multi-match L4_TO_REALS
  class CATCH_ALL_VIP
    loadbalance vip inservice
    loadbalance policy L7PMAP_TO_REALS
   
service-policy input mgmt-policy

interface vlan 21
  ip address 21.1.1.1 255.255.255.0
  access-group input acl1
  service-policy input L4_TO_FWS
  no shutdown
interface vlan 111
  description inside FW vlan
  ip address 10.10.50.1 255.255.255.0
  mac-sticky enable
  access-group input acl1
  service-policy input L4_TO_REALS
  no shutdown

Configuring the Switch Mode Feature

Use the switch mode feature to change the way that the ACE handles TCP connections that are not destined to a particular VIP and those connections that do not have any policies associated with their traffic. When you enable this feature, the ACE still creates connection objects for those TCP sessions that are not destined to the VIP. The ACE processes these connections as stateless connections, which means that they do not undergo any TCP normalization checks (for example, TCP window, TCP state, TCP sequence number, and other normalization checks).

The ACE also creates stateless connections for non-SYN TCP packets if they satisfy all other configured requirements, for example, ACLs and other policies. This process ensures that a long-lived persistent connection passes through the ACE successfully (even if it times out) by being reestablished by any incoming packet related to the connection.

By default, these stateless connections time out after 2 hours and 15 minutes unless you configure the timeout otherwise. When a stateless connection times out, the ACE does not send a TCP RST packet but instead closes the connection silently. Even though these connections are stateless, the TCP RST and FIN-ACK flags are honored and the connections are closed when the ACE sees these flags in the received packets.

To change the default timeout for these stateless connections, use the set timeout inactivity command in parameter map connection configuration mode. For details about this command, see theCisco Application Control Engine Module Security Configuration Guide.

The SYN cookie feature still operates normally for these stateless connections that are not destined to any VIP.

The default timeout value of 2 hours and 15 minutes is also applicable to the UDP connections that are not destined to any VIP.

To enable the switch mode feature, use the switch-mode command in configuration mode. The syntax of this command is as follows:

switch-mode

For example, to enable the switch mode feature, enter the following command:

host1/Admin(config)# switch-mode

To disable the switch mode feature, enter the following command:

host1/Admin(config)# no switch-mode

New Software Features in Version A2(1.0)

The A2(1.0) software release provides the following expanded features and functions:

Enhanced load-balancing support:

SIP

Extended RTSP

RADIUS

RDP

Generic protocol parsing

Enhanced predictors:

Adaptive algorithms

Least loaded

Least bandwidth

General SLB enhancements:

KAL-AP

HTTP header rewrite

Partial server farm failover

Application-based probes

SNMP-based probes

UDP fast age

SSL enhancements:

Hardware accelerated

Hardware-assisted probes

Session ID stickiness

Session ID reuse

SSL queue delay

Client authentication

URL rewrites for SSL

Fast DNS load balancing—UDP booster

XML-tagged configuration

ANM 1.2 support

Real-time TCP dump

Management traffic protection

Redundancy (high availability) sync improvements

Source NAT changes

Source NAT using a VIP

Server-farm based NAT

Protocol inspection enhancements:

SIP

ILS/LDAP

Skinny

ACL improvements—object grouping

Denial-of-service protection—SYN cookie per interface

Rate-limiting enhancements:

Connection-rate

Bandwidth-rate

HTTP firewall features:

Inspect HTTP POST body

Inspect HTTP secondary cookies

Available ACE Licenses

By default, the ACE supports virtualization with one Admin context and five user contexts, 4 gigabits per second (Gbps) module bandwidth, and 1,000 SSL transactions per second (TPS). You can increase the number of default user contexts, module bandwidth, and SSL TPS by purchasing the following licenses:

ACE-VIRT-020—20 virtual contexts.

ACE-VIRT-050—50 virtual contexts.

ACE-VIRT-100—100 virtual contexts.

ACE-VIRT-250—250 virtual contexts.

ACE-08G-LIC—8 Gbps bandwidth.

If you purchase an ACE with a bandwidth of 4 Gbps, you can upgrade the module bandwidth to 8 Gbps by using the ACE-UPG1-LIC license.

ACE-16G-LIC—16 Gbps bandwidth (ACE20-MOD-K9 module only)

If you purchase an ACE with a bandwidth of 8 Gbps, you can upgrade the module bandwidth to 16 Gbps by using the ACE-UPG2-LIC license (ACE20-MOD-K9 module only).

ACE-SSL-5K-K9—SSL with 5,000 TPS.

ACE-SSL-10K-K9—SSL with 10,000 TPS.

ACE-SSL-15K-K9—SSL with 15,000 TPS.

You can upgrade virtualization in increments, provided that you do not exceed the limits of the ACE (a maximum of 250 contexts), by using the following licenses:

ACE-VIRT-UP1—Upgrades 20 to 50 contexts

ACE-VIRT-UP2—Upgrades 50 to 100 contexts

ACE-VIRT-UP3—Upgrades 100 to 250 contexts

You can upgrade SSL in 5,000 TPS increments up to a maximum of 15,000 TPS by using the following SSL upgrade licenses:

ACE-SSL-UP1-K9—Upgrades SSL from 5,000 TPS to 10,000 TPS (3.0(0)A1(3) or later).

ACE-SSL-UP2-K9—Upgrades SSL from 10,000 TPS to 15,000 TPS (3.0(0)A1(3) or later).

You can also obtain an ACE demo license for each type of virtualization, bandwidth, or SSL TPS license, including upgrade increments for contexts. A demo license is valid for only 60 days. At the end of this period, you will need to update the demo license with a permanent license to continue to use the ACE software. To view the expiration of the demo license, use the show license usage command in Exec mode. If you need to replace the ACE module, you can copy and install the licenses onto the replacement module.


Note You can access the license and show license commands only in the Admin context. You must have the Admin role in the Admin context to perform the tasks of installing, removing, and updating the license.


Ordering an Upgrade License and Generating a License Key

This section describes the process to order an upgrade license and to generate a license key for your ACE. To order an upgrade license, perform the following steps:


Step 1 Order one of the licenses from the list in the "New Software Features in Version A2(1.0)" section using any of the available Cisco ordering tools on Cisco.com.

Step 2 When you receive the Software License Claim Certificate from Cisco, follow the instructions that direct you to the cisco.com website. As a registered user of cisco.com, go to this URL:

http://www.cisco.com/go/license

Step 3 Enter the Product Authorization Key (PAK) number found on the license certificate as your proof of purchase.

Step 4 Provide all the requested information to generate a license key.

Step 5 After the system generates the license key, you will receive a license key e-mail with an attached license file and installation instructions. Save the license key e-mail in a safe place in case you need it in the future (for example, to transfer the license to another ACE).


For information about installing and managing ACE licenses, refer to Chapter 3, Managing ACE Software Licenses, in the Cisco Application Control Engine Module Administration Guide.

Upgrading Your ACE Software

For complete instructions on how to upgrade your ACE software, see the Cisco Application Control Engine Module Administration Guide.


Note To upgrade your ACE software to version A2(1.x), your ACE must be running software version 3.0(0)A1(5a) or higher.


An incompatibility exists between certain ACE software versions in the 3.0(0)A1.6.3x and A2.1x release trains. In a redundant configuration, the FT ACE pairs will not recognize each other and will report the following status as part of the show ft peer detail command output:

SRG Compatibility: INCOMPATIBLE

The following software version combinations that are indicated with an "x" are incompatible:

A1(6.3x) Release
A2(1.0)
A2(1.0a)
A2(1.1)
A2(1.1a)
A2(1.2)
A2(1.3)
A2(1.4)

3.0(0)A1(6.3b)

x

 

x

x

     

3.0(0)A1(6.3c)

x

x

x

x

     


Note If you plan to configure IP-address stickiness in your network, we strongly recommend that you upgrade your ACE software to version A2(1.5a). For details, see resolved caveat CSCsz77633 in the Software Version A2(1.5a) Resolved Caveats section.


Before you upgrade your ACE software, be sure that your ACE configurations meet the upgrade prerequisites in the following sections:

Changing the Admin Password

Changing the www User Password

Checking Your Configuration for FT Priority and Preempt

Creating a Checkpoint

Updating Your Application Protocol Inspection Configurations

Changing the Admin Password

Before you upgrade to software version A2(1.1) or higher, you must change the default Admin password, if you have not already done so. Otherwise, after you upgrade the ACE software, you will be able to log in to the ACE only through the console port or through the supervisor engine of the Catalyst 6500 series switch or the Cisco 7600 series router. For details about changing the Admin password, see the Cisco Application Control Engine Module Administration Guide.

Changing the www User Password

Before you upgrade to software version A2(1.1) or higher, you must change the default www user password if you have not already done so. Otherwise, after you upgrade the ACE software, the www user will be disabled and you will not be able to use Extensible Markup Language (XML) to remotely configure an ACE until you change the default www user password. For details about changing the www user password, see the Cisco Application Control Engine Module Administration Guide.

Checking Your Configuration for FT Priority and Preempt

If you want the currently active ACE to remain active after the software upgrade, be sure that the active ACE has a higher priority than the standby (peer) ACE and that the preempt command is configured. To check the redundant configuration of your ACEs, use the show running-config ft command. Note that the preempt command is enabled by default and does not appear in the running-config.

Creating a Checkpoint

We strongly recommend that you create a checkpoint in the running-configuration file of each context in your ACE. A checkpoint creates a snapshot of your configuration that you can later roll back to in case a problem occurs with an upgrade and you want to downgrade the software to a previous release. Use the checkpoint create command in Exec mode in each context for which you want to create a configuration checkpoint and name the checkpoint. For details about creating a checkpoint and rolling back a configuration, see Cisco Application Control Engine Module Administration Guide. For information about downgrading your ACE, see the "Downgrading Your ACE Software from Version A2(1.0) to 3.0(0)A1(6.x) in a Redundant Configuration" section.

Updating Your Application Protocol Inspection Configurations

Because the ACE version A2(1.x) software has stricter error checks for application protocol inspection configurations than A1(x) software versions, be sure that your inspection configurations meet the guidelines that follow. The error checking process in A2(1.x) software denies misconfigurations in inspection classifications (class maps) and displays error messages. If such misconfigurations exist in your startup- or running-configuration file before you load the A2(1.x) software, the standby ACE in a redundant configuration may boot up to the STANDBY_COLD state. For information about redundancy states, see the Cisco Application Control Engine Module Administration Guide.

If the class map for the inspection traffic is generic (match . . . any or class-default is configured) so that noninspection traffic is also matched, the ACE displays an error message and does not accept the inspection configuration. For example:

switch/Admin(config)# class-map match-all TCP_ANY
switch/Admin(config-cmap)# match port tcp any
 
switch/Admin(config)# policy-map multi-match FTP_POLICY
switch/Admin(config-pmap)# class TCP_ANY 
switch/Admin(config-pmap-c)# inspect ftp
Error: This class doesn't have tcp protocol and a specific port

The following examples show some of the generic class-map match statements and an ACL that are not allowed in A2(1.x) inspection configurations:

match port tcp any

match port udp any

match port tcp range 0 65535

match port udp range 0 65535

match virtual-address 192.168.12.15 255.255.255.0 any

match virtual-address 192.168.12.15 255.255.255.0 tcp any

access-list acl1 line 10 extended permit ip any any

For application protocol inspection, the class map must have a specific protocol (related to the inspection type) configured and a specific port or range of port numbers.

For HTTP, FTP, RTSP, Skinny, and ILS protocol inspection, the class map must have TCP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port tcp eq www

For SIP protocol inspection, the class map must have TCP or UDP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port tcp eq 124

or

host1/Admin(config-cmap)# match port udp eq 135

For DNS inspection, the class map must have UDP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port udp eq domain

For ICMP protocol inspection, the class map must have ICMP as the configured protocol. For example, enter the following commands:

host1/Admin(config)# access-list ACL1 extended permit icmp 192.168.12.15 255.255.255.0 
192.168.16.25 255.255.255.0 echo

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match access-list ACL1

Downgrading Your ACE Software from Version A2(1.0) to 3.0(0)A1(6.x) in a Redundant Configuration

If you need to downgrade your ACE software from version A2(1.0) to an earlier version, use the procedure that follows. You can downgrade your ACE from software version A2(1.0) to 3.0(0)A1(6.1) or higher. Downgrading your ACE software to a software version below 3.0(0)A1(6.1) is not supported and not recommended. We recommend that you downgrade to the highest 3.0(0)A1(6.x) software version that is available. This procedure assumes that your ACEs are configured as redundant peers to ensure that there is no disruption to existing connections during the downgrade process. In the following procedure, the active ACE is referred to as ACE-1 and the standby ACE is referred to as ACE-2.

This section contains the following topics:

Before You Begin

Downgrade Procedure

Before You Begin

Before you downgrade your ACE software, ensure that the following conditions exist:

Identical versions of 3.0(0)A1(6.x) software images reside in the image: directory of both ACEs.

The active ACE has a higher priority than the standby ACE and preempt is enabled on the FT group if you want the active ACE to remain active after the downgrade procedure.

Downgrade Procedure

To downgrade your A2(1.0) software in a redundant configuration, perform the following steps:


Step 1 If you have created checkpoints in your 3.0(0)A1(6.x) running-configuration files (highly recommended), roll back the configuration in each context on each ACE to the check-pointed configuration. For example:

host1/Admin# checkpoint rollback CHECKPOINT_ADMIN
host1/Admin# changeto C1
host1/C1# checkpoint rollback CHECKPOINT_C1

Do the same on the other ACE. For information about creating checkpoints and rolling back configurations, see Chapter 4, Managing the ACE Software.

Step 2 Configure ACE-1 to automatically boot from the 3.0(0)A1(6.x) image. To set the boot variable and configuration register to 1, use the boot system image: and config-register commands in configuration mode. For example, enter:

host1/Admin# config
host1/Admin(config)# boot system image:c6ace-t1k9-mzg.3.0.0_A1_6_3.bin
host1/Admin(config)# config-register 1
host1/Admin(config)# exit
host1/Admin# 

You can set up to two images through the boot system command. If the first image fails, the ACE tries to boot from the second image.


Note Use the no boot system image: command to remove the configured A2(1.0) boot variable.


Step 3 Verify that the boot variable was synchronized to ACE-2 by entering the following command on ACE-2:

host1/Admin# show bootvar
BOOT variable = "disk0:c6ace-t1k9-mzg.3.0.0_A1_6_3.bin"
Configuration register is 0x1
host1/Admin#

Step 4 Use the show ft group detail command to verify the state of each module. Upgrade the ACE that has its Admin context in the STANDBY_HOT state (ACE-2) first by entering the reload command.When ACE-2 loads the startup-configuration file, you may observe a few errors if you did not roll back the configuration to a checkpoint. These errors are harmless and occur because the 3.0(0)A1(6.x) software does not recognize the A2(1.0) commands in the startup-configuration file. After ACE-2 boots up, it may take a few minutes to reach the STANDBY_HOT state again. At this time, configuration synchronization is disabled, but the connections through ACE-1 are still being replicated to ACE-2.

host1/Admin# reload
This command will reboot the system
Save configurations for all the contexts. Save? [yes/no]: [yes]

Step 5 Perform a graceful failover of all contexts from ACE-1 to ACE-2 by entering the ft switchover all command in Exec mode on ACE-1. ACE-2 becomes the new active ACE and assumes mastership of all active connections with no interruption to existing connections.

host1/Admin# ft switchover all

Step 6 Reload ACE-1 with the same 3.0(0)A1(6.x) software version as ACE-2. Again, you may observe a few errors as ACE-1 loads the startup-configuration file.

host1/Admin# reload

After ACE-1 boots up, it assumes the role of standby and enters the STANDBY_HOT state (this may take several minutes). You can verify the states of both ACEs by entering the show ft group detail command in Exec mode. Because both ACE-1 and ACE-2 are running the same version of software now, configuration mode is enabled. The configuration is synchronized from ACE 2 (currently active) to ACE-1. If ACE-1 is configured with a higher priority and preempt is configured on the FT group, ACE-1 reasserts mastership after it has received all configuration and state information from ACE-2, making ACE-2 the new standby. ACE-1 becomes the active ACE once again.

Step 7 Perform manual cleanup in the running-configuration files of both ACEs to remove unnecessary version A2(1.0) configuration elements. For example, you may need to remove a service policy from an interface that was part of the version A2(1.0) configuration that is no longer needed in version 3.0(0)A1(6.x).

Step 8 Enter the write memory all command in both ACEs to save the running-configuration files in all configured contexts to their respective startup-configuration files. This action will eliminate future errors when the ACEs reload their startup-configuration files.


ACE Operating Considerations

This section provides the operating considerations for the ACE:

The ACE requires a route back to the client before it can forward a request to a server. If the route back to the client is not present, the ACE cannot establish a flow and drops the client request. Make sure that you configure the appropriate routing to the client network on the ACE VLAN where the client traffic enters the ACE module.

Software version A2(1.0) introduces hardware-assisted SSL (HTTPS) probes. For that reason, the ACE uses the all option for the default SSL version and uses the routing table (which may bypass the real server IP address) to direct HTTPS probes to their destination regardless of whether you specify the routed option in the ip address command. If you are using HTTPS probes in your A1(6.x) configuration with the default SSL version (SSLv3) or without the routed option, you may observe that your HTTPS probes behave differently with version A2(1.0). For more information about HTTPS probes, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

Additionally, hardware-assisted probes are subject to the same key-pair size limitations as SSL termination. The maximum size of a public key in a server SSL certificate that the ACE can process is 2048 bits. For more information about HTTPS probes, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

In software version A2(1.2), the maximum number of match statements per ACE has been increased from 4,096 to 16,384.

The Total Conn-failures counter in the show rserver detail command displays the total number of connection attempts that failed to establish a connection to the real server.

For Layer 4 traffic with normalization on, the count increments if the three-way handshake fails to be established for either of the following reasons:

An RST comes from the client or the server after a SYN-ACK.

The server does not reply to a SYN. The connection times out.

For Layer 4 traffic with normalization off, the count does not increment.

For Layer 7 traffic (normalization is always on), the count increments if the three-way handshake fails to be established for either of the following reasons:

An RST comes from the server after the front-end connection is established

The server does not reply to a SYN. The connection times out.

In software version A2(1.6), the ACE introduces the STANDBY_WARM and WARM_COMPATIBLE redundancy states to handle any CLI incompatibility issue between peers during the upgrading and downgrading of the ACE software. When you upgrade or downgrade the ACE software in a redundant configuration with different software version, the STANDBY_WARM and WARM_COMPATIBLE states allow the configuration and state synchronization process to continue on a best-effort basis. This basis allows the active ACE to synchronize configuration and state information to the standby even though the standby may not recognize or understand the CLI commands or state information. These states allow the standby ACE to come up with best-effort support. In the STANDBY_WARM state, as with the STANDBY_HOT state, configuration mode is disabled on the standby ACE and configuration and state synchronization continues. A failover from the active to the standby based on priorities and preempt can still occur while the standby is in the STANDBY_WARM state.

When redundancy peers run on different version images, the SRG compatibility: field of the show ft peer detail command output displays WARM_COMPATIBLE instead of COMPATIBLE. When the peer is in the WARM_COMPATIBLE state, the FT groups on standby go to the STANDBY_WARM state instead of the STANDBY_HOT state. The following software version combinations indicate whether the SRG compatibility: field displays WARM_COMPATIBLE (WC) or COMPATIBLE (C):

Active ACE Software Version
Standby ACE Software Version
A2(1.3) or less
A2(1.4)
A2(1.5)
A2(1.6)
A2(2.0)
A2(2.1)
A2(2.2)
A2(1.3) or less

C

C

C

C

C

C

C

A2(1.4)

C

C

C

WC

C

C

WC

A2(1.5)

C

C

C

WC

C

C

WC

A2(1.6)

C

WC

WC

C

C

WC

WC

A2(2.0)

C

C

C

C

C

C

C

A2(2.1)

C

C

C

WC

C

C

WC

A2(2.2)

C

WC

WC

WC

C

WC

C


By design, if you set the maximum resources for sticky to unlimited using the limit-resource command, the ACE ignores the setting and sets the maximum value to equal-to-min. In addition, the maximum resource value for sticky in the show resource usage command output displays as 0. This behavior occurs because the ACE does not allow sticky resources to become oversubscribed as with other configurable resources. Instead, when the sticky resource usage reaches the minimum value, the ACE ages out older sticky entries in the sticky table and reuses them for new sticky entries.

When the ACE times out a RADIUS load-balanced (RLB) sticky entry, it only uses connections for the end-user traffic towards the connection count. It does not use connections for the RADIUS traffic towards the connection count, whether or not you configure the timeout activeconns command. The only exception is when a connection has an outstanding RADIUS request for that sticky entry.

Per CSCsz87533, the outbound UDP connection may timeout shortly after the ACE receives a RADIUS request, but before it gets the response for this request from the server. This situation can cause the ACE to improperly forward subsequent RADIUS traffic. If the server is not expected to initiate connections through the ACE, we recommend that you apply an inbound ACL on the server interface to block these connections.

If you downgrade the ACE software from software version 2(1.4), the following messages appear as the ACE boots up:

Starting sysmgr processes.. Please wait...Done!!! 
ACE login: sys_line_cfg_mts_send_receive() fails : Broken pipe 
sys_line_cfg_mts_send_receive() fails : Broken pipe 
...
ACE login: 

When you configure HTTPS probes on the ACE, if a probe fails and the show probe command displays the "Last disconnect err" field with the "Connection reset by server" error message, this message does not accurately reflect the failure. The error could be caused by any number of conditions including expired certificates or unsupported keys.

In software version A2(1.6), the ACE now supports \n as an end of header string for HTTP and HTTPS probes.

ACE Documentation Set

In addition to this document, the ACE documentation set includes the following publications:

Document Title
Description

Cisco Application Control Engine Module Hardware Installation Note

This guide provides information for installing the ACE into the Catalyst 6500 series switch and the Cisco 7600 series router.

Cisco Application Control Engine Module Getting Started Guide

This guide describes how to perform the initial setup and configuration tasks for the ACE.

Cisco Application Control Engine Module Administration Guide

This guide describes how to perform administration tasks on the ACE, including initial setup, establish remote access, configure class maps and policy maps, manage the ACE software, configure SNMP, define system message logging, configure redundancy, and upgrade your ACE software.

Cisco Application Control Engine Module Virtualization Configuration Guide

This guide provides instructions on how to operate your ACE in a single-context or in multiple-contexts. Multiple-contexts use the concept of virtualization to partition your ACE into multiple virtual devices or contexts.

Cisco Application Control Engine Module Routing and Bridging Configuration Guide

This guide provides instructions for configuring the routing and bridging features of the ACE. This guide provides a routing overview and describes how to perform ACE configuration tasks, including:

Configuring VLANs

Configuring routing

Configuring bridging

Configuring Address Resolution Protocol (ARP)

Configuring Dynamic Host Configuration Protocol (DHCP)

Cisco Application Control Engine Module Server Load-Balancing Configuration Guide

This guide describes how to perform ACE server load-balancing configuration tasks, including:

Server health monitoring

Real servers and server farms

Stickiness

Class maps and policy maps to load-balance traffic to real servers in server farms

Firewall load balancing

TCL scripts

Cisco Application Control Engine Module Security Configuration Guide

This guide describes how to perform ACE security configuration tasks, including:

Security access control lists (ACLs)

User authentication and accounting using a TACACS+, RADIUS, or LDAP server

Application protocol and HTTP deep packet inspection

TCP/IP normalization and termination parameters

Network address translation (NAT)

Cisco Application Control Engine Module SSL Configuration Guide

This guide describes how to perform ACE SSL configuration tasks, including:

SSL certificates and keys

SSL initiation

SSL termination

End-to-end SSL

Cisco Application Control Engine Module System Message Guide

Describes how to configure system message logging on the ACE. This guide lists and describes the system log messages generated by the ACE.

Cisco Application Control Engine Module Command Reference

This reference provides an alphabetical list of all command line interface (CLI) commands including syntax, options, and related commands.

Cisco CSM-to-ACE Conversion Tool User Guide

Describes how to use the CSM-to-ACE conversion tool to migrate Cisco Content Switching Module (CSM) running-configuration or startup-configuration files to the ACE.

Cisco CSS-to-ACE Conversion Tool User Guide

Describes how to use the CSS-to-ACE conversion tool to migrate Cisco Content Services Switches (CSS) running-configuration or startup-configuration files to the ACE.

Cisco Application Control Engine (ACE) Module Troubleshooting Guide, Release A2(x)

Describes the procedures and methodology in wiki format to troubleshoot the most common problems that you may encounter during the operation of your ACE.


Software Version A2(1.6a) Resolved Caveats and Open Caveats


Note Software version A2(1.6a) has replaced software version A2(1.6).


The following sections contain the resolved and open caveats in software version A2(1.6a):

Software Version A2(1.6a) Resolved Caveats

Software Version A2(1.6a) Open Caveats

Software Version A2(1.6a) Resolved Caveats

The following resolved caveats apply to software version A2(1.6a):

CSCtc46913—For all proxied connections, the ACE may send packets to a client with a maximum segment size (MSS) of 536 bytes regardless of the maximum transmit unit (MTU) that is configured on the client interface of the ACE. Such proxied connections including the following:

Layer 7 SSL

Layer 7 HTTP traffic with a chunked response

All Layer 7 connections using a connection parameter map with the set tcp wan-optimization rtt command set to 0


Note For a Layer 7 connection, the behavior remains as long as the connection is in the proxied state. When the ACE unproxies the connection, the behavior is not seen.


This behavior does not apply to the following traffic:

Layer 4 connections (for example, regular Layer 4 load balancing, IP stickiness, and so on)

L7 connections where proxy-unproxy occurs. When the ACE unproxies the connection, the behavior is not observed. However, the behavior is seen during the proxied state.

Workaround: Downgrade to software version A2(1.5a). No software workaround is available.

CSCtc55134—When persistence rebalance is configured on the ACE and an MTU that is lower than the default MTU is configured on the client interface, reproxied Layer 7 connections may not learn the MTU that is configured on the client interface. This behavior causes the ACE to send unfragmented packets to the fast path where the packets are dropped and the Drop: No fragmentation of L3 Encap field of the show np 1 me-stats "-s fp" command is incremented. This behavior occurs only for Layer 7 reproxied connections that hit the persistence rebalance configuration. For all other Layer 7 connections, including proxied-reproxied, fully proxied, and SSL, and all Layer 4 connections, this behavior is not seen. Workaround: Disable persistence rebalance or remove the client MTU configuration.

Software Version A2(1.6a) Open Caveats

The open caveats in software version A2(1.6a) are the same as those in software version A2(1.6) except for the two resolved caveats in the "Software Version A2(1.6a) Resolved Caveats" section. For details, see the Software Version A2(1.6) Open Caveats section.

Software Version A2(1.6) Resolved Caveats, Open Caveats, and Command Changes

The following sections contain the resolved and open caveats, and command changes in software version A2(1.6):

Software Version A2(1.6) Resolved Caveats

Software Version A2(1.6) Open Caveats

Software Version A2(1.6) Command Changes

Software Version A2(1.6) Resolved Caveats

The following resolved caveats apply to software version A2(1.6):

CSCse71077—When you configure multiple static routes for the same destination but only one route is reachable, the route table output for the show ip route and show ip fib commands displays that the ECMP flag is set for the unique route entries. This flag should be set only if more than one route for the prefix is in the routing table. Workaround: None.

CSCsi61783—If you initially configure a real server as a Layer 2 real server, and then the interface goes down or is deleted from the configuration, the real server may transition to an ARP_FAILED state and remain in this state after it becomes a Layer 3 real server. Workaround: Reconfigure the real server.

CSCsm92045—When you configure server-farm NAT on the ACE and remove a policy map, the ACE does not remove the association between the interface and NAT. Workaround: To remove the association between the interface and NAT, first remove the Layer 3 rules and then remove the policy map.

CSCsr01570, CSCsy90965—When you change the default class map from a sticky-server farm to none, it does not eliminate the inserting of a cookie and the Set-Cookie: length is null. Workaround: Remove and reconfigure class class-default command.

CSCsu88684, CSCsq27062—When you configure the ACE with a large number of contexts and enable redundancy, as traffic flows on the ACE, the ACE becomes unresponsive and displays the following messages on the console:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

The ACE then reboots. Workaround: None.

CSCsu94371—When you remove a VIP from a policy map, the show cfgmgr internal table icmp-vip command continues to display the removed VIP. Workaround: Reboot the ACE.

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv54222—When an HTTP client sends pipelined requests, if the next request comes in the middle of the server response, the HTTP connection becomes unresponsive and data is missing on the web page. Workaround: Configure a connection parameter-map with the set tcp wan-optimization rtt 0 command.

CSCsw22826—When you configure sticky on the ACE and the traffic generates dynamic sticky entries, if you change the configuration from a sticky to a nonsticky configuration through a rollback or manually, the old sticky entries remain. Workaround: Clear the sticky entries before changing a configuration to a nonsticky configuration.

CSCsx19525—When you configure a large number of SSL VIPs (for example, 1,000 VIPs for the whole system) and configure changes that affect these VIPs, a buffer leak occurs as displayed by the show np 1 me-stats "-scommon" command. Workaround: None.

CSCsx83292—When MTU is configured on the client, the ACE drops Layer 4 class-default packets. Workaround: Remove the MTU configuration.

CSCsy29181—If either of the DP processors is at MAXCONN, the ACE should show MAXCONN in the output of the show serverfarm command. However, the ACE waits until both network processors are at MAXCONN. This issue occurs when the hw-module cde-same-port-hash command is configured. Workaround: None.

CSCsy54551—The show service-policy command displays the connection counts from the service policy but it does not display the Layer 3 rule in the service policy. Workaround: None.

CSCsy58843—When the ACE has a high rate of management traffic, it may become unresponsive due to an ARP failure. Workaround: None.

CSCsy65650—When the ACE reports the termination of TCP flows, it may display incorrect values for the duration and amount of data transferred. This issue occurs with HTTP and connections that are terminated with a TCP RST. Workaround: None. If accounting is needed and relies on this log, use another method.

CSCsy68974—When you configure the SYN cookie and FTP inspection features on the ACE, and the number of embryonic connections reach the threshold, the first FTP inspection connection may encounter a problem if the same connection issues more than one FTP GET request, causing the second FTP GET request to fail. This problem only applies to the first FTP inspection requests that trigger the SYN cookie feature. Subsequent FTP connections succeed as long as the SYN cookie feature is not triggered. Workaround: Disable the SYN cookie feature.

CSCsy88379—The TAC diagnostic script showtech generates large output due to the show xlate command. Workaround: None.

CSCsz09362—When pinging the ACE with small packets, the ACE inserts Ethernet padding into the ICMP data field of a request less than 18 bytes. Workaround: Use larger ICMP packets to stop the ACE from inserting the padding. See the remove-eth-pad command in Table 3.

CSCsz09364—When you create a context with a name that includes a space and allocate an interface VLAN to it, if you either remove the configured context or issue the write memory command, the SSL process becomes unresponsive and the ACE reboots and displays the following message:

Service name:itasca_ssl(922) has terminated on receiving signal 11 

Workaround: Do not configure a context with space.

CSCsz10107—When you configure preempt and the Catalyst switch with an active ACE module reboots, the ACE may not correctly replicate connections after rebooting and becoming active again. Some connections may get dropped. This issue does not occur when rebooting only the ACE or if preempt is not configured. Workaround: None.

CSCsz14634—When you add and remove contexts over a period of time, and you reuse a context ID that was previously configured with the snmp-server community command, the running configuration for the new context contains the snmp-server community command without configuring the command in that context. Workaround: None.

CSCsz18739—When the ACE is configured with RADIUS AAA, the ACE may reboot. Workaround: None.

CSCsz20325—If you attempt to remove a nonexisting inspection policy map and then attempt to remove a configured inspection policy map, the ACE displays an error and does not remove the policy map. Workaround: Reboot the ACE.

CSCsz21527—When you configure an SNMP V3 user with authentication and privacy options on the ACE and attempt to perform an snmpwalk with the authNoPriv option for the same user, the snmpwalk succeeds. Workaround: None.

CSCsz25000—When the ACE is running front-end SSL traffic, a memory leak occurs on both IXPs. This leak happens if the tcp-env information is very lossy and many drop packets in the network occur with duplicate packets and fragmentation. Workaround: None.

CSCsz27257—When you configure the ACE for SSL termination and a client sends multiple single-byte SSL records, the ACE advertises a zero TCP window when terminating the front-end SSL connection and subsequently does not open the window after the underlying data is processed. In some packet scenarios, the ACE does not open the TCP window after the server acknowledges the payload. Part of the scenario also involves the server advertising a zero window to the ACE in conjunction with the ACE advertising a zero window to the client. Workaround: None.

CSCsz28035—Access to the qnx shell from the physical console port of either NP on an ACE places you in a shell. If you type exit, the NP console hangs and becomes inaccessible.

CSCsz34011—After a series of reboots, both ACE modules lose their context configurations. If the active ACE halts and reboots, after it reboots it reads the first half of the startup-config file, establishes FT with the standby ACE (the new active), and synchronizes the configuration to obtain the rest of the configurations from the other ACE. If the other ACE stops functioning, the active ACE does not obtain the rest of the configurations, including context configurations. Context configurations may be lost, although they still exist in the startup-config file. Workaround: None.

CSCsz34933—When you configure a probe with the connection term forced command, the ACE may send a reset with sequence number zero for probe traffic. Workaround: Use the graceful termination no connection term command.

CSCsz40699—When you use the SLB-Admin, Server-Appln-Maintenance, or a custom role with a "create feature server farm" rule, you cannot bring real servers in or out of service under the server farm. Workaround: There are currently no workarounds using these specific roles. However, you can complete these tasks using the Admin role.

CSCsz49088—When you monitor the ACE CPU, you can only monitor it using an Admin role. The show processes cpu command is only available in the Admin role. The Network-Monitor role, which should have access to all show commands, cannot access the show processes cpu command. Configuring a new role on the ACE does not allow you to monitor the system feature. Therefore, only Admin users can run this command. Workaround: Run the show processes cpu command in an Admin role.

CSCsz50090—When you quickly remove a NAT pool and add a new one with more IP addresses, the ACE does not respond to an ARP request sent for IP addresses in its NAT pool. Workaround: None.

CSCsz58417—When you configure any inline match statement in a policy map, the ACE becomes unresponsive for a few minutes and does not apply the configuration. Workaround: None.

CSCsz63457—When you add inspect RTSP under a Layer 4 policy map that is already configured with inspect RTSP, the ACE triggers a download configuration to the data plane. Workaround: None.

CSCsz68435—When the ACE has many concurrent SSL connections and high peak rates, the ACE becomes unresponsive under the SSL traffic load. Workaround: None.

CSCsz82740—When you attempt to disable DHCP relay, the ACE fails to delete the ACL and displays the following error:

Failed to delete acl

Workaround: None.

CSCsz83033—When traffic on the ACE matches a Layer 7 rule, the DSCP/TOS bits set in the packets received from the server are not preserved. Workaround: None.

CSCsz84462—When you configure redundancy on the ACE and then add or delete interface VLANs in a loop or frequently, the active ACE becomes unresponsive and generates an IFMGR core file. Workaround: Do not add or delete VLAN or BVI interfaces in a loop or frequently.

CSCsz92671—When you configure the ACE in bridged mode with a Layer 3 VIP, the ACE bridges relayed DHCP packets in bridged mode instead of load balancing these packets if they match a configured VIP. Workaround: None.

CSCta01789—When the ACE has a large configuration with multiple contexts, and each context has a unique route for the same destination with a different next hop, clearing and copying this configuration can cause the SE flag to be set incorrectly in the routing table. Workaround: None.

CSCta08715—When you configure CSR fields on the ACE, the following error message occurs:

Error: Organization-unit name cannot be composed of these special characters.

Workaround: Use an external tool to generate a CSR (for example, OpenSSL) or ask the CA to generate a key pair and certificate for the ACE.

CSCta09574—When you configure TACACS on the ACE and a TACACS key with a comma (,) character and you reboot the ACE, you must enter the key again for TACACS to work properly. Workaround: Configure the TACACS key on the ACE and TACACS server without a comma character.

CSCta20756, CSCsx15558—Certain crashes on the ACE generate new core files containing debug data. Workaround: None.

CSCta25613—When using RADIUS load balancing, the ACE may become unresponsive and generate a loadBalance_g_ns core file. Workaround: None.

CSCta28624—When you configure the MTU in an interface to a value other than the default of 1,500, reuse and reproxy fail. When you configure the MTU in the client interface, SYN cookie fails. Workaround: Remove the MTU configured for the interface.

CSCta30959—When you configure redundancy on the ACE, configuration mode is enabled on the active ACE when the standby ACE is in the standby-configuration state. During standby-configuration synchronization, configuration mode is enabled for a short time and any command that you enter during that time is lost. Workaround: Do not enter or change any command during a bulk configuration synchronization.

CSCta41421—The ACE module may become unresponsive due to an internal error, but it does not reboot and it does not generate complete core files. Workaround: None.

CSCta43466—When you do not configure a real server in the server farm, the ACE does not generate the closing XML tag for the server farm detail output. Workaround: Configure a dummy real server on the server farm.

CSCta53085—When you configure scripted probes on the ACE, if the disk is full and the ACE retrieves the exit_msg command from the script, occasionally the ACE reboots. Workaround: None.

CSCta57280—When you use the capture command to take packet captures on the ACE, some frames may be truncated. Workaround: None.

CSCta78220—When the ACE is under heavy load through XML connections to the local interface, the ACE can reboot without a core file, generate a kernel crash, or lock out management functions. This condition is due to over consumption of resources by XML of memory and CPU. Workaround: Disable XML access to the ACE or stop XML polling of the ACE from customer management stations.

CSCtb03844—When you configure failaction reassign on a server farm configured with cyclic backup and both real servers are in the failed state, the ACE becomes unresponsive. Workaround: None.

CSCtb07772—When the ACE is reproxying, it drops server packets larger than the server advertised maximum segment size (MSS) which leads to the stalling and eventual timeout of the connection. Workaround: Configure a parameter map with the exceed-mss allow command.

CSCtb08318—When you configure the snmp-server unmask-community command in a non-Admin context on the active ACE, incremental synchronization does not synchronize this command on the standby ACE. Workaround: Perform bulk synchronization to the standby ACE. You can execute the no ft auto-sync running-config and ft auto-sync running-config commands on the active ACE whenever you configure or unconfigure the snmp-server unmask-community command in a non-Admin context.

CSCtb13426—After the ACE runs for a long time without a reboot or there is a lot of communication between the supervisor engine and the ACE, when you enter the show scp stats command, the TX bytes field displays a negative byte count in its output. Workaround: None.

CSCtb13438—When you enter the supervisor no power enable module slot_number command for the slot number of the standby ACE, the standby ACE asserts itself to be the active ACE before the shutdown and both ACEs become active. Workaround: None.

CSCtb23312—The ACE becomes unresponsive when its uptime reaches approximately 485 days. Workaround: Gracefully reboot the ACE before its uptime reaches 480 days.

CSCtb28897—If you repeatedly enter commands related to SNMP traps for the server farm or the username command on the ACE CLI, an MTS buffer can leak. Overtime, a shortage of MTS buffers can cause the ACE to be unresponsive to management commands. Workaround: Do not repeatedly enter commands related to SNMP traps for the server farm or username command from the CLI. Monitor the MTS buffers through the show system internal mts buffer details command. If you detect a leak, schedule a reboot of the ACE.

CSCtb35900—When all of the ports for the first IP address in the NAT pool are used up, NAT pool exhaustion occurs and ACE-wide problems occur. Workaround: Configure a single NAT pool range, for example, nat-pool 5 10.147.2.11 10.147.2.14 netmask 255.255.255.255 pat.

CSCtb38297—When you configure the weighted leastconn configuration on the ACE, the ACE sends a majority of the traffic to a few of the real servers in a server farm and very little traffic to the other real servers. When the real servers are in a failed state (PROBE_FAILED) and configured with custom weights, a configuration download occurs. Workaround: Perform one of the following:

Change any configuration on the affected server farm when all the real servers are operational. For example, enter the no inservice and inservice commands of any real server in the server farm.

Remove the weight configuration.

Remove the probe configuration and then make a configuration change when all real servers are operational. Readd the probe configuration after 30 seconds.

CSCtb60118—After you reboot the ACE, the SSH key for management connections is different from the SSH key prior to the reboot. When the SSH key is generated on an active ACE and synchronized to the standby ACE, the standby ACE does not properly store the new SSH key in NVRAM. Workaround: If you remove the SSH key, use the write memory command. After a key is generated, use the write memory command on the active and standby ACE prior to the reboot.

CSCtb68393—When you configure the ACE for LDAP authentication but incorrectly define an LDAP server, the ACE CLI becomes unresponsive if there are not enough MTS buffers for intrabox communication. Workaround: Remove the LDAP authentication configuration. Then, properly configure the LDAP server.

Software Version A2(1.6) Open Caveats

The following open caveats apply to software version A2(1.6):

CSCse12120—When you press Ctrl-D and attempt to log in to the ACE with a valid username and password using the session command through EOBC from the supervisor engine, the login attempt fails. Workaround: Press Ctrl-D twice to access the switch login, and then log in to the ACE.

CSCsk82966—Occasionally, when the allocation of the regex resource is out of memory, the regex deny counter displayed by the show resource usage command does not increment. Workaround: None.

CSCsr76812—When you configure the ACE with Layer 7 load balancing, TCP connections may be disrupted. Packets arrive at the client in reverse order or packets are forced to be resent. Workaround: None.

CSCsv31046—When you configure the least-connections predictor on the ACE, the ACE may not sustain 160,000 CPS traffic. Workaround: None.

CSCsv92321, CSCsx25981—The ACE module reboots unexpectedly and writes a core file to disk. Workaround: None.

CSCsx13853—When you specify TCP as the protocol in a global access list configured for DNS traffic, DNS inspection fails. Workaround: Specify only UDP as the protocol in the global access list configured for DNS traffic.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx41539—The ACE module may reboot and generate the following core files:

last boot reason: NP 0 Failed : NP Process Crashed
   182284  Feb 1 15:53:45 2009 qnx_1_mecore_log.999.tar.gz
   687601  Feb 1 15:53:41 2009 qnx_1_io-net_core_log.114693.tar.gz
   113726  Feb 1 15:53:47 2009 ixp1_crash.txt

Workaround: None.

CSCsx81743—HSRP or other multicast control packets might be either lost for up to 10 seconds toward the CPU or flooded in case of a link flap, as observed in the following conditions:

The Catalyst 6500 series switch is running Cisco IOS release 12.2(33)SXH3a or 12.2(33)SXI.

The port channel spans multiple modules. This condition has been seen in a combination of WS-X6708-10GE and supervisor engine EtherChannel or WS-X6708-10GE and WS-X6708-10GE EtherChannel.

The Supervisor Engine 720 and Supervisor Engine 4 is in the Catalyst 6500 series switch chassis.

The port that is flapping is not a port on the supervisor.

The ACE is load balancing traffic in the chassis.

Workaround: None.

CSCsy34814— The syslog message 305010 includes the duration of the Xlate translation. However this duration is always equal to the Xlate idle timeout. Workaround: Use the timestamps in the creation and tear down of the Xlate connections to calculate the Xlate duration.

CSCsy98701—When you configure two ACEs as FT pairs that are replicating sticky entries and you enter certain show commands on the active ACE, the standby ACE generates a Load Balance core file. Workaround: None.

CSCsz19782—When you convert the configuration from a non-full proxy to a full proxy configuration for full proxied new connections and you add new VIPs for load balancing, traffic to these VIPs do not go through the ACE. Workaround: Reboot the ACE.

CSCsz22742—When you copy a large configuration to the running-config file, API timeout errors occur. Workaround: None.

CSCsz85367—When you configure and unconfigure access lists in a loop, the ACE leaks memory. Workaround: Do not configure and then unconfigure access lists in a loop.

CSCta03202, CSCsz92427—When you remove and readd the inspect protocol command under a VIP class from a multi-match policy map, the following error occurs:

Error: This class doesn't have tcp protocol and a specific port

You cannot unconfigure inspection other than HTTP inspection from a policy map. Workaround: Remove the VIP class from the multi-match policy map and reconfigure it.

CSCta13446—When you remove and then reapply the inspect ftp command, the ACE drops connections. Workaround: None.

CSCta47529—When you configure the ACE for DHCP relay on the interface, it can fail to forward unicast DHCP packets for DHCP relay processing. Workaround: None.

CSCta49917—When Telnet connections, SSH connections, or a debug session are active for a long time on the ACE, they do not close properly as indicated by one of the following:

The MTS buffers increases after each changeto command as displayed by the show system internal mts buffers command.

The following error message occurs:

IPC queue full. Clear idle telnet/ssh connections or debug plugin sessions to 
recover err

Workaround: You can either Telnet to each context to make configuration changes or reboot the ACE.

CSCta77955—The ACE may unexpectedly reboot and generate a minimal core file on the disk. Workaround: None.

CSCtb00726—If the VIP address conflicts with the shared interface address across contexts, the standby ACE goes into the cold state with the show ft config-error command displaying the following error message:

interface vlan number 
Error: Global Policy applied, conflicts with VIP, NAT or Interface IP in shared 
interface!

Workaround: Do not configure a VIP address with the same address as the shared interface IP address on which the service policy is configured.

CSCtb15183—When you perform multiple dynamic configurations and use of the resequence option on an access list, duplicate access-list line numbers may occur on the ACE, additional resequence commands will fail, and you cannot add an object. Workaround: Reboot the ACE to clear this condition.

CSCtb16605—When you add the cookie secondary command to a sticky group after you assigned the group to a policy and an interface, this command has no effect. Workaround: Remove the policy and reconfigure it.

CSCtb21313—When you configure persistence rebalance in a configuration with two server farms containing the same real server with different port numbers and attached to two different Layer 7 policy maps, connections are dropped intermittently after a rebalance occurs to a different Layer 7 policy. Workaround: None.

CSCtb25491—After modifying an access list and then resequencing it in quick succession, the following error message appears in the syslog file:

WARNING: Unknown error while processing access-group. Incomplete rule is currently 
applied on interface vlanXXXX. 

Workaround: Manually roll back to a previous access rule configuration on the interface. Do not enter resequence commands in quick succession. After you execute a command, reenter it with a different line number.

CSCtb27018—When you configure the ACE for SIP UDP, the ACE does not accept the SIP UDP probes requests because the source port of the 200 OK message is different than the destination port of the OPTIONS method. Workaround: None.

CSCtb29571—After you repeatedly configure and unconfigure DHCP in Admin and user contexts, the DHCP relay service may restart. Workaround: None.

CSCtb44729—When you configure the ACE for Layer 7 load balancing and a connection is closed before it is processed by the load balancer, the show conn command displays no connections but the show serverfarm command displays the current connection for the real server even after all traffic has stopped. Workaround: Remove the real server and readd it.

CSCtc46913—For all proxied connections, the ACE may send packets to a client with a maximum segment size (MSS) of 536 bytes regardless of the maximum transmit unit (MTU) that is configured on the client interface of the ACE. Such proxied connections including the following:

Layer 7 SSL

Layer 7 HTTP traffic with a chunked response

All Layer 7 connections using a connection parameter map with the set tcp wan-optimization rtt command set to 0


Note For a Layer 7 connection, the behavior remains as long as the connection is in the proxied state. When the ACE unproxies the connection, the behavior is not seen.


This behavior does not apply to the following traffic:

Layer 4 connections (for example, regular Layer 4 load balancing, IP stickiness, and so on)

L7 connections where proxy-unproxy occurs. When the ACE unproxies the connection, the behavior is not observed. However, the behavior is seen during the proxied state.

Workaround: Downgrade to software version A2(1.5a). No software workaround is available.

CSCtc55134—When persistence rebalance is configured on the ACE and an MTU that is lower than the default MTU is configured on the client interface, reproxied Layer 7 connections may not learn the MTU that is configured on the client interface. This behavior causes the ACE to send unfragmented packets to the fast path where the packets are dropped and the Drop: No fragmentation of L3 Encap field of the show np 1 me-stats "-s fp" command is incremented. This behavior occurs only for Layer 7 reproxied connections that hit the persistence rebalance configuration. For all other Layer 7 connections, including proxied-reproxied, fully proxied, and SSL, and all Layer 4 connections, this behavior is not seen. Workaround: Disable persistence rebalance or remove the client MTU configuration.

Software Version A2(1.6) Command Changes

Table 3 lists the commands that have changed in software version A2(1.6).

Table 3 CLI Commands Changed in Version A2(1.6)  

Mode
Command and Syntax
Description

Exec

clear stats resource-usage

The new resource-usage keyword clears the Peak and Denied fields displayed by the show resource usage command.

Exec

copy checkpoint:name {disk0:[path/]filename | image:[image_name] | startup-config | ftp://server/path[/filename] | sftp://[username@]server/path[/filename] | tftp://server[:port]/path[/filename]}

The new checkpoint keyword allows you to copy the checkpoint file to disk0, the image directory, the startup configuration file, or a remote server.

Exec

copy {disk0:[path/]filename | image:[image_name] | running-config | startup-config | ftp://server/path[/filename] | sftp://[username@]server/path[/filename] | tftp://server[:port]/path[/filename]} checkpoint:name

The new checkpoint keyword allows you to copy the checkpoint file from disk0, the image directory, the running configuration file, the startup configuration file, or a remote server.

Exec

show accounting log all

The new all option in the Admin context displays the accounting log for all contexts.

Exec

show interface

This command now displays the following:

The reason for the interface to transition to the Up state

Time stamp when the last change occurred

Number of transitions the interface experienced since it was created

Last three previous states including the timestamp and the reason for the Up or Down transitions

Exec

show np np_number nat policies

This command no longer displays bitmap information.

Exec

show service-policy [policy_name] summary

This command now displays a summary of current, hit and drop connections for all VIP addresses in a Layer 3 rule. Previously, this command displayed connection counts for each VIP address even if the address was not hit. However, the ACE calculates connection counts per Layer 3 rule, not per VIP address.

Exec

show stats loadbalance

This command now displays the following two counters:

Total proxy misses—Total number of dropped connections when the related proxy is closed, the connection is dead, or the proxy sequence number does not match.

Total misc errors—Total number of dropped connections for miscellaneous errors, for example, remote sticky lookup timeout, pmap errors, or POST message to an HTTP failure.

Total L4 Close Before Process—For future use. Currently, this counter does not increment.

Total L7 Closs Before Parse—For future use. Currently, this counter does not increment.

Total Close Msg for Valid Real—Total number of close connection messages with a valid real server ID.

Total Close Msg for Invalid Real—Total number of Total number of close connection messages with a valid real server ID. This counter increases only in the Admin context.

Exec

show system resources

This command is now available in all user contexts. Previously, this command was only available in the Admin context.

It also now displays the Average ME Utilization statistics.

Exec

show tech support

The CLIs that the show tech support command executes are no longer logged.

Also, the show tech support command includes the show accounting log all command in the Admin context.

Configuration

context name

Per CSCsu76777, this command now prohibits you from configuring a context name containing opening braces ({), closing braces (}), white spaces, or any of the following symbols: ` $ & * ( ) \ | ; ' " < > / ?

Configuration

logging reject-newconn

This command has been removed from the ACE CLI.

If you upgrade the ACE to software release A2(1.6) but had previously configured the logging reject-newconn command in the earlier release, the ACE will display the following execution error message:

'logging reject-newconn keyword' 
*** Context number: cmd parse error *** 

To avoid this error message, delete the logging reject-newconn command from the startup-config file before you upgrade the ACE.

Configuration

snmp-server enable traps slb serverfarm

The new serverfarm option sends a trap when all real servers are down in the server farm or the server farm changes state.

The CISCO-SLB-EXT-MIB MIB now includes the cslbxServerFarmStateChange trap. This notification is supported with the following varbinds:

cslbxServerFarmName

cslbxServerFarmState

cslbxServerFarmStateChangeDescr

cslbxServerFarmNumOfTimeFailOvers

cslbxServerFarmNumOfTimeBkInServs

The server farm can change from the inactive to active state or active to inactive state. The reasons for changing from the active to inactive state are as follows:

All the real servers are down.

One or more real server is in the maximum connection or maximum load state.

The server farm reaches its partial limits.

Interface

remove-eth-pad

The remove-eth-pad command has been added in A2(1.6) to enable an internal length check and remove any trailer bytes (appended to the end of an Ethernet IP frame) coming into the ACE. This check is performed for each interface and is disabled by default.

(Added per CSCsz09362)

Parameter map

description string

no description

This new command allows you to provide a description for the parameter map. The string argument is a maximum of 240 characters. Use the no form of the command to remove the description.

The show parameter-map command displays the description string.

Parameter map HTTP configuration

persistence-rebalance strict

Per CSCsy21634, the new strict option for this command allows you to configure the ACE to load balance each subsequent GET request on the same TCP connection independently. This command allows the ACE to load balance each HTTP request to a potentially different Layer 7 class and/or real server.

For example, enter:

host1/Admin(config)# parameter-map type http 
http_parameter_map
host1/Admin(config-parammap-http)# 
persistence-rebalance strict

By default, persistence rebalance is disabled. To reset persistence to the default setting of disabled, enter:

host1/Admin(config-parammap-http)# no 
persistence-rebalance

To change to persistence rebalance behavior that does not load balance successive GET requests to the same TCP connection, use the persistence-rebalance command.

Policy map

description string

no description

This new command allows you to provide a description for the policy map. The string argument is a maximum of 240 characters. Use the no form of the command to remove the description.

Server farm

use-same-np

This new command enables the full maximum connection calculation in a single NP. Use the no form of the command to disable the full maximum connection calculation in a single NP.

Before configuring the use-same-np command, configure the hw-module cde-same-port-hash command in configuration mode.


Software Version A2(1.5a) Resolved Caveats and Open Caveats

The following sections contain the resolved and open caveats, and command changes in software version A2(1.5a):

Software Version A2(1.5a) Resolved Caveats

Software Version A2(1.5a) Open Caveats

Software Version A2(1.5a) Resolved Caveats

The following resolved caveats apply to software version A2(1.5a):

CSCsx68671—When a mix of UDP and TCP Layer 7 traffic is flowing through the ACE, the ACE may experience a large memory leak in the internal buffers of the data plane. This memory leak occurs with L7 UDP connections, generic protocol parsing, payload sticky, and UDP fast age traffic. Workaround: None.

CSCsz77633—When the ACE is receiving Layer 7 traffic, it may discard Layer 4 sticky connection requests on the same or on a different context because the ACE may incorrectly reset the connection after traffic is sent for some duration. You should not encounter this issue with only Layer 4 traffic or only Layer 7 traffic. The issue is seen only with the combination of the two types of traffic. Workaround: None.

CSCsz86630—DNS inspection may not work after you upgrade from software version A2(1.1) to a higher release. The issue occurs only for a percentage of responses and it builds over the time. The following errors appear in the output of the show np me-stats -sfixup command in the higher release:

+[Hash miss errors]

+ [NAT app fixup response error]

Workaround: Disable inspection and configure more aggressive timeouts (for example, 4 seconds) for UDP and port 53.

CSCta03825—When UDP booster is configured, the ACE does not forward every first packet from a new client's DNS request to a real server on each network processor (NP). Therefore, two packets (one for each NP) are dropped for each session. Workaround: Disable UDP booster.

CSCta29049—When UDP booster is configured, the ACE drops UDP packets originating from the server. Workaround: Disable UDP booster.

Software Version A2(1.5a) Open Caveats

The open caveats that apply to software version A2(1.5a) are identical to the "Software Version A2(1.5) Open Caveats" list except the caveats that have been resolved in A2(1.5a).

Software Version A2(1.5) Resolved Caveats, Open Caveats, and Command Changes


Note If you plan to configure IP-address stickiness in your network, we strongly recommend that you upgrade your ACE software to version A2(1.5a). For details, see resolved caveat CSCsz77633 in the Software Version A2(1.5a) Resolved Caveats section.


The following sections contain the resolved and open caveats, and command changes in software version A2(1.5):

Software Version A2(1.5) Resolved Caveats

Software Version A2(1.5) Open Caveats

Software Version A2(1.5) Command Changes

Revised System Log Messages

Software Version A2(1.5) Resolved Caveats

The following resolved caveats apply to software version A2(1.5):

CSCsh04655—When you use the Generic Protocol parser to load balance some types of TCP traffic, connections may hang and no outbound leg is established if fewer than the configured max-parse-length number of bytes are sent by the client.

CSCsi87346—The ACE capture file may be written to disk with no read bit set, which results in a failure when you attempt to copy the capture from a disk to FTP. This fault is triggered when you enter the show capture capture detail command before the capture is written to disk.

CSCsk89686—On ANM 1.1, when you import an ACE module using the "Perform initial setup and import"option, the operation may fail with an error. This issue occurs when you use ANM 1.1 and ACE A2(1.6(2)).

CSCsm08521—A stale MAXCONN state is displayed in the show serverfarm command when the difference between the Max and Min config is so low that the real state oscillates between the OPERATIONAL and MAXCONN states and the ACE experiences large amounts of traffic.

CSCso66776—The following error message is displayed when a server farm goes down even if there is no backup server farm:

%ACE-5-441001: Serverfarm () failed over to backup. Number of failovers = 6, number of 
times back in service = 6 

CSCsr19340—When you configure authentication on Cisco ACS (TACACS or RADIUS), the user admin cannot log in the ACS for console authentication.

CSCsr73873—When you configure PAT on the ACE, if there is a very large amount of traffic, the show xlate command displays the following output, "Got no reply."

CSCsu01728—SSL URL rewrite does not work when the server sends a location that is not exactly spelled "Location."

CSCsu19052—Connection replication to standby stops after you remove and readd a peer IP address. This issue does not exist when you directly change the IP address by using the peer ip address command to overwrite the existing IP address on standby without first removing it.

CSCsu31311—When an active ACE of a redundant pair attempts to open a connection to one of its real servers that is remote, the ACE sends a packet to the next hop. However, the next hop has no route to the real server and sends back ICMP 3 (unreachable) to the ACE. The ACE sends this packet back to the next hop even though the packet is destined to the ACE physical interface. The ICMP packet bounces back and forth until the TTL expires.

CSCsu87573—When an ANM sends a CLI through the XML agent and it fails, the configuration count increases.

CSCsu96977—When you configure more than 640 action lists and enter the do show action_list command with the Tab or ? key for help, the ACE becomes unresponsive.

CSCsv50138—SSL-terminated connections fail due to an SSL resource allocation failure. New connections will perform a 3-way TCP handshake, but will be terminated with a reset.

CSCsv60332—When you add a new match statement to a class map, the cfgmgr sends duplicate line numbers to the ACL module.

CSCsv60443—When you enable and start, the ARP entries fail to refresh, which causes the connections to go down. This issue may occur when the packet capture is running for a long time (over 15 minutes to hours).

CSCsv74527—When DNS traffic runs consistently at more than 10000 CPS, proxy entries are leaked on the standby ACE in a HA environment after approximately two hours. Proxy entries are leaked and not cleared on the standby ACE due to connection validation errors.

CSCsv82638—When you use the XML interface to configure ip commands on the ACE, the INVALID_ATTR error is displayed.

CSCsw20096—Configuring a logging level does not work for some syslogs. The running-config shows the updated value, but the actual syslog generation is based on the default level. This issue is applicable to the syslogs generated from the dataplane.

CSCsw23356—The show serverfarm command output may show some current connection entries even when there is no traffic. This issue may be seen when the module is configured for point-to-multi point and point-to-point traffic and inspect enable is configured for the protocol.

CSCsw34919—In a redundant configuration that has not been rebooted, when you add a context, the active or standby ACE includes the newly created context in SNMP traps; however, the peer does not include it.

CSCsw42866—When two ACEs are in the active-standby state and the Admin context configuration is not synchronized but the user contexts are synchronized, the standby ACE does not allocate sticky resources. When you configure a sticky group on the active ACE, inside a user context, the configuration will not be synchronized because no resources are available on the standby ACE, although the standby ACE stays in the standby-hot state.

CSCsw66106—When a server sends traffic to another server in the same subnet using the following as destinations:

L3: the IP address of the server

L2: a multicast MAC address

the ACE sends the same packet to the server but changes the source MAC address to its own MAC address on the VLAN. The destination server sees duplicate packets as a result.

CSCsw67027—When you enter the show conn count command, the connections increase continuously without any traffic flowing through ACE.

CSCsw70487—Under normal conditions, the ACE unexpectedly generates a kernel core file.

CSCsw80486—When an HTTP probe contains a URL with a space character and a bulk configuration synchronization is triggered, the operation fails and the standby ACE status changes to the standby_cold state. This condition affects bulk configuration syncs only (for example, after a switchover) and only when the configurations of the two ACEs are out of synchronization. If the two configurations are in synchronization (equal), then the bulk configuration synchronization will perform as expected and not fail. Also, when you initially configure the probe, it successfully synchronizes to the standby ACE during the incremental configuration synchronization.

CSCsw81300—When you configure the ACE with an HTTP inspection and HTTP load-balance policy map with only a class-default class, server-connection reuse does not allow traffic.

CSCsw88171—When you make health monitoring changes, MTS data corruption occurs. The ACE reboots and generates a core file.

CSCsw97987—Traffic destined for a class map gets a hit when you try to readd the same class-map to the same policy map. This issue occurs only if you have deleted and readded the other class maps that belong to the multi match policy map.

CSCsx24507—The ACE may stop functioning when you make SSL configuration changes during SSL traffic. The ACE displays the following message:

map_sram_particle_v2p: 180246 Invalid SRAM address physical 0 virtual c0099678 

CSCsx26195, CSCsx44351—When you configure the ACE with the failaction purge command and then enter the no service command, the ACE continues to receive and create connections for a few seconds. This problem does not occur when the probe goes down.

CSCsx38506—When you enable the display of raw XML request show command output in XML format on the ACE through the xml-show on command, the following policy maps are missing their XML closing tags:

policy-map type loadbalance generic

policy-map type loadbalance sip

policy-map type loadbalance rtsp

CSCsx41818—Some SSL connections may continue to be accepted in the ACE even though the reference CRL against which the revocation check needed to be performed gets removed from the system.

CSCsx42081—The ACE may perform a system reload when receiving FTP traffic at a high connection rate.

CSCsx46701—When you have a match-all VIP of 0.0.0.0 and you attempt to remove an rserver IP address, you will receive the following error message:

Error: Rserver address is the same as a VIP address. 

CSCsx47594—The ACE stops functioning during SSL back-end traffic that includes HTTPS probes. The SSL server does not use an RSA certificate.

CSCsx48066—In software version A210a, the ACE may experience certain command failures that indicated a full disk on the system. The ACE also experiences repeated core files at the same time.

CSCsx48286—The ACE may experience a delay when it processes Multicast Entry Table (MET) update messages received from the supervisor engine over Switch Mode Configuration Protocol (SCP). This delayed processing causes the installation of entries in the hardware to take longer than usual.

CSCsx49315—When np1 reaches the MAXCONN state, it drops the next request packet. When np2 reaches the MAXCONN state, it forwards the next request packet to another rserver and overwrites the sticky entry. Both network processors should treat packets in the same manner even after reaching the MAXCONN state.

CSCsx52625—An invalid reference to object policy-map_loadbalance may occur inside the dtd.

CSCsx53491—When a bad SSL packet causes the ACE to reset the SSL handshake, the ACE does not issue a fatal alert for ERR_SSL_MAC_MISCOMPARE in response to the bad SSL packet.

CSCsx56801—The memory that stores the dataplane code may encounter a bit-flip error. This error is rare and indicates only a transient hardware fault, which may cause the ACE to become unresponsive.

CSCsx57861—When the standby ACE receives an ACK from a bad client, it sends an ACK that contains a virtual MAC address (VMAC).

CSCsx64561—gslb_proto does not create a core dump because of a segmentation fault.

CSCsx65121—A system (Fastpath) failure occurs when the ACE sends arbitrary HTTP traffic with SYN-ATTACK and syn-cookie enabled.

CSCsx65467—The VLAN interfaces appear in the Down state on the standby ACE. The VLAN interfaces appear as Up only when certain attributes (such as the autostate \ flag) have been met. This issue occurs when the ACE modules are in a VSS setup with a DFC card in the VSS setup.

CSCsx67908—When you configure ACEs for redundancy and Route Health Injection (RHI) and the standby ACE reboots, duplicate RHI entries can exist on the supervisor.

CSCsx71830—SSL probes fail on bootup for typically a minute. The server farm or real server to which the probe is attached is out of service on bootup. This behavior is seen only on bootup.

CSCsx72444—When you configure a syslog over TCP to send messages to a server and the server closes the connection due to a failure or a restart, the ACE closes its own socket. When the ACE closes the socket, it never tries to reopen it and no more messages are sent.

CSCsx73473—When you configure the ACE for a primary and secondary RADIUS server and the primary RADIUS server is down, the RADIUS Access-Request has a duplicate attribute pair (NAS-IP-Address) that causes the RADIUS authentication to the secondary RADIUS server to fail.

CSCsx76500—When you use the crypto verify command on an Elliptic Curve certificate, the ACE stops functioning.

CSCsx78153—When you log into the web GUI from an ACE module, a check on the username is done. If the username does not start with "admin" in the GUI, it will not be shown in the CSS2ACE tool.

CSCsx80946—After a switchover, repeatedly running the show mac-address-table | inc 000b.fcfe.1b command on the Catalyst 6500 series switch displays entries in the ACE FT VLAN. The ACE virtual MAC address is of the form 000b.fcfe.1bXX.

CSCsx80991—When you configure a real server in a server farm with least connections (leastconns) without the slowstart option, the ACE stops using the least connection setting when you add real servers to the server farm.

CSCsx81701—When you create the server farm and attach the real servers to it, memory is used. Even after deleting the server farms, used memory is not released.

CSCsx83706—On an FT switchover, mac-move port flapping may occur on the Catalyst 6500 series supervisor due to spanning tree convergence or residual traffic that the ACE device that has newly transitioned to standby may continue to send.

CSCsx93208—After an FT switchover, the supervisor on the Catalyst 6500 series switch that has the newly standby ACE may have CAM entries for the ACE virtual MAC addresses (MAC addresses starting with 000b.fcfe.1bXX) in one or more VLANs that point to the standby ACE. This issue causes problems if VIP traffic from clients comes in from the Catalyst 6500 series switch as it gets blackholed.

CSCsy00532—Due to an error in the coring process, file permissions on the core files are not set properly. Therefore, it is not possible to copy the cores from the core: directory using the copy command. Access to the debug shell is required to recover the cores.

CSCsy00984—The ACE does not preserve the DSCP value that comes in the packet from the client side and remarks it to 0.

CSCsy01051—Even if all the real servers in the server farm are in the MAXCONN state, the ACE will not fail over to the backup server farm. As a result, the ACE will reject all the new connections that hit the VIP. This issue occurs only if the conn-limit is applied at the rserver level.

CSCsy01247—When ssl proxy has both a chaingroup and an authgroup with large size CA or intermediate CA certificates configured, the ACE has issues.

CSCsy05586—During bootup, the admin context may come up in the STANDBY_COLD state on the standby side, if the configuration is large in the Admin context.

CSCsy05677—An HA_DP_MGR crash will cause the ACE to reload. This issue occurs when an invalid FT group ID is received by the ha_dp_mgr.

CSCsy07862—The ACE stops functioning after you remove the RADIUS server configuration.

CSCsy10361—When the ACE experiences heavy XML traffic, the available memory will go down drastically. The following message will be displayed and the ACE may become unresponsive:

Available CP memory less than 5%: 41033728 bytes.  
Free high memory: 15376384 bytes 
Total memory: 847978496 bytes, Total high memory: 671088640 bytes 
System running low on direct mapped memory 
Please issue 'show system kcache' to diagnose further 
Available CP memory less than 1%: 6746112 bytes. Free high memory: 2654208 bytes

CSCsy13724—If the transparent probe traffic that is destined to multiple real servers using a single probe address is interleaved, the ACE will get its destination MAC address mixed up. This issue will not occur if a probe runs from start to finish without interruption from any other probes to the same probe address.

CSCsy16332—When the ACE reaches the MAXCONN state, NPL-dropped connections may get listed in the show conn command display and are recorded in the show logging command display.

CSCsy18932—When IPCP messages are received from the CP, all commands time out. This issue occurs when you add a backup real server in the server farm that has the conn-limit configured for the real servers and traffic is running in the background.

CSCsy26136—The ACE stops functioning and is unable to post a message to CFGMGR.

CSCsy27041—When packets from an ACE are sent with the source MAC address set to the same MAC address as the next hop router, the MAC table on the intermediate switch is corrupted. This issue is an error-case when there is a loop in the network.

CSCsy27632—When you configure the logging level 251010 to any (non default) level, and then remove the command using the no logging message 251010 level command, logging continues at a previous level even though the show logging message 251010 command shows default logging level.

CSCsy29247—The ACE experiences a memory leak if a delete/add match statement exists in the loadbalance class-map.

CSCsy29490, CSCsm65862—When the existing connection and proxy ID allocation scheme is based on a Last in First Out (LIFO) scheme, under traffic stress conditions, this condition causes multiple issues due to immediate connection or proxy ID reuse.

CSCsy41558—The show stats crypto server and show stats crypto client commands are timing out when where large numbers of VIPs (for example, 1500-2000) exist.

CSCsy42160—A network processor memory dump process halts when you apply a CRL to an SSL proxy. When traffic is flowing and you reconfigure a large number of VIPs simultaneously, the network processor state is reported as unresponsive. This unresponsive network processor state causes the ACE to reboot and to produce a network processor core dump.

CSCsy44007—For some connections, sticky cookies may fail to stick clients to the proper server. This issue only appears to be specific when the server is doing a set-cookie that is causing the cookie to be inserted into the sticky database. If the client makes a second request that contains the cookie before the ACE has completed insertion of the sticky entry into the sticky database, stickiness may fail.

CSCsy47190—When a stray connection to the TL server is attempted, the TL server halts when that connection is closed.

CSCsy53839—Syslogs generated by the ACE are using the wrong data format and the wrong source IP address. This issue occurs for the syslogs generated for ICMP connections.

CSCsy55230—The cookie-insert sticky setting may stick unexpectedly when using a backup server farm.

CSCsy58285—SSL connections stall on invalid SSL messages while waiting for more data.

CSCsy59156—When you remove a real server from a server farm in a particular context, the ACE checks if sticky entries are associated with the real server. If so, then the entries need to be removed.

CSCsy59246—The HA manger process halts when you upgrade to software version A3(2.2).

CSCsy61151—When you attempt to configure the ACE for SSL termination with the XML interface results in an error response from ACE, the ACE is not able to process the ssl-proxy configuration request and instead responds with an XML_ERR_ATTR_INVALID error.

CSCsy73632—The ACE does not send a fatal alert and the connection stalls. When the SSL Application_data and SSL alert headers are interchanged on SSL client machines, if a client sends a request for the get/post data after a successful handshake, then the ACE does not send an alert to the client.

CSCsy83533—You cannot initiate new management connection or ping to the ACE 4710 under the following conditions:

The 100percent bandwidth resource is allocated for another context.

The 100 percent bandwidth resource is used as a result of configuring shared bandwidth resource usage of maximum to "unlimited" and because the traffic is overwhelming the ACE to maximum capacity.

CSCsy84285—The ACE does not properly handle incremental and bulk configuration synchronizations when using the snmp-server engineid command.

CSCsy84895—Server packets that are larger than the server-advertised MSS are dropped in the ACE. This issue causes connections to stall and eventually time out.

CSCsy95865—When you execute the show np 1 me-stats -d command, the subsequent calls to show np fails.

CSCsz04613—The show np 1 me-stats -d and the show np 2 me-stats -d commands may have unexpected results on an ACE module.

CSCsz08089—When you configure the conn-limit and backup on a real server in a loop on the server farm, traffic stops getting load balanced through the ACE. The Xscale load-balance process (loadBalance_g_ns) shows 99 percent utilization.

CSCsz10384—The active and standby ACEs may have different configurations with the standby ACE showing the logging rate-limit configured but the active ACE may show that it is not configured. The standby ACE may remain in the standby_hot state.

CSCsz14803—The ACE reboots while processing internal statistics under normal conditions.

CSCsz15005—The configuration synchronization failed between the active and the standby ACE and the standby remained in the standby_hot state. This issue is observed in a CRL parameter configuration.

CSCsz16064—When you add one or more RADIUS servers to an AAA server group of type RADIUS, enable that server group for accounting, and then remove the servers from the group, when you remove the last server, the ACE stops functioning.

CSCsz20653—In a high traffic scenario, IXP halts with a core dump due to a deadlock between two micro engines in the data plane. This issue occurs when one micro engine is in the process of closing the connection when the other micro engine is internally processing a RST/FIN for the same connection.

CSCsz24484—The ACE reboots with a core file and there are existing core files of the same type on the disk.

CSCsz26513—When you transfer a large file, the ACE sends an encrypted alert to the client. Prior to this action, the ACE reduces its TCP window to zero, bumps up the size, receives the packet that it was acknowledging from the client, and sends the encrypted alert.

CSCsz28857—FP threads are all stuck because the Fast-tx received a RAW packet to be sent out that has a reference count of zero.

CSCsz29437—A crypto file import fails when a file is imported in a context with a name 64 characters in length, which is the maximum character length allowed in a context name.

CSCsz35051—An attempt to pass a configuration to xmlagent that is larger than 4kb causes the ACE to halt. This issue may prevent ANM from functioning properly.

CSCsz43769—When you run a script on the ACE that leaves telnet open, memory decreases and eventually causes the ACE to reboot. Workaround: Terminate the script.

CSCsz46264—The ACE stop functioning when a user attempts to log in after the only server in the RADIUS group is dynamically removed. Workaround: Do not remove the only server in a RADIUS group when the ACE is configured for RADIUS authentication.

Software Version A2(1.5) Open Caveats

The following open caveats apply to software version A2(1.5):

CSCse12120—When you press Ctrl-D and attempt to log in to the ACE with a valid username and password using the session command through EOBC from the supervisor engine, the login attempt fails. Workaround: Press Ctrl-D twice to access the switch login, and then log in.

CSCso93479—The Current Connections counter that is displayed in the output of the show serverfarm name command is not accurate. The output of the show service-policy command does have an accurate counter. Workaround: None.

CSCsr76812—When you configure the ACE with Layer 7 load balancing, TCP connection may be disrupted. Packets arrive at the client in reverse order or packets are forced to be resent. Workaround: None.

CSCsu88684, CSCsq27062—When you configure the ACE with a large number of contexts and enable redundancy, as traffic flows on the ACE, the ACE becomes unresponsive and displays the following messages on the console:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

The ACE then reboots. Workaround: None.

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv31046—When you configure the least-connections predictor on the ACE, the ACE may not sustain 160,000 CPS traffic. Workaround: None.

CSCsv54222—When an HTTP client sends pipelined requests, if the next request comes in the middle of the server response, the HTTP connection becomes unresponsive and data is missing on the web page. Workaround: Configure a connection parameter-map with the set tcp wan-optimization rtt 0 command.

CSCsv92321, CSCsx25981—The ACE module reboots unexpectedly and writes a core file to disk. Workaround: None.

CSCsx15558—When the ACE has over 120,000 concurrent SSL connections, it displays SSL connection rate denies, FastQ transmit backpressure, and SSL RX backpressure. Eventually, the ACE becomes unresponsive. Workaround: None

CSCsx19525—A buffer leak may occur after you use the show np 1 me-stats command. This issue occurs when you configure a large number of SSL VIPs (such as 1000 VIPs for the whole system) and you configure changes that affect those VIPs. Workaround: None

CSCsx41539—The ACE module may reboot and generate the following core files:

last boot reason: NP 0 Failed : NP Process Crashed
   182284  Feb 1 15:53:45 2009 qnx_1_mecore_log.999.tar.gz
   687601  Feb 1 15:53:41 2009 qnx_1_io-net_core_log.114693.tar.gz
   113726  Feb 1 15:53:47 2009 ixp1_crash.txt

Workaround: None.

CSCsy29181—If either of the DP processors is at MAXCONN, the ACE should show MAXCONN in the output of the show serverfarm command. However, the ACE waits until both network processors are at MAXCONN. This issue occurs when the cde-same-port-hash command is configured. Workaround: None.

CSCsy34814— The syslog message 305010 includes the duration of the Xlate translation. However this duration is always equal to the xlate idle timeout. Workaround: Use the timestamps in the creation and tear down of the xlate connections to calculate the xlate duration.

CSCsy65650—When the ACE reports the termination of TCP flows, it may display incorrect values for the duration and amount of data transferred. This issue occurs with HTTP and connections that are terminated with TCP RST. Workaround: None. If accounting is needed and relies on this log, use another method.

CSCsy88379—The TAC diagnostic script showtech generates large output due to the show xlate command. Workaround: None.

CSCsy90965—The Set-Cookie: length is null. Changing the default class-map from a sticky serverfarm to none does not eliminate the insertion of a cookie. Workaround: Remove and reconfigure the class class-default command .

CSCsy98701—The standby ACE generates a Load Balance core file when you configure two ACEs as FT pairs that are replicating sticky entries and you enter certain show commands on the active ACE. Workaround: None.

CSCsz10107—When preempt is enabled and the Catalyst 6500 with an active ACE module is reloaded, the ACE may not correctly replicate connections when it reboots and becomes active again. Some connections may get dropped. Workaround: None. This issue does not occur when reloading only the ACE or if preempt is not configured.

CSCsz14634—The ACE has issues when you copy large configurations from TFTP to the running-configuration and use the snmp-server community command to add the public group Network-Monitor to a context when the command was not in the original configuration. Workaround: None.

CSCsz18739—The ACE reloads when running software version A2(1.4) with RADIUS AAA configured. Workaround: None.

CSCsz19849—You cannot import an ACE VIP in WAF. Importing works in software version A2(1.2) and in A2(1.3). Workaround: None.

CSCsz28035—Access to the qnx shell from the physical console port of either NP on an ACE puts you in a shell. If you type exit, the NP console hangs and becomes inaccessible.

CSCsz34011—After a series of reboots, both ACE modules lose their context configurations. If the active ACE halts and reloads, after it reboots it will read the first half of the startup-config, establish FT with the standby ACE (the new active), and synchronized the configuration to obtain the rest of the configurations from the other ACE. If the other ACE stops functioning, the active ACE will not have obtained the rest of the configurations, including context configurations. Context configurations may be lost, although they still exist in the startup-config. Workaround: None.

CSCsz34933—The ACE may send a reset with sequence number zero for probe traffic for a probe configured with the connection term forced command. Workaround: Use the graceful termination no connection term command.

CSCsz40699—When you use the SLB-Admin, Server-Appln-Maintenance, or a custom role with a "create feature server farm" rule, you cannot bring real servers in or out of service under the server farm. Workaround: None. There are currently no workarounds using these specific roles. However, you can complete these tasks using the Admin role.

CSCsz49088—When you monitor the ACE CPU, you can only monitor it using an Admin role. The show processes cpu command is only available in the Admin role. The Network-Monitor role, which should have access to all show commands is unable to access the show processes cpu command. Configuring a new role on the ACE does not allow you to monitor the system feature. Therefore, only Admin users are able to run this command. Workaround: Run the show processes cpu command in an Admin role.

Software Version A2(1.5) Command Changes

Table 4 lists the command that has been changed in software version A2(1.5).

Table 4 CLI Commands Changed in Version A2(1.5)  

Mode
Command and Syntax
Description

Exec

show acl-merge {acls internal-vlan [vlan_id] {in | out} [summary]} | {match internal-vlan [vlan_id] {in | out} ip_address1 ip_address2 protocol src_port dest_port} | {merged-list internal-vlan [vlan_id] {in | out} [non-redundant | summary]}

The new internal-vlan keyword displays the ACL merge information for VLAN 1.

Exec

show parameter-map [name]

Per CSCsx75858, this command now displays the urlcookie-start field. This field displays one of the following:

The start string of the secondary cookie or the none setting configured by the set secondary-cookie-start command in parameter map HTTP configuration mode.

The default string of ?.

Exec

show serverfarm [name] [detail]

The fields displayed by this command now include the real server description field as defined by the description command in serverfarm host real server configuration mode.

Exec

show stats http

The TCP fin/rst msgs sent, Bounced fin/rst msgs sent, SSL fin/rst msgs sent fields have been expanded to the following fields:

TCP fin msgs sent

TCP rst msgs sent

Bounced fin msgs sent

Bounced rst msgs sent

SSL fin msgs sent

SSL rst msgs sent

Configuration

snmp-server unmask-community

no snmp-server unmask-community

The unmask-community keyword allows you to unmask the snmpCommunityName and snmpCommunitySecurityName OIDs of the SNMP-COMMUNITY-MIB. By default, they are masked. Use the no form of the command to mask them.

Class map configuration

[line_number] match ...

The line_number option now is an integer from 1 to 255. Previously, this option was an integer from 2 to 255.

Parameter map HTTP configuration

set secondary-cookie-start {none | text}

no set secondary-cookie-start

Per CSCsx75858, this new command either defines the ASCII-character string at the start of a secondary cookie in a URL or ignores any start string of a secondary cookie in the URL and considers the secondary cookie part of the URL.

The keyword and argument for this command are as follows:

none—The secondary cookie start is not configured or the ACE ignores any start string of a secondary cookie in the URL and considers the secondary cookie as part of the URL.

When you configure the none keyword to consider the entire URL query string as part of a URL, the commands that rely on the URL query, such as the match cookie secondary and predictor hash cookie secondary commands, do not work. Do not configure these commands under the same real server.

text—The start string of the secondary cookie. Enter a maximum of two characters. The default is ?.

Use the no form of this command to reset the start string to the default of ?.

Serverfarm host real server configuration

description text

The new description command allows you to provide a description for the real server in a server farm. Enter an unquoted text string with a maximum of 240 alphanumeric characters. If the text string includes spaces, enclose the string in quotes.


Revised System Log Messages

Software version A2(1.5) includes the following revised system log (syslog) messages.

253004

Error Message    %ACE-6-253004: Certificate subject_of_certificate revoked, ssl-proxy: 
proxy_name, reason: reason 

Explanation    This message is logged during the SSL handshake when client authentication is enabled. The ACE determines that the client certificate has been revoked by the CA. The subject_of_certificate variable is the subject field of the certificate. The proxy_name is the name of the SSL proxy service. The reason is the reason for the revocation of the certificate and has one of the following messages:

revoked—The certificate is revoked by the CA.

no workable cdps in cert—The certificate does not have a workable CRL distribution point (CDP). A CDP indicates the location of the CRL in the form of a URL.

crl download failure—The download of the CRL failed.

Recommended Action    None required.

441001

Error Message    %ACE-5-441001: Serverfarm (name) failed over to backupServerfarm 
(backup_name) in policy_map (lb_Policy_Map). Number of failovers = count1, number 
of times back in service = count2 

Explanation    A serverfarm failover event has occurred. The name variable is the name of the serverfarm. The backup_name is the name of the backup serverfarm. The lb_Policy_Map is the name of the load-balancing policy map. The count1 variable is the number of times that the primary serverfarm failed over to the backup serverfarm. The count2 variable is the number of times the primary serverfarm returned to service.

Recommended Action    None required.

441002

Error Message    %ACE-5-441002: Serverfarm (name) is now back in service in policy_map 
(lb_Policy_Map). Number of failovers = count1, number of times back in service = 
count2  

Explanation    A serverfarm in service event has occurred. The name variable is the name of the serverfarm. The lb_Policy_Map is the name of the load-balancing policy map. The count1 variable is the number of times that the primary serverfarm failed over to the backup serverfarm. The count2 variable is the number of times the primary serverfarm returned to service.

Recommended Action    None required.

Software Version A2(1.4a) Resolved Caveats, Open Caveats, and Command Changes

The following sections contain the resolved and open caveats, and command changes in software version A2(1.4a):

Software Version A2(1.4a) Resolved Caveats

Software Version A2(1.4a) Open Caveats

Software Version A2(1.4a) Command Changes

Software Version A2(1.4a) Resolved Caveats

The following resolved caveats apply to software version A2(1.4a):

CSCsy17648—Unable to log into the ACE with TACACS Authentication. You will receive the following error message: "Your account has expired; please contact your system administrator." Workaround: Delete the remote user account through admin user using the no username remote-username command.

CSCsy45802—A process on the Control Plane becomes unresponsive when show crypto files or show tech commands are executed. This issue occurs due to the implementation of the an internal function that removes some orphaned files from the crypto storage area.Workaround: Limiting execution of show crypto files and show tech will limit the risk of encountering this issue.

CSCsy77342—The ACE will not allow a slash ( \) to be used in a username when TACACS is configured.

CSCsx81954—When a GET spans two packets, the ACE may drop the second packet that requires the client to retransmit the packets.

CSCsy85870—Context 0: cmd exec error on standby ACE for the ssh key dsa 2048 force command. This occurs when the ACE is configured for FT and has the ssh key dsa 2048 force command in the configuration, and one or both of the FT peers are running A2(1.4). The key file is not synchronized to standby properly. Therefore, standby moves its state to standby-cold.

CSCsy91217—The show accounting log does not show accounting messages within each context. The Admin context shows logs for all contexts.

CSCsy91285—Using the last modifier on a command yields an error message, such as:

switch/Admin# show run | last 
Exec Error: : Bad address 
Generating configuration.... 

CSCsy95509—When the ACE is configured for TACACS and the username entered contains the "@" sign, the TACACS authentication fails.

Software Version A2(1.4a) Open Caveats

The open caveats that apply to software version A2(1.4a) are identical to the "Software Version A2(1.4) Open Caveats" list except for the caveats that have been resolved in A2(1.4a).

Software Version A2(1.4a) Command Changes

Table 5 lists the command that has been changed in software version A2(1.4a).

Table 5 CLI Commands Changed in Version A2(1.4a)  

Mode
Command and Syntax
Description

Configuration

username name1 ...

The name1 argument now supports the following non-alphanumeric characters:

- _ @ \

This argument does not support the following characters:

$ / ; ! #

Note Per CSCsy95433, the "." character is not supported on the local database but a username with this character is authenticated when it is configured on an ACS server.

Previously, this argument supported only alphanumeric characters.


Software Version A2(1.4) Resolved Caveats, Open Caveats, and Command Changes

The following sections contain the resolved and open caveats, and command changes in software version A2(1.4):

Software Version A2(1.4) Resolved Caveats

Software Version A2(1.4) Open Caveats

Software Version A2(1.4) Command Changes

Software Version A2(1.4) Resolved Caveats

The following resolved caveats apply to software version A2(1.4):

CSCsm57204—When a loopback IP address is configured, the expected IP address in a DNS probe configuration can be denied. Workaround: None.

CSCsj94366—When you attempt to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears:

console configuration can only be done on console

Workaround: None.

CSCso12560—The show resource usage command may display a nonzero number for some resources that have their maximum value set to equal-to-min. Workaround: None.

CSCso60304—When an invalid XML attribute is sent to the ACE, it does not respond as expected. Instead, the ACE displays a 500 Internal Server Error message. No negative impact to the ACE is observed. Workaround: None.

CSCso85236—When you enable persistence rebalance and connection reuse on the ACE, a subsequent request, other than the first request, over the same connection does not increment the current connection count for real server while it traverses the ACE and before the server response is received. Workaround: None.

CSCso85522, CSCsw78847—Changing the default password for the admin user in the Admin context causes the XML agent and the CLI to behave abnormally. Workaround: None.

CSCsq94865—When the ACE is 2 MB or over and the show run command is executed, XMLAGENT returns a 500 error. Workaround: You can still display separate parts of the configuration as long as the parts are under 2 MB. Also, you can still view the entire configuration from the terminal. The problem is limited to XMLAGENT.

CSCsq98541—When you change the request method for an RTSP probe from describe to options, the probes start to fail. The RTSP probes fail with the Server Reply Timeout error. Workaround: Remove the association of the probe from the real server or server farm and readd it.

CSCsr09129—When you configure SIP load balancing with inspection enabled, the ACE should open a pinhole to the address in the Via header for the server response. However, the server responses remain in the data channel. Workaround: None.

CSCsr94846—The radius keyword is deprecated and is now radius-auth for Remote Authentication Dial-in User Service (port 1812). Workaround: None.

CSCsu42225—When you configure the ACE with a Layer 4 load-balancing policy map and it receives a series of UDP requests with a payload of 3,200 bytes that spans three nonfragmented packets, the ACE drops two packets from the first request. For subsequent requests, the ACE load balances all packets successfully. Workaround: None.

CSCsu68314—When the ACE becomes unresponsive and generates a core dump, the core-dump file contains three different types of files. These files should be separate files. Workaround: Use the file command to uncompress the core-dump files.

CSCsv04319—If you create a TACACS+ server with a numeric key, the ACE sends a warning about the key; however, it does not create the server. The message should be an error and not a warning. Workaround: Use a key that is not entirely numeric.

CSCsv04848—When you configure RADIUS on the ACE and a user logs off, the RADIUS client sends an accounting stop message to the server for that user but the ACE does not immediately delete all connections for that user. If the source IP address for the user is immediately reassigned to another user, the new user could open a new connection before the old connections from previous user times out. The result is that the ACE incorrectly forwards the new connections and does not load balance the packets. Workaround: Set the UDP connection timer to a smaller number (for example, 10 seconds).

CSCsv10547—The config-register setting does not synchronize after an ACE module boots. The config-register setting synchronizes only when you configure it with ACE modules in active or standby mode. Workaround: None.

CSCsv31394, CSCsm46044, CSCsw80024—When you modify the policy-map configuration on an interface, the ACE occasionally records a service-policy download error. Workaround: None.

CSCsv31476—When the ACE generates a core-dump file for the kernel or Virtual Shell (VSH) applications, the file does not contain the code-train version information. Workaround: None.

CSCsv32122—The download of 16K source IP-address match statements can take 40 seconds. Workaround: None.

CSCsv33051—When you configure RADIUS load balancing and create a RADIUS-attribute sticky group with the sticky radius framed-ip command, if the Framed-IP-Address is reused and load balanced to a different rserver, the ACE may not update the sticky entry. Workaround: Configure the RADIUS client to issue Framed-IP-Addresses and include them in the RADIUS access request messages or configure separate Framed-IP-Address pools for each RADIUS real server.

CSCsv47724—The heartbeats on fault-tolerant (FT) ACE modules occasionally miss due to late TCP timers. The FT ACEs increment the Heartbeats Missed counter on the standby ACE and the Unidirectional HB's Received counter on the active ACE. Workaround: None.

CSCsv48498—When you enable FTP inspection and disable normalization on the client-side interface, the ACE inserts the TCP Option Timestamp in packets to the client and the FTP server, even if both the client and the server are not using this option. Workaround: Enable normalization or disable FTP inspection.

CSCsv52288—The ACE supports only 8K match source-address statements entries. Workaround: None.

CSCsv52331—The ACE becomes unresponsive due to an SRAM parity error. Workaround: None.

CSCsv52887—When an ACE with a large number of match source-address entries is under a high traffic load, modifying the match source-address entries may cause the console or terminal to lock briefly. Workaround: None.

CSCsv53187—The ACE generates an NP ha_hb_g_ns core dump during a standard operation. Workaround: None.

CSCsv56991—When you change the configuration of a real server on a server farm, the ACE does not replicate the connections. Workaround: None.

CSCsv59066—When using KAL-AP to report the VIP address status, all VIPs with the same addresses report a load of 255 if one is out of service. Workaround: Do not use KAL-AP to monitor multiple VIPs with the same IP addresses.

CSCsv63407—Issuing a show tech command can cause a redundant configuration to flap, especially if the command results in a lot of data to be fetched. Workaround: None.

CSCsv63786—When end-to-end SSL traffic is running at a high rate and you enter the show tech command, the ACE generates a core dump. Workaround: None.

CSCsv65178—When you specify TCP as the protocol in a class map configured for DNS traffic, the ACE allows the configuration and DNS inspection fails. Workaround: Specify UDP as the protocol in a class map configured for DNS traffic.

CSCsv69769—When you configure an expect regex value, the ACE allows a space in the quoted name of the value. Workaround: Do not use a space. Instead, use a search character (.*) or allow the variable to be on a long string input.

CSCsv79452—When the mapped VLAN ID is equal to the real server VLAN ID in the static nat command, you cannot delete the command. The ACE displays the following error:

Error: Bi-directional Static NAT config is not allowed

Workaround: None.

CSCsv83292—The ACE does not allow you to create user names that start with numeric characters. Also, it does not allow TACACS authentication of a TACACS user that consists of all numeric characters. Workaround: Start username with an alpha character.

CSCsv89719—When the ACE CLI is slow in responding, if you use the SIGQUIT or CTRL-\ key sequence to exit out of a command, the ACE generates a core dump. Workaround: Do not use the CTRL-\ or SIGQUIT sequence to exit out of slow responding CLI commands.

CSCsv89746— In the ACE 2(1.2) release, the logging rate-limit command adds an extra "1" in the running configuration, which causes the command to function incorrectly. Workaround: Do not use the logging rate-limit command.

CSCsv92091, CSCsx73626—When making an XML request over the XML interface to the ACE to modify the access list on an interface, the XML DTD file (documentation) does not match the expected input. The DTD says "name" and the ACE uses "access-name." Workaround: Use "access-name" to format the XML request.

CSCsv95254, CSCsv53112—When an IP address conflict occurs on a bridged VLAN, the ARP manager may become unresponsive which causes the ACE to generate a core dump. Workaround: None.

CSCsv96075—When using a client that sends some newer ciphers, such as ECC ciphers, in the ClientHello message, the ACE may select a cipher that does not match. Workaround: Remove unsupported ciphers from the clients cipher list.

CSCsv96914—A standby ACE sends an ICMP destination port unreachable in place of the heart beat (HB). The Active module sends HBs to the standby. Workaround: Either configure a query interface, or once in this state, remove the FT peer statements and add them back in to restart the HBs.

CSCsv97400—Using SNMP in a multicontext configuration can cause the ACE to reboot with the last reboot reason as service "snmpd". Workaround: Disable SNMP on the ACE.

CSCsw17457—If the service policy for the /32 VIP is associated to an interface first, and then you configure a static route, multiple routes are displayed in the FIB entry for the /32 VIP address. Workaround: Do not configure static routes to /32 VIP. The /32 VIP is considered to be owned by the ACE.

CSCsw19694—When the inactivity timeout is less than three minutes, the standby ACE has fewer connections than on the active ACE. This condition occurs because the standby ACE is clearing flows before getting the updates on the status of each established connection. Workaround: Increase the inactivity timeout greater than three minutes.

CSCsw19712—When the client or server closes a TCP-based To-CP (HTTP, HTTPS, Telnet) connection to the standby ACE interface through an RST with wrong sequence number, the standby ACE sends a TCP ACK with the virtual MAC address that belongs to the active ACE. Workaround: None.

CSCsw22221—When the ACE is configured with a backup server farm as a redirect type and a generic policy type, traffic to the virtual servers halts. Workaround: Change the policy type to load balance or use the nonredirect server farm as a backup.

CSCsw37439—When you change the configuration changes to expect IP addresses for a DNS probes associated to several real servers or server farms, health monitoring may crash with signal 11, segmentation fault. Workaround: None.

CSCsw39289—When you attempt to change to another context from the Admin context, the ACE does not allow it, and the ACE reboots several times and generates core dumps. Workaround: You can either Telnet to each context to make configuration changes or reboot the ACE.

CSCsw52831 —If a RADIUS packet is the second packet on a UDP connection and it is received shortly after the first RADIUS packet on the connection, it may be dropped. Workaround: None.

CSCsw57082—When a malformed DNS packet is sent as a response to the a DNS probe, health monitoring may crash with signal 11, segmentation fault. Workaround: None.

CSCsw63921—When you configure the ACE with a Layer 7 rule and persistence rebalance, it does not load balance a large Post packet correctly. The ACE sends half of the data to one server and the second half to another server within the default class. The show http stats command displays static parse errors. Workaround: Remove the persistent rebalance configuration.

CSCsw75536—The ACE may stop splicing TCP sequence numbers between the front-end and back-end connections of a load-balanced connection. Initially, the connection may operate with several successful HTTP transactions. However, the connection may eventually fail due to the ACE sending the TCP sequence numbers from the front-end connection to the back-end real server. Workaround: None.

CSCsw77807—SIP probes with random Call-IDs and From-Tags in the SIP options may fail with the Cisco Session Border Controller (SBC). The SBC responds with a SIP "482 Loop Detected" message because the same Call-Id and From-Tag are used in all requests. Workaround: Do not use SIP probes with Cisco SBC.

CSCsw83500—The show conn protocol tcp | inc CLSRST command displays a large number of connections. Workaround: Enter the clear flow command for all flows in the CLSRST state to free the buffers.

CSCsw86783—When the ACE is running SIP traffic or executing the show conn | inc in | inc EST | count command after stress traffic occurs on a server reuse connection, the ACE becomes unresponsive. Workaround: None.

CSCsw99769, CSCsz02078—Under some conditions with the A2(1.2) and A2(1.3) releases, when some QNX processes (such as ssl_Hs) receive an abort signal, the ACE may not create a set of core files and does not reboot. Instead, the ACE may become unresponsive and the core files may be incomplete or nonexistent. The behavior is different between NP1 and NP2. Workaround: Manually reboot the ACE.

CSCsx09418—When you configure the HSRP standby use-bia option on routers or the use of checkpoints, and outgoing connections flow through the ACE but are not directly connected, the ACE selects the LEARNED ARP entries of the physical interface for the connection Encap ID. The ACE should always do reverse route lookup and should select the ARP entries of the default gateway for the connection Encap ID. Workaround: On the upstream HSRP pair, use the virtual MAC address that is different from the physical interfaces.

CSCsx10212—When an active ACE has a single persistent TCP (http 1.1) connection that sends a series of nonpipelined GET requests to a Layer 7 VIP configured with rebalance, the connection is not replicated on the standby ACE. Workaround: None.

CSCsx10422When you configure persistence rebalance in a redundant ACE configuration, if no rebalancing is occurring and a single TCP (HTTP 1.1) connection sends a series of GET requests (not pipelined) to an Layer 7 VIP, each request counts as a total connection through the show serverfarm command on the standby ACE. The primary ACE correctly shows this as one connection. Workaround: None.

CSCsx33405—When you have more than 500 VLAN interfaces and hundreds of ACLs, the show tech command output is very large because of show acl-merge commands. This defect is an enhancement request to remove show acl-merge commands from the show tech command output and display a maximum of four VLANs. Workaround: Make sure that there is enough space on the compact flash.

CSCsx63421, CSCsx45782—On rare occasions, when you configure SSL on the ACE, it may reboot and generate the following core files:

ixp2_crash.txt
qnx_1_mecore_log.999.tar.gz
qnx_1_inspectHttp_g_ns_core_log.172052.tar.gz
qnx_2_mecore.999

Workaround: None.

Software Version A2(1.4) Open Caveats

The following open caveats apply to software version A2(1.4):

CSCse12120—When you press Ctrl-D and then attempt to log into the ACE with a valid username and password by using the session command through EOBC from the supervisor engine, the login attempt fails. Workaround: Press Ctrl-D twice to get to the actual "switch login" and then login.

CSCse14161—When the ACE has a large number of connections and you enter the pipe option with the show conn command, the ACE takes a long time to process the command and then display the results of the filter applied through the pipe. Workaround: Use the filters available through the show conn command itself. For example, instead of using the pipe option on the show conn command to find out the number of connections, use the show conn count command that displays the total connection count.

CSCsi61783—If you initially configure a real server as a Layer 2 real server, and then the interface goes down or is deleted from the configuration, the real server may transition to an ARP_FAILED state and remain in this state after it becomes a Layer 3 real server. Workaround: Reconfigure the real server.

CSCsj68643—The following log messages may appear sporadically in the ACE log:

can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.

can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.

These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl75662—You may observe that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time that the probe is sent even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from the real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsm65862—When you configure sticky on the ACE and high levels of Layer 7 traffic occurs, the show serverfarm command output may display connection entries even when there are no valid connections. Workaround: None.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform either of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, and then log in again to access a different context than the original context with the configured packet capture function.

CSCso38618—When you configure a large number of real servers and server farms on the ACE, the percentage of performance degradation varies upon the number of real servers and server farms on the ACE. The performance starts to drop more when the real server number increases from 64 to 256 which hits the cache limit of the ACE. Workaround: None.

CSCso81785—If you are using TACACS+ and the Cisco Access Control Server (ACS) with an RSA authentication manager, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another network access server (NAS) to set your PIN.

CSCso81811—If you are using TACACS+ and the Cisco ACS with an RSA authentication manager and your account is in next token mode, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another NAS to enter the next token code and make your account accessible again.

CSCso93479—The Current Connections counter that is displayed in the output of the show serverfarm name command is not accurate. The output of the show service-policy command does have an accurate counter. Workaround: None.

CSCsq14440—The aclmerged process in the ACE may not complete or may exceed the available system resources. With very large configurations where there are many ACLs, NAT statements, and class maps, the processing of these elements can require a significant amount of time and internal resources. In some cases, the processing (as displayed by the show proc cpu | include aclmerged command) may become unresponsive and never complete. In other cases, the processing may complete, but the output could exceed the resources available on the ACE, which may cause the ACE to not function properly.

Workaround and recovery: Currently, there is no method to predict the aclmerged response. However, in most cases, the commands eventually complete and the ACE continues to function properly. The suggested workaround is to allow aclmerged to complete without any intervention, assuming that there is no external impact to traffic. If the process does not complete or if there is a significant disruption to traffic flow, then reboot your ACE. If you enter the write memory command prior to the reboot, then the ACE attempts to come up in the post-change configuration. This may allow the desired configuration to be applied properly after the reboot. If you do not enter the write memory command before rebooting the ACE, then the ACE should reload and continue to operate in the same manner as before the change.

CSCsq27062—After toggling the state of the FT port channel in the Catalyst 6500 series switch 110 times, the primary ACE module generated a core dump and reloaded. Workaround: None.

CSCsr19340—When you configure authentication on Cisco ACS (TACACS or RADIUS), the user admin cannot log in the ACS for console authentication. Workaround: Configure a user role to Admin for the user admin on the ACS server.

CSCsr72591—When you need to import many SSL keys and certificates, it may take a long time (approximately 30 minutes to import 1000 keys and certificates). You must import them one at a time; there is no bulk import feature available. Workaround: None.

CSCsr73873—When you configure PAT on the ACE, if there is a very large amount of traffic, the show xlate command displays the following output, "Got no reply." Workaround: Reenter the command under a lighter load of traffic to display the desired output.

CSCsr76812—When you configure the ACE with Layer 7 load balancing, TCP connection may be disrupted. Packets arrive at the client in reverse order or packets are forced to be resent. Workaround: None.

CSCsu01728—SSL URL rewrite does not work when the server sends a location that is not exactly spelled "Location." Workaround: Configure a header replace function that exactly matches the field name sent by the server.

CSCsu31311—When an active ACE of a redundant pair attempts to open a connection to one of its real servers that is remote, the ACE sends a packet to the next hop. However, the next hop has no route to the real server and sends back ICMP 3 (unreachable) to the ACE. The ACE sends this packet back to the next hop even though the packet is destined to the ACE physical interface. The ICMP packet bounces back and forth until the TTL expires. Workaround: Configure a management policy that permits ICMP on the interface.

CSCsu67523, CSCsu67556, CSCsw68320—Upgrading the ACE software to version A2(1.1a) causes the ACE to reboot and generate a core dump. Workaround: None.

CSCsu67539—When you upgrade the ACE software to version A2(1.1), the ACE reboots and generates a core dump. Workaround: None.

CSCsu86606—When you reboot the ACE and as it powers up, the Catalyst 6500 series switch disables the ACE and displays the following log messages:

Oct  1 07:43:25.710 EDT: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off 
(Reset)
Oct  1 07:43:41.611 EDT: %OIR-SP-6-PWRFAILURE: Module 1 is being disabled due to power 
convertor failure 0x1 

Workaround: None.

CSCsu87573—When an ANM sends a CLI through the XML agent and it fails, the configuration count increases. Workaround: None.

CSCsu88684—When you configure the ACE with a large number of contexts and enable redundancy, as traffic flows on the ACE, the ACE becomes unresponsive and displays the following messages on the console:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

Then the ACE reboots. Workaround: None.

CSCsu95887—After the active ACE module completes configuration synchronization, it generates a core dump. Workaround: None.

CSCsu96977—When you configure more than 640 action lists and enter the do show action_list command with the Tab or ? key for help, the ACE becomes unresponsive. Workaround: None.

CSCsv02224, CSCsv52478—When you configure and remove an SSL-proxy service after you configure and remove multiple class maps under a policy map, the following error appears on the console:

Error: Called API encountered error appears console.

The ACE rejects the ssl-proxy command and the command does not appear in the configuration. Workaround: None.

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv31046—When you configure the least-connections predictor on the ACE, the ACE may not sustain 160,000 CPS traffic. Workaround: None.

CSCsv49606—When you configure stickiness on the ACE, the ACE becomes unresponsive. Workaround: None.

CSCsv52478—When you reboot the Catalyst 6500 series chassis, the ACE may reboot as Active. Workaround: None.

CSCsv54222—When an HTTP client sends pipelined requests, if the next request comes in the middle of the server response, the HTTP connection becomes unresponsive and data is missing on the web page. Workaround: Configure a connection parameter-map with the set tcp wan-optimization rtt 0 command.

CSCsv60332—When adding a new match statement to a class map, the cfgmgr sends duplicate line numbers to the ACL module. Workaround: Use a larger explicit line number when adding the statement.

CSCsv80430—RBAC on the ACE allows any user, regardless of the permission settings, to run any show command. Workaround: None.

CSCsv82638—When you use the XML interface to configure ip commands on the ACE, the INVALID_ATTR error is displayed. Workaround: None.

CSCsv82779—When you configure the deny function inside a management policy or class map, the ACE does not deny the traffic. Instead, the ACE skips the class and tries to match another one. Workaround: None.

CSCsv94951—When you configure virtualization on the ACE, and multiple user contexts exist where the aggregate of guaranteed resources (minimum X maximum equal-to-min) allocated to them is 100 percent and the Admin context is not a member of any resource class, the Admin context receives none of the system resources. As a result, the Admin context becomes unreachable through the network, cannot access outbound resources, and could cause FT host tracking failures, among other problems. Workaround: Do not allocate 100 percent of the resources to user contexts. Create a resource class for the Admin context that guarantees a minimum percentage of resources, make Admin a member of that class, and then make all user context resource allocations.

CSCsw34919—In a redundant configuration that has not been rebooted, when you add a context, the active or standby ACE includes the newly created context in SNMP traps; however, the peer does not include it. Workaround: Save the context and reboot the ACE.

CSCsw70487—Under normal conditions, the ACE unexpectedly generates a kernel core file. Workaround: None.

CSCsw80486—When an HTTP probe contains a URL with a space character and a bulk configuration synchronization is triggered, the operation fails and the standby ACE status changes to the standby_cold state. This condition affects bulk configuration syncs only (for example, after a switchover) and only when the configurations of the two ACE's are out of synchronization. If the two configurations are in synchronization (equal), then the bulk configuration synchronization will perform as expected and not fail. Also, when you initially configure the probe, it successfully synchronizations to the standby ACE during the incremental configuration sync. Workaround: A URL that contains a space may not be RFC 1738 compliant. If the space character is required, encode it as "%20".

CSCsw81300—When you configure the ACE with a combination of HTTP inspection and HTTP load-balance policy map with only a class-default class, server-connection reuse does not allow traffic. Workaround: Change the class map in the HTTP load-balance policy map from a class-default class map to a type HTTP load-balance class map.

CSCsw82591—When Layer 7 load-balanced UDP traffic that contains approximately 1,000 packets per second is sent to the ACE and the source and destination IP addresses and UDP port numbers are the same, the ACE may drop the traffic because of excessive internal buffer usage. Workaround: Either configure the client to use multiple UDP source ports or use Layer 4 load balancing.

CSCsw88171—When you make health monitoring changes, MTS data corruption occurs. The ACE reboots and generates a core file. Workaround: None.

CSCsw98274—When you add and remove the class map with the SSL proxy from a multi-match policy map multiple times, if you attempt to add a class map and then try to apply an SSL proxy, the "Error: Called API encountered error" message occurs and the proxy is not applied to the class map. Workaround: Do not add and remove the class map from a multi-match policy map too quickly. If this situation continues, reboot the ACE.

CSCsx15558—When the ACE has over 120,000 concurrent SSL connections, the ACE displays SSL connection rate denies, FastQ transmit backpressure, and SSL RX backpressure. Eventually, the ACE becomes unresponsive. Workaround: None.

CSCsx25981—Under normal conditions, the ACE becomes unresponsive because of an invalid buffer address. Workaround: None.

CSCsx38506—When you enable the display of raw XML request show command output in XML format on the ACE through the xml-show on command, the following policy maps are missing their XML closing tags:

policy-map type loadbalance generic

policy-map type loadbalance sip

policy-map type loadbalance rtsp

Workaround: None.

CSCsx41539—The ACE module may reboot and generate the following core files:

last boot reason: NP 0 Failed : NP Process Crashed
 
   182284  Feb 1 15:53:45 2009 qnx_1_mecore_log.999.tar.gz
   687601  Feb 1 15:53:41 2009 qnx_1_io-net_core_log.114693.tar.gz
   113726  Feb 1 15:53:47 2009 ixp1_crash.txt

Workaround: None.

CSCsx44351—When you enter the no inservice command on a real server in a PROBE-FAILED state, the state changes to the OUTOFSERVICE and the server becomes active and forwards packets for a few seconds. Workaround: None.

CSCsx53491—When a bad SSL packet causes the ACE to reset the SSL handshake, the ACE does not issue a fatal alert for ERR_SSL_MAC_MISCOMPARE in response to the bad SSL packet. Workaround: None.

CSCsx57861—When the standby ACE receives an ACK from a bad client, it sends an ACK that contains a virtual MAC address (VMAC). Workaround: None.

CSCsx67908—When ACEs are configured for redundancy and Route Health Injection (RHI) and the standby ACE reboots, duplicate RHI entries can exist on the supervisor (SUP). Workaround: Enter the global ft switchover force command to properly update the RHI routes on the supervisor.

CSCsx72444—When you configure a syslog over TCP to send messages to a server and the server closes the connection due to a failure or a restart, the ACE closes its own socket. When the ACE closes the socket, it never tries to reopen it and no more messages are sent. Workaround: Remove and reenter the syslog host configuration or use a syslog over a UDP configuration.

CSCsx73473—When you configure the ACE to operate with a primary and secondary RADIUS server and the primary RADIUS server is down, the RADIUS Access-Request has a duplicate attribute pair (NAS-IP-Address) that causes the RADIUS authentication to the secondary RADIUS server to fail. Workaround: Configure the secondary RADIUS server as the first one in the RADIUS group list.

CSCsx80946—After a redundancy switchover, repeatedly running the show mac-address-table | inc 000b.fcfe.1b command on the Catalyst 6500 series switch displays entries in the ACE FT VLAN. The ACE virtual MAC address is of the form 000b.fcfe.1bXX. Workaround: None.

CSCsx80991—When you configure a real server in a server farm with least connections (leastconns) without the slowstart option, the ACE stops using it when you add real servers to the server farm. Workaround: Configure either roundrobin or slowstart.

CSCsx81954—When a GET spans two packets, the ACE may drop the second packet that requires the client to retransmit the packets. Workaround: None.

CSCsy17648—Unable to log into the ACE with TACACS Authentication. You will receive the following error message: "Your account has expired; please contact your system administrator." Workaround: Delete the remote user account through admin user using the no username remote-username command.

CSCsy45802—A process on the Control Plane becomes unresponsive when show crypto files or show tech commands are executed. This issue occurs due to the implementation of the an internal function that removes some orphaned files from the crypto storage area.Workaround: Limiting execution of show crypto files and show tech will limit the risk of encountering this issue.

CSCsy77342—The ACE will not allow a slash ( \) to be used in a username when TACACS is configured.

CSCsx81954—When a GET spans two packets, the ACE may drop the second packet that requires the client to retransmit the packets.

CSCsy85870—Context 0: cmd exec error on standby ACE for the ssh key dsa 2048 force command. This occurs when the ACE is configured for FT and has the ssh key dsa 2048 force command in the configuration, and one or both of the FT peers are running A2(1.4).

CSCsy91217—The show accounting log does not show accounting messages within each context. The Admin context shows logs for all contexts.

CSCsy91285—Using the last modifier on a command yields an error message, such as:

switch/Admin# show run | last 
Exec Error: : Bad address 
Generating configuration.... 

CSCsy95509—When the ACE is configured for TACACS and the username entered contains the "@" sign, the TACACS authentication fails.

Software Version A2(1.4) Command Changes

Table 6 lists the commands and options that have been changed in software version A2(1.4).

Table 6 CLI Commands Changed in Version A2(1.4)  

Mode
Command and Syntax
Description

Exec

crypto crlparams crl_name cacert ca_cert_filename

no crypto crlparams crl_name

Configures signature verification on a CRL to determine that it is from a trusted certificate authority (CA). The arguments are as follows:

crl_name— Name of an existing CRL.

ca_cert_filename— Name of the CA certificate file used for signature verification.

Use the no version of this command to remove signature verification from the CRL.

Exec

show conn [{address ip_address1 [ip_address2] netmask mask [detail]}
| count | detail | {port number1 [number2] [detail]} | {protocol {tcp | udp} [detail]} | {rserver rs_name [port_number serverfarm sfarm_name1 | serverfarm sfarm_name1] [detail]} | {serverfarm sfarm_name2 [detail]}]

Per CSCsg75273, the detail option has been added for a specified address, port, protocol, real server, or server farm. This option displays additional information for the connection including idle time, elapsed time, byte count, packet count, and, if applicable, the state of the connection in the reuse pool.

Exec

show crypto cdp-errors

The new cdp-errors keyword displays the statistics for discrepancies in CRL Distribution Points (CDPs) for the certificates on the ACE; not context specific. A CDP indicates the location of the CRL in the form of a URL. CDP parsing in the certificate occurs only when best effort CRL is in use.

The output for this command includes the following fields:

Incomplete—Number of times that the CDPs are missing information required to download the CRLs, for example, host, file name or base information.

Unrecognized Transports—Number of times that the ACE does not recognize or support the transport mechanism in the CDP for the CRL.

Malformed—Number of times that the CDPs are malformed with erroneous information, for example, specifying an incorrect attribute or base information. This counter also includes CDPs with URL lengths exceeding the ACE limit of 255 characters; a truncated URL could point to the wrong CRL.

Missing from cert—Number of times that the CDPs are missing from the certificate.

Exec

show crypto crl name detail

The new detail keyword displays additional statistics for CRL download failures. For information on the fields for this command, see the "Displaying Detailed CRL-Downloading Statistics" section.

Exec

show ft config-error [context_name]

In a redundant configuration, the new config-error keyword displays the commands that fail on the standby ACE during bulk synchronization. If all commands succeed on the standby ACE, the command displays the following message:

No bulk config apply errors

In the Admin context, the optional context_name argument is the name of a user context. If you do not enter the argument, the command uses the Admin context. In a user context, this argument is not available.

Exec

show sticky cookie-insert group sticky_group_name

The new show sticky cookie-insert command displays information that correlates the inserted cookie, the sticky entry, and the final destination for the cookie insert configuration.The output for this command includes the following fields:

Cookie—Cookie-insert hash string for each real server in the associated server farm.

HashKey—64-bit hash value associated with the cookie.

rserver-instance—String containing the server-farm name, real-server name, and real-server port in the following format:

server_farm_name/real_server_name:rserver_port

Exec

show sticky database static | i never

The "| i never" modifier filters the show sticky database static command for the "never" time-to-expire flag.

Exec

show sticky database static http-cookie cookie_value

This command no longer displays the hash key.

Exec

show tech-support

Per CSCsx33405, this command no longer displays the following:

All show acl-merge acls vlan command output

All show acl-merge merge-list vlan number out command output

It also now displays a maximum of four VLANs.

Object group

udp operator radius-auth ...

Per CSCsr94846, the radius keyword is deprecated and is now radius-auth for Remote Authentication Dial-in User Service (port 1812).

Server farm

predictor hash cookie secondary cookie_name

The new secondary keyword selects the server by using the hash value based on the specified name in the cookie name in the URL query string, not the cookie header.

For the cookie_name argument, enter a cookie name as an unquoted text string with no spaces and a maximum of 64 alphanumeric characters.

For example, consider the following request:

GET /index.html?TEST=test123
Cookie: TEST=456

If you configure the predictor hash cookie secondary TEST command, it selects the server using the hash value based on test123. If you configure the predictor hash cookie TEST command, it selects the server using the hash value based on test456.

This option allows the ACE to correctly load balance in cases when the query string identifies the actual resource, instead of the URL. In the following example, if the ACE hashes on the URL, it would load balance on the same real server:

http://youtube.com/watch?v=C16mk4OfcuM
http://youtube.com/watch?v=cJ3jPzs2NLk


Displaying Detailed CRL-Downloading Statistics

To display the detailed statistics for the downloading of a CRL including failure counters, use the show crypto crl name detail command. Table 7 describes the fields displayed by this command.

Table 7 Field Descriptions for the show crypto crl crl_name detail Command 

Field
Description

URL

URL where the ACE downloads the CRL.

Last Downloaded

Last time the ACE downloaded the CRL. If the CRL is configured on an SSL-proxy service on a policy map that is not active or the service is not associated with a policy map, the field displays the "not downloaded yet" message.

Total Number of Download Attempts

Number of times the ACE attempted to download the CRL.

Failed Download Attempts

Numbers of times that the ACE failed to download the CRL.

Successful Loads

Number of times that the ACE successfully loaded the CRL.

Failed Loads

Number of times that the ACE could not load the CRL because of a failure.

Hours since Last Load

Number of hours that elapsed since the ACE last successfully downloaded the CRL. If no successful download has occurred, this field displays NA, not applicable.

No IP Addr Resolutions

Number of times the DNS resolution for the server host address of CRL the failed.

Host Timeouts

Number of download retries to the CRL that had timed out.

Next Update Invalid

Number of times that the next update field of the CRL was invalid.

Next Update Expired

Number of times that the next update field of the CRL was expired.

Bad Signature

Number of times that the signature mismatch for the CRL was detected, with respect to the CA certificate configured for signature verification of the CRL.

CRL Found-Failed to load

Number of times that the ACE could not load the CRL because of the maximum size limitation of 10MB on ACE or the formatting of the CRL was not recognized. The ACE recognizes only DER and PEM encoded CRLs.

File Not Found

Number of times that the server responded that the CRL file was not found at the server.

Memory Outage failures

Number of times that the ACE failed to download the CRL because it temporarily could not provide memory to store the CRL data.

Cache Limit failures

Number of times that the ACE could not load the CRL because the CRL cache was exhausted.

Conn Failures

Number of times that the ACE failed to download the CRL because it could not establish a connection with the server or no server entity was listening on the destination system.

Internal Failures

Number of internal failures in the ACE that hampered downloading the CRL, for example, internal communication failures between components responsible for the downloading the CRL.

Not Eligible for download

Number of times that the CRL was found ineligible for downloading because the following conditions:

The downloading of the same CRL is in progress.

The CRL has already been loaded successfully earlier and has not expired yet.

HTTP Read Failures

Number of times that the ACE encountered an error when downloading the CRL because it could not read data on the connection established with server.

HTTP Write failures

Number of times that the ACE encountered an error when downloading the CRL because it could not write the CRL download request from the connection established with the server.


System Log Messages

Software version A2(1.4) introduces the following new or revised system log (syslog) messages.

New syslog Message

253011

Error Message    %ACE-6-253011: The CRL crl_Name may not be from a trusted source. 
Signature mismatch detected for CRL. 

Explanation    When the ACE performs signature verification on a CRL with a CA certificate configured with the crypto crlparams command, it detects a signature mismatch. Either the CRL (crl_name) download failed or the CRL has been removed from the ACE.

Recommended Action    Verify the CRL configuration for the crypto crlparams command.

Revised syslog Message

253006

Error Message    %ACE-6-253006: Error peer sent invalid or nonexistent certificate 
subject_of_peer_certificate, reason: reason 

Explanation    This message is logged during the SSL handshake when client authentication is enabled. The ACE determines a certificate is invalid or nonexistent. The subject_of_peer_certificate variable is the subject field of the peer certificate. The reason variable is the reason for rejecting the certificate and can be one of the following messages:

bad modulus length

error in not before field

error in not after field

Recommended Action    None required.

253008

Error Message    %ACE-6-253008: CRL crl_name could not be retrieved, reason: reason 

Explanation    This message is logged when the ACE failed to retrieve a CRL. If you define CRL checking for SSL client authentication, the ACE periodically retrieves a CRL. Due to a variety of reasons, these attempts can occasionally fail. The crl_name variable is the name of the CRL as defined by the crypto crl command. The reason variable is the reason for the CRL download failure. and can be one of the following messages:

DNS error

host conn timeout

memory outage

crl max size limit violation

crl cache full

crl data/file not found

invalid format of data

crl signature mismatch

next update field erroneous

next update field expired

internal error

not okay to download

http connection error

http file read error

http request writing error

ldap bind error

ldap search error

Recommended Action    Check to see if there is a network connectivity problem or if the server location of the CRL has changed.

Software Version A2(1.3) Resolved Caveats, Open Caveats, and Command Changes

The following sections contain the resolved and open caveats, and command changes in software version A2(1.3):

Software Version A2(1.3) Resolved Caveats

Software Version A2(1.3) Open Caveats

Software Version A2(1.3) Command Changes

Software Version A2(1.3) Resolved Caveats

The following resolved caveats apply to software version A2(1.3):

CSCsk57007—When you use the predictor leastconns command to configure the ACE to select the server with the fewest number of connections based on the server weight, the weighted ConnCount becomes corrupted if you do not set the minimum and maximum connection values to equal values and the real server reaches its maximum connection state. Workaround: Either make the minimum and maximum connection values equal or remove the maximum connections configuration parameter.

CSCsk58027—The ACE does not allow the configuration of cyclic backup real servers. This configuration can restrict its use in a firewall load-balancing application that requires the ability to have a single level of cyclic recursion. For example, a single level of cyclic recursion would allow FW1 to be the FW2 backup, and FW2 to be the FW1 backup. Workaround: None.

CSCsl88669—When you configure load balancing with a small maximum connections limit and the minimum and maximum connection values are equal, if there is a high amount of traffic that causes the servers to rapidly transition in and out of the maximum connection state, the ACE may become unresponsive. Workaround: None.

CSCsm57955—When you configure the sticky timeout for three minutes, after the ACE removes the connections, the sticky database does not clear. Workaround: Use the clear sticky database all command to clear the sticky database.

CSCso00234—After the ClientHello and the ServerHello, the ACE responds to the client with the Fatal, description:Bad Record MAC alert. Currently, the ACE cannot process non-minimally padded block ciphers, which is a TLS 1.0 feature. You employ non-minimally padded block ciphers in the following situations:

You use TLS version 1.0.

You negotiate a block cipher (AES256).

The Finished message is 256 bytes.

Workaround: If possible, restrict the SSL protocol version to SSL version 3. Alternatively, allow only stream ciphers, such as RC4.

CSCso19129—When you configure the ACE and enable load balancing or server-side NAT and RTSP inspection, the Windows Media Server may reject an RTSP session if you are using Windows Media player 10. If a real server IP address to VIP (or vice versa) translation is required, the ACE translates the IP address in the SDP part but does not update the content length in the header part. The message is then rejected by the server. This behavior does not occur if the real address length and VIP address length are the same. If the IP address length does not change between the VIP (mapped address) and the real server (real address), then this behavior does not occur. Workaround: None.

CSCso21587—When you enable RTSP inspection and the ACE performs load balancing or destination NAT on an RTSP session, the Windows Media Server may reject the session. If the media stream is interleaved with the RTSP control connection (the transport type is RTP/TCP or RDT/TCP), then the ACE incorrectly unproxies the control connection as soon as it detects the transport type as TCP which causes the rest of the messages between the client and the server to pass through the ACE without any inspection. As a result, subsequent SETUP messages are not fixed (NATed) and the server rejects the SETUP message with the VIP address (instead of the real server address) in it. Workaround: Use UDP instead of TCP as the transport mode for the media streams.

CSCso33550—When you configure the ACE for client authentication with certificate revocation list (CRL) checking, it does not verify that the downloaded CRL loaded was signed by a trusted certificate authority (CA). This behavior allows CRL substitution and bypasses the CRL check. Workaround: None.

CSCso47783—When you configure the ACE for NAT and you are using the NAT counters for troubleshooting, the NAT failure counter does not provide enough granularity for all cases that may cause the counter to increment. Workaround: None.

CSCso69044—With SYN cookie enabled, embryonic connections (incomplete TCP handshakes) may remain on the ACE after more than 24 hours. Workaround: None.

CSCso80600—When the ACE sends an HTTP probe and receives a 404 error message or when the probe time interval is exceeded, the probe does not fail. Workaround: None.

CSCsq34204—When the match criteria of a match-any class map consists of multiple VIP match statements and you try to remove the first VIP statement by using the line number method or entering the entire no match virtual-address command, the ACE does not remove it from the class map but removes it from the cfgmgr internal VIP table. The ACE becomes unresponsive and generates a core dump. You can successfully remove subsequent VIP match statements. Workaround: None.

CSCsq38638—When the ACE performs an SRAM operation and detects an SRAM parity error, it reboots and generate a core dump. This condition may cause other ACE operations, such as the IFMGR, to fail. Workaround: None.

CSCsq69818—When you configure the same connection rate limit for a real server at both the real server level and server farm level, its connections fail. Workaround: Manually take the real server out of service and then place it in service.

CSCsq97246—When you configure the ACE with a large number of real servers and then enter the show rserver details command, the ACE generates a Virtual Shell (VSH) core file. Workaround: Use the nonverbose show rserver command.

CSCsr00851—A user with the configured RBAC user role of Network-Monitor is allowed to delete other users' directories on the ACE Flash memory. Workaround: Do not configure users with the Network-Monitor role.

CSCsr16179—When system logging is enabled with the logging fastpath command, IP addresses in the Built TCP Connection syslog messages may be incorrectly swapped. Workaround: None.

CSCsr16201—Built TCP Connection and Teardown TCP Connection syslog messages may continue to be sent to a syslog server even after disabling this functionality with the no logging fastpath command. Workaround: Set logging trap 4.

CSCsr28182—When a class-map any command is combined with a class-map all command and more than ten header matches of the same type are used, the ACE CLI displays the following error message:

Error: Maximum 10 http header map is allowed per policy!

Workaround: Decrease the number of header matches to 10 or less by using regular expressions (regexes).

CSCsr38682—The ACE CLI allows you to configure overlapping IP addresses for both an alias IP address and a VIP address. When you add a service policy to an interface with the overlapping VIP and alias IP addresses, the ACE displays the following error message:

Error: vip address duplicates with an existing interface ip address! 

However, if you remove the alias IP address and add the service policy to the interface, and then reconfigure the same alias IP address, the ACE allows the configuration. Also, when you reboot the ACE with this configuration, you receive the "*** cmd exec error ***" parser error and the ACE removes the service policy from the interface. Workaround: Ensure that the VIP address and the alias IP address are unique within the context.

CSCsr43445—LbInspectTool displays an incorrect default value of 2147483648 for the conn-limit max conn command. The CLI and the A2(1.0) Cisco Application Control Engine Module Server Load-Balancing Configuration Guide both show the correct default value of 4294967295. Workaround: None.

CSCsr50367—When you configure multiple contexts with the same connection and bandwidth rate limits on the parent real servers, if traffic is running and the configuration changes, the ACE becomes unresponsive. Workaround: None.

CSCsr57510—When you configure a VIP with a subnet mask of 255.255.255.255 and you configure a policy map to forward traffic, the ACE may drop the packets because of a route lookup failure. Workaround: Configure a class map to match on the destination address.

CSCsr62027—When TCP normalization is disabled, the ACE places replicated TCP connections in the INIT state on the standby ACE. After the normal embryonic connection timeout occurs, the ACE removes the replicated connections from the standby. Workaround: Do not disable normalization.

CSCsr67565—When you create a Certificate Signing Request (CSR) parameter set on the ACE, the CLI does not allow special characters, for example, a comma (,) or a period (.), in the following CSR fields:

State

Locality

Organization-name

Organization-unit

Serial-number

This behavior occurs only in software releases starting with version A2(1.1). Previous releases allow these characters in the CSR fields. Workaround: Use a previous release to generate a CSR.

CSCsr75832—When you modify the configuration of the ACE, the module may classify traffic using the wrong class map and, consequently, forward traffic to the wrong server farm. Workaround: Wait for a few seconds for the modified configuration to take effect.

CSCsr81482—When upgrading the ACE from A1(6.3b) to A2(1.1a), the ACE generates a core dump file in the system manager application. Workaround: None.

CSCsr87665—When you configure a real server on the ACE using the same IP address used by a gateway for the ACE, load balancing stops functioning and the Encap ID displays 0. This condition prevents the ACE configuration manager from downloading the address to the data plane (DP) and creating an Encap ID because the IP address shows as a default route. Workaround: None.

CSCsr89398—The ACE becomes unresponsive and generates exceptionally large ME CORE files that are not formatted correctly and contains repeated statistics. The analyzed core files produce very large output files. Workaround: None.

CSCsr98689—When the ACE operates in Port Address Translation (PAT) mode and DNS servers are deployed behind it, the DNS infrastructure may be at risk because of a known DNS problem in which the fixes to DNS implementations to use random source ports when sending DNS queries may be negated when such queries traverse PAT devices.

After the initial multi-vendor DNS advisory was published on July 8th, 2008, it was discovered that in some cases, the fixes to DNS implementations to use random source ports when sending DNS queries could be negated when such queries traverse PAT devices. In these cases, the network device that performs PAT uses a predictable source port allocation policy, such as incremental allocation, when performing the Layer 4 rewrite operation that is necessary for PAT. Under this scenario, the fixes made by DNS vendors can be greatly diminished because although the DNS queries seen on the inside network have random source port numbers, the same queries have potentially predictable source port numbers when they leave the private network, depending on the type of traffic that transits through the device.

The ACE is affected by this issue. Although the ACE does not use an incremental source port allocation policy, it uses a hash algorithm that may make the source port for a specific destination port predictable.


Note Traditional NAT (for example, allocating one public IP address for each private IP address) is not affected by this problem because, unlike PAT, NAT only rewrites Layer 3 information and does not modify Layer 4 header information of the packets that traverse the NAT device.


For more information about the DNS vulnerability, refer to the multivendor advisory at:

http://www.kb.cert.org/vuls/id/800113

or at the Cisco-specific advisory at:

http://www.cisco.com/warp/public/707/cisco-sa-20080708-dns.shtml

CSCsu02242—When you enter the show service-policy detail command, the ACE may generate a core dump. Workaround: None.

CSCsu04042—The ACE HTTP XML management access fails with the following error: Authorization Required. Workaround: None.

CSCsu29977—When the active ACE transitions over to the standby ACE after losing an interface link, it removes all connection information that relates to the lost link interface from the connection table. When a failover recovery occurs and the standby ACE becomes the active ACE again, all client connections using that interface need to reconnect. Workaround: None.

CSCsu37177—When you configure the ACE to accept an SSH connection and to authenticate virtual users using TACACS+ or RADIUS, an SSH client that uses password authentication as the default authentication type cannot connect to the ACE unless the same user has logged in and been authenticated by TACACS+ or RADIUS. To see if a user has logged in and been authenticated using TACACS+ or RADIUS, use the show user-database command. The SSH clients that use password authentication by default are F-SECURE, SecureCRT, and Cisco Works. Workarounds: Use an SSH client that defaults to Keyboard Interactive Authentication or configure your SSH client to use Keyboard Interactive Authentication. Connect to the ACE using an SSH client that uses Keyboard Authentication by default or a Telnet client and be authenticated using TACACS+ or RADIUS. Ensure that the user is shown as a remote user in the output of the show user-database command. Define the username to be authenticated using TACACS+ or RADIUS locally on the ACE.

CSCsu37501—When you use a terminal to import PEM-encoded SSL certificates and keys with line wrapping that is greater than 70 characters, the ACE fails to install the root certificate and issues an error message stating that the input string is too long. Workarounds: Before you import unwrapped certificates and keys or certificates and keys wrapped to an unusual length through a terminal, manually wrap them to a width similar to that produced by OpenSSL by default. Alternatively, import them by means other than a terminal (for example, SFTP, TFTP).

CSCsu45248—When you use the loadbalance vip icmp-reply active command to enable a VIP to reply to ICMP requests on a policy, the ACE responds to ICMP messages destined to a real server even when the server is out of service. This behavior occurs when you use the same VIP under multiple polices, but you do not have the same loadbalance icmp-reply active command enabled on all the policies. Workaround: Use a different VIP for each class map or policy.

CSCsu47532—When you configure the ACE for redundancy and sticky, the active ACE replicates connections to the standby ACE due to sticky replication. This behavior causes the standby ACE to become unresponsive. Workaround: None.

CSCsu51821—When two or more policy maps use the same sticky group and you delete one of the policy maps, the ACE clears the sticky database. Workaround: None.

CSCsu51920—When you paste copied information into a class map configuration, the pasted information merges with the existing information instead of creating a new line. Workaround: None.

CSCsu54509—When you configure the ACE for redundancy and a custom role rule on the active ACE with the rule 2 permit create feature fault-tolerant command, the ACE writes it into the configuration as rule 2 permit create fault-tolerance. This behavior causes the standby ACE to reject the command during a bulk synchronization and remain in a standby_cold state. Workaround: Enter the no ft auto-sync run command on the active ACE and manually configure the role rule on the standby ACE.

CSCsu55180, CSCsv02360—When you configure the ACE for reuse and the MTU is any value other than 1460 in either the client, server, or both interfaces, the traffic does not reuse an existing server connection. Workaround: Configure the interface with reuse and the default MTU value of 1460.

CSCsu55935—A TCP probe fails because the server terminates the connection with a FIN immediately after the three-way handshake. Workaround: None.

CSCsu56682—A TCP probe fails because the server terminates the connection with an RST immediately after the three-way handshake. Workaround: None.

CSCsu59116—When the ACE encounters a segmentation fault during standard operations, it generates a core dump and reboots. After the reboot, the last boot reason is "Unknown" and the output of the dir core: command contains a sysmgr_log.858.tar.gz file with a timestamp that corresponds to the reboot time. Workaround: None.

CSCsu64736—When you configure the rate-limit connection command on a real server and the number of connections exceeds the configured maximum value, the server may remain in the MAXCONNS state. Workaround: None.

CSCsu67719—When you configure a class map VIP address as a range of IP addresses and the range overlaps with any of the network interface IP address configurations, the ACE does not reject the configuration. This condition causes the ACE to load balance the interface connections. Workaround: When configuring VIP addresses, do not overlap IP addresses.

CSCsu69544—When you reconfigure the request method of an existing HTTP probe, the ACE configuration manager (cfgmgr) receives a signal 11 and the ACE becomes unresponsive. Workaround: None.

CSCsu71822—In Layer 7 policy maps, the ACE drops packets that exceed the maximum segment size (MSS) even when a parameter map explicitly allows them. You can see the drop packets with the following command.

switch/Admin# show np 1 me-stats "-stcp" | i MSS
Drops due to packet size exceed MSS:              5             0
switch/Admin#

Workaround: None.

CSCsu73506—When you use the username name password 5 password command, the 5 option specifies an MD5-hashed strong encryption password (password) and it must be 16 bytes. However, if it is not 16 bytes, the ACE still accepts it. When you change the Admin password and mistakenly enter a clear-text password instead of the MD5-hash password, you can lock yourself out of the ACE. Workaround: Perform a password recovery.

CSCsu74351—When a client attempts to access an SSL (HTTPS) web page over a remote WAN link, the client sends a Bad Record MAC fatal alert due to TCP retransmissions. Workaround: Reload the page.

CSCsu78560—After you use the rate-limit connection command to change the limit on the connection rate for a real server, the ACE may drop packets and may not establish connections. The hit counter in the output of the show service-policy detail command does not increase, but dropped connections may increase. Workaround: Stop the traffic, and then enter the no inservice command followed by the inservice command on the real server.

CSCsu83647—When you configure FTP inspection on the ACE, the ACE reboots and generates a core dump. Workaround: None.

CSCsu84998—When you configure connection limits on the real server and the leastconns predictor on the server farm, if heavy traffic enters the ACE, the real server may remain in the MAXCONNS state when the number of concurrent connections has dropped below the conn-limit min value. Workaround: Remove the real server from the server farm and then add it back into the server farm.

CSCsu86686, CSCsu58683—When you change a large configuration on the ACE, one or more VIP addresses on the ACE stop taking connections because the VIP address that fails is using a stale virtual server ID. Workaround: Remove and readd the service policy. If that does not resolve the problem, then remove and readd the VLAN interface configuration.

CSCsu87044—When the ACE runs software version A2.1.2, it accesses invalid memory and generates an SNMPD core dump. Workaround: None.

CSCsu87321—The out-of-rotation-count counter in the show serverfarm detail command output indicates the number of times that a real server has reached the MAXCONNS state. In some scenarios involving heavy traffic, this counter increments incorrectly and does not reflect the actual number of times that the real server entered the MAXCONNS state. This behavior does not affect service, but the counter does not reflect the correct status. Workaround: None.

CSCsu87844—After the peer closes a connection, the connection remains persistent on the ACE. The connection clears after the inactivity timeout occurs. Workaround: Reduce the inactivity timeout.

CSCsu87852—When you configure an SSL probe to hit a non-SSL service port, such as HTTP, the buffer memory may leak to the point that the ACE denies connections. This leak is temporary. The memory is recovered if the SSL context state is reused. These states are last-in-first-out. All 100K connections must be used to release the memory. Workaround: None.

CSCsu87863—If a client sends a ClientHello message followed by a corrupt or non-SSL record the ACE, the ACE does not release the buffer memory until the next use of the SSL state and may cause the denial of new connections. These states are last-in-first-out, thus all 100K connections must be used to release the memory. Workaround: None.

CSCsu88070—When you configure the failaction purge command in a server farm, if a real server in that farm is in the MAXCONNS state and the probe to that real server fails, this command does not clean up connections to the real server. However, the failaction purge command works properly if the real server is not in the MAXCONNS state. Workaround: None.

CSCsu89251—After you delete a certificate and key, if you attempt to see the details of the chain group or certificate, the ACE becomes unresponsive. Workaround: None.

CSCsu89261—When you enable both the SSL session reuse and authentication group, after a full handshake is completed, any change in the authentication group does not clean up the existing SSL session cache. The client can continue to reuse the same session ID without performing a full handshake with the new authentication group configuration. Workaround: None.

CSCsu90625—When you configure the failaction purge command in a server farm, if a real server in that farm is in the MAXCONNS state and the associated real servers for that real server are in the same farm, and then the real server enters the OUTOFSERVICE state, the ACE does not purge the connections. Existing connections to the real server are left to complete or time out on their own and the ACE does not reset them. Workaround: Manually clear the connections using the clear conn rserver name command.

CSCsu91422—When there is more than one real server in a server farm and you set the conn-limit min command value to 1 and the conn-limit max command value equal to the rate-limit connection command value on one of the real servers, that server does not receive any connections. Workaround: Enter the no inservice command followed by the inservice command.

CSCsu94919—When you configure the ACE for logging through the logging enable and logging console 7 commands, if you enter the no logging message number command, the messages stop as expected. However, if you reconfigure logging for the message, the ACE does not display the message. Workaround: Reconfigure logging by including the syslog level (for example, the logging message log_number level 6 command).

CSCsu99354—When you reorder class maps under a multi-match policy, other class maps under the same policy may stop working. Workaround: Remove and readd any class map that is not working in the multi-match policy.

CSCsv01139—When you configure RTSP load balancing and inspection with the leastconns predictor, the ACE becomes unresponsive due to a memory allocation failure. Workaround: None.

CSCsv01152—When the ACE runs RTSP traffic, it becomes unresponsive. Workaround: None.

CSCsv05109—When you repeatedly add and remove the same set of match source-address ip_address netmask commands from a class map with a large number of match statements, the ACE becomes unresponsive and generates a core dump. Workaround: None.

CSCsv08314—When you configure a bandwidth policy, the ACE becomes unresponsive. Workaround: None.

CSCsv10306—When you configure an ACE for front-end and back-end SSL and the SSL clients are on a dialup link, if the server resets, the ACE immediately sends a reset to the client on the SSL connection without first sending the buffered data to the client and waiting for the client to acknowledge the sent data. This behavior causes the client to lose data. Workaround: None.

CSCsv15341—When you delete class maps, the deletions take a long time (for example, 30 minutes). During this time, the ACL-merge process takes almost 100 percent of the CPU and any new configuration command times out and returns with a command execute error. Workaround: None.

CSCsv15558—The show sticky database static [http-cookie cookie_value | type http-cookie] command does not work and returns no result. Workaround: None.

CSCsv18454—The ACE places a real server under the connection rate limit when no rate limits are configured. Workaround: Remove the real servers in the server farm and then readd them.

CSCsv21228—When the ACE runs a large number of HTTP and HTTPS probes, if it probes for a large file (approximately 1 megabyte in size), the ACE reboots when it runs low on memory and then enters a continuous reboot loop. Workaround: When the ACE runs a large number of probes, reduce the size of the file being probed.

CSCsv23350—After the fix of CSCso80600, some probes may not work properly. Workaround: None.

CSCsv24818—When you configure a match source-address statement with an invalid network mask, if you modify the class map using the statement, the ACE generates a core dump and may become unresponsive. Workaround: None.

CSCsv30691—When you use LbInspectTool to collect diagnostic data from the ACE and enter the Ctrl+\ key combination, the ACE becomes unresponsive. Workaround: Do not use LbInspectTool.

CSCsv31345—When you configure the ACE with more than 1,500 parameter maps and then configure one of them in another configuration (for example, adding an SSL-proxy service to a parameter map), the ACE becomes unresponsive and then reboots. Workaround: None.

CSCsv31397—When you change an access group configuration, an access-group download error occurs on the ACE. Workaround: None.

CSCsv40516, CSCsr22703, CSCsv67574—The ACE became unresponsive and generated a core dump while it was executing an OS kernel function. This behavior appears to have been a one-time event. Workaround: None.

CSCsv41126, CSCsu80235—After the ACE runs for a long period of time, it may drop connections because of the Drop On Max Remote Stky counter. The Pending Remote Sticky Conns counter displays 8192. Workaround: None.

CSCsv46419—When you configure a backup real server, the rate-limit bandwidth command does not work and the server never goes to the out-of-rotation state. Workaround: Do not configure backup real servers with this command.

CSCsv50144—When you remove and add an FT group on an active ACE module that has traffic flowing through it and sticky entries are being replicated, load balancing on the standby ACE becomes unresponsive. Workaround: None.

CSCsv60118—When you delete or change configurations that include multiple rate and bandwidth-limited real servers, the ACE reboots and generates a core dump. Workaround: None.

CSCsv61295—When you configure SIP inspection and the SIP message contains the letters "tel" before the sip: header information, SIP parsing fails. Workaround: None.

CSCsv63192—When the ACE is under stress and reaches connection thresholds, some ICMP connections become stuck and cannot be cleared. Workaround: None.

CSCsv63364, CSCsu95356—When you configure the ACE for stickiness with the leastconns predictor and enable the slow start algorithm, the ACE fails to load balance properly where some servers receive almost zero connections and other servers receive thousands of connections. When a probe fails, it causes the real server to go out-of-service. When the server comes back and the probe succeeds, the real server distribution does not recover. Workaround: Temporarily setting the load-balance algorithm to roundrobin and then back to the leastconns predictor may clear the issue.

CSCsv71260, CSCsr68233—When server load balancing is configured on the ACE, the current connections counter of the show serverfarm command appears to be incorrect because it is greater than the show conn command counter. Workaround: None.

CSCsv82791—When the ACE generates a core-dump file for a legitimate error condition and reboots, the file on the disk is truncated or incomplete and may not contain the information to identify the cause of the reboot. Workaround: None.

Software Version A2(1.3) Open Caveats

The following open caveats apply to software version A2(1.3):

CSCsj68643—The following log messages may appear sporadically in the ACE log:

can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.

can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.

These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.

CSCsj74250—When you configure the TACACS+ server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, then there is a key mismatch when the ACE attempts to authenticate a user. Workaround: Paste the properly encrypted key into the running-configuration file.

CSCsj94366—When you attempt to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears:

console configuration can only be done on console

Workaround: None.

CSCsl21191—When you enter the show module command on the supervisor engine for a running ACE, the command output may fail to display the software version information from the ACE. When this behavior occurs, the command output displays similarly to the following example output:

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
 4  0018.b9a6.88fc to 0018.b9a6.8903   1.1   8.6(0.252-En 8.6(0.252-En Ok  

This behavior rarely occurs, but once it does, the behavior will continue every time that you enter the show module command. The ACE continues to forward traffic normally. This is a display problem only. Workaround: Reboot the ACE.

CSCsl46334—When a high rate of Layer 7 load-balanced traffic is flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse is configured, the ACE may start dropping traffic after a few hours. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl75662—You may observe that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time that the probe is sent even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from the real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsm46044—ACL-MERGE-ERROR messages may be logged as follows:

ACL-MERGE-ERROR:cannot find ACL in acl_merge_rem_acl_from_list 
../security/acl/acl_merge.c:xxx

You may observe this behavior when you enable the debug access-list merge errors command in debug mode and then add new configurations to the ACE. Workaround: None.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform either of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, and then log in again to access a different context than the original context with the configured packet capture function.

CSCso12560—The show resource usage command may display a nonzero number for some resources that have their maximum value set to equal-to-min. Workaround: None.

CSCso38853—After four consecutive Route Processor Redundancy (RPR) failovers in the Catalyst 6500 series switch, the primary and standby ACEs may enter the Active-Active state. This state is not resolved until you reload the primary ACE. Workaround: None.

CSCso60304—When an invalid XML attribute is sent to the ACE, it does not respond as expected. Instead, the ACE displays a 500 Internal Server Error message. No negative impact to the ACE is observed. Workaround: None.

CSCso81785—If you are using TACACS+ and the Cisco Access Control Server (ACS) with an RSA authentication manager, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another network access server (NAS) to set your PIN.

CSCso81811—If you are using TACACS+ and the Cisco ACS with an RSA authentication manager and your account is in next token mode, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another NAS to enter the next token code and make your account accessible again.

CSCso93479—The Current Connections counter that is displayed in the output of the show serverfarm name command is not accurate. The output of the show service-policy command does have an accurate counter. Workaround: None.

CSCso95620—With long-lived HTTP, SSL, FTP and UDP traffic running on the ACE, you may observe a memory loss of approximately 333 KB in the ACE during an EtherChannel link (FT port channel) failure and recovery on the Catalyst 6500 series switch. Workaround: None.

CSCsq14440—The aclmerged process in the ACE may not complete or may exceed the available system resources. With very large configurations where there are many ACLs, NAT statements, and class maps, the processing of these elements can require a significant amount of time and internal resources. In some cases, the processing (as displayed by the show proc cpu | include aclmerged command) may become unresponsive and never complete. In other cases, the processing may complete, but the output could exceed the resources available on the ACE, which may cause the ACE to not function properly.

Workaround and recovery: Currently, there is no method to predict the aclmerged response. However, in most cases, the commands eventually complete and the ACE continues to function properly. The suggested workaround is to allow aclmerged to complete without any intervention, assuming that there is no external impact to traffic. If the process does not complete or if there is a significant disruption to traffic flow, then reboot your ACE. If you enter the write memory command prior to the reboot, then the ACE attempts to come up in the post-change configuration. This may allow the desired configuration to be applied properly after the reboot. If you do not enter the write memory command before rebooting the ACE, then the ACE should reload and continue to operate in the same manner as before the change.

CSCsq23701—After an FT VLAN failure, which resulted in an Active/Active FT state, has been resolved, the ACE with the higher priority should take over as the active ACE (even though the preempt command is disabled) through the election process, but did not. Workaround: Enter the preempt command.

CSCsq27062—After toggling the state of the FT port channel in the Catalyst 6500 series switch 110 times, the primary ACE module generated a core dump and reloaded. Workaround: None.

CSCsr09129—When you configure SIP load balancing with inspection enabled, the ACE should open a pinhole to the address in the Via header for the server response. However, the server responses remain in the data channel. Workaround: None.

CSCsr72591—When you need to import many SSL keys and certificates, it may take a long time (approximately 30 minutes to import 1000 keys and certificates). You must import them one at a time; there is no bulk import feature available. Workaround: None.

CSCsu42225—When you configure the ACE with a Layer 4 load-balancing policy map and it receives a series of UDP requests with a payload of 3,200 bytes that spans three nonfragmented packets, the ACE drops two packets from the first request. For subsequent requests, the ACE load balances all packets successfully. Workaround: None.

CSCsu67523 and CSCsu67556—Upgrading the ACE software to version A2(1.1a) causes the ACE to reboot and generate a core dump. Workaround: None.

CSCsu67539—When you upgrade the ACE software to version A2(1.1), the ACE reboots and generates a core dump. Workaround: None.

CSCsu68314—When the ACE becomes unresponsive and generates a core dump, the core-dump file contains three different types of files. These files should be separate files. Workaround: Use the file command to uncompress the core-dump files.

CSCsu86606—When you reboot the ACE and as it powers up, the Catalyst 6500 series switch disables the ACE and displays the following log messages:

Oct  1 07:43:25.710 EDT: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off 
(Reset)
Oct  1 07:43:41.611 EDT: %OIR-SP-6-PWRFAILURE: Module 1 is being disabled due to power 
convertor failure 0x1 

Workaround: None.

CSCsu88684—When you configure the ACE with a large number of contexts and enable redundancy, as traffic flows on the ACE, the ACE becomes unresponsive and displays the following messages on the console:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

Then the ACE reboots. Workaround: None.

CSCsu95356—When you configure the ACE with the predictor least conn command, the real server does not get the expected number of connections. Workaround: Remove the real server from the server farm and readd it.

CSCsu95887—After the active ACE module completes configuration synchronization, it generates a core dump. Workaround: None.

CSCsu96977—When you configure more than 640 action lists and enter the do show action_list command with the Tab or ? key for help, the ACE becomes unresponsive. Workaround: None.

CSCsv02224—When you configure and remove an SSL-proxy service after you configure and remove multiple class maps under a policy map, the following error appears on the console:

Error: Called API encountered error appears console.

The ACE rejects the ssl-proxy command and the command does not appear in the configuration. Workaround: None.

CSCsv04319—If you create a TACACS+ server with a numeric key, the ACE sends a warning about the key; however, it does not create the server. The message should be an error and not a warning. Workaround: Use a key that is not entirely numeric.

CSCsv04848—When you configure RADIUS on the ACE and a user logs off, the RADIUS client sends an accounting stop message to the server for that user but the ACE does not immediately delete all connections for that user. If the source IP address for the user is immediately reassigned to another user, the new user could open a new connection before the old connections from previous user times out. The result is that the ACE incorrectly forwards the new connections and does not load balance the packets. Workaround: Set the UDP connection timer to a smaller number (for example, 10 seconds).

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv10547—The config-register setting does not synchronize after an ACE module boots. The config-register setting synchronizes only when you configure it with ACE modules in active or standby mode. Workaround: None.

CSCsv31046—When you configure the least-connections predictor on the ACE, the ACE may not sustain 160,000 CPS traffic. Workaround: None.

CSCsv31394—When you modify the policy-map configuration on an interface, the ACE occasionally records a service-policy download error. Workaround: None.

CSCsv31476—When the ACE generates a core-dump file for the kernel or Virtual Shell (VSH) applications, the file does not contain the code-train version information. Workaround: None.

CSCsv32122—The download of 16K source IP-address match statements can take 40 seconds. Workaround: None.

CSCsv33051—When you configure RADIUS load balancing and create a RADIUS-attribute sticky group with the sticky radius framed-ip command, if the Framed-IP-Address is reused and load balanced to a different rserver, the ACE may not update the sticky entry. Workaround: Configure the RADIUS client to issue Framed-IP-Addresses and include them in the RADIUS access request messages or configure separate Framed-IP-Address pools for each RADIUS real server.

CSCsv47724—The heartbeats on fault-tolerant (FT) ACE modules occasionally miss due to late TCP timers. The FT ACEs increment the Heartbeats Missed counter on the standby ACE and the Unidirectional HB's Received counter on the active ACE. Workaround: None.

CSCsv48498—When you enable FTP inspection and disable normalization on the client-side interface, the ACE inserts the TCP Option Timestamp in packets to the client and the FTP server, even if both the client and the server are not using this option. Workaround: Enable normalization or disable FTP inspection.

CSCsv49606—When you configure stickiness on the ACE, the ACE becomes unresponsive. Workaround: None.

CSCsv52288—The ACE supports only 8K match source-address statements entries. Workaround: None.

CSCsv52331—The ACE becomes unresponsive due to an SRAM parity error. Workaround: None.

CSCsv52478—When you reboot the Catalyst 6500 series chassis, the ACE may reboot as Active. Workaround: None.

CSCsv52887—When an ACE with a large number of match source-address entries is under a high traffic load, modifying the match source-address entries may cause the console or terminal to lock briefly. Workaround: None.

CSCsv53112—When you enter the show xlate command, the ACE may generate a core dump. Workaround: None.

CSCsv53187—The ACE generates an NP ha_hb_g_ns core dump during standard operation. Workaround: None.

CSCsv53620—When you add an SSL proxy class to a policy map, the following error occurs:

Error: Called API encountered error

Workaround: Remove the class from the policy map and then readd it.

CSCsv56991—When you change the configuration of a real server on a server farm, the ACE does not replicate the connections. Workaround: None.

CSCsv59066—When using KAL-AP to report the VIP address status, all VIPs with the same addresses report a load of 255 if one is out of service. Workaround: Do not use KAL-AP to monitor multiple VIPs with the same IP addresses.

CSCsv65178—When you specify TCP as the protocol in a class map configured for DNS traffic, the ACE allows the configuration and DNS inspection fails. Workaround: Specify UDP as the protocol in a class map configured for DNS traffic.

CSCsv69769—When you configure an expect regex value, the ACE allows a space in the quoted name of the value. Workaround: Do not use a space. Instead, use a search character (.*) or allow the variable to be on a long string input.

CSCsv95254—When an IP address conflict occurs on a bridged VLAN, the ARP manager may become unresponsive causing the ACE to generate a core dump. Workaround: None.

CSCsw88171—When you make health monitoring changes, MTS data corruption occurs. The ACE reboots and generates a core file. Workaround: None.

Software Version A2(1.3) Command Changes

Table 8 lists the commands and options that have been changed in software version A2(1.3).

Table 8 CLI Commands Changed in Version A2(1.3)  

Mode
Command and Syntax
Description

Exec

clear serverfarm name predictor

The new predictor keyword resets the the average bandwidth field for each real server in the specified server farm, as displayed by the show serverfarm name detail command.

Exec

crypto delete

crypto export

crypto generate csr

crypto generate key

crypto import

crypto verify

ft swtichover

The crypto and ft commands are now disabled by default for the network-monitor role.

Class map

[line_number] match virtual-address address {[mask] | any | {tcp | udp {any | eq port_number | range port1 port2}} | protocol_number}

Previously, the ACE allowed you to configure a class-map VIP address that overlaps with an ACE interface IP address. Per CSCsu67719, the ACE no longer allows this configuration and displays the following warning:

Error: Entered VIP address is not the first address in 
the VIP range

Class map HTTP inspection

[line_number] match request-method {ext subscribe | unsubscribe | poll | notify | x-ms-enumatts}

The match statement HTTP inspection extension method for this command now includes the subscribe, unsubscribe, poll, notify, and x-ms-emumatts options.

Policy map HTTP inspection

match request-method {ext subscribe | unsubscribe | poll | notify | x-ms-enumatts}

The inline match HTTP inspection extension method for this command now includes the subscribe, unsubscribe, poll, notify, and x-ms-emumatts options.

Role

rule number {permit | deny} {create | modify | debug | monitor} [feature changeto-command | exec-commands]

Previously, you could not configure user-defined roles to use the changeto command. Per CSCsr90230, the new changeto-command option allows a user-defined role to use the changeto command. Also, users retain their privileges when accessing different contexts. By default, this command is disabled for user-defined roles.

Previously, the ACE enabled Exec mode commands for user-defined roles. Per CSCsr00851, the new exec-commands option allows a user-defined role to use the capture, clear, debug, delete, gunzip, mkdir, move, rmdir, set, setup, system, tac-pac, untar, write, and undebug commands. By default, these commands are now disabled for user-defined roles.


Software Version A2(1.2) Resolved Caveats, Open Caveats, and Command Changes

The following sections contain the resolved and open caveats in software version A2(1.2):

Software Version A2(1.2) Resolved Caveats

Software Version A2(1.2) Open Caveats

Software Version A2(1.2) Command Changes

Software Version A2(1.2) Resolved Caveats

The following resolved caveats apply to software version A2(1.2):

CSCsi13378—If you configure certain commands in the ACE (for example, object-group, action-list, and so on) and you enable the xml-show command, the output of the show running-config command displays data outside the XML tags or incorrect XML tags. Workaround: None.

CSCsm65534—You may find that sequential readings of the Client Byte Count and the Server Byte Count fields in the show service-policy command output increment or decrement by large values without the expected changes in network traffic. This behavior is a display-only issue and does not affect traffic forwarded by the ACE. You may encounter this behavior after the byte counters exceed the maximum of 4294967295 bytes. Workaround: None.

CSCsm89594, CSCsr14898—XML outputs are not well formatted for the following show commands:

show ft stats

show ft track 1 detail

show ft track 1 summary

show ft track 1 status

show tacacs-server sorted

show running-config policy-map

show running-config probe

show serverfarm sf1 detail

show rserver detail

Workaround: None.

CSCsm89835—The ACE rejects HTTP requests that contain non-ASCII characters that are not percent-encoded and are placed after the question mark (?) in a URL. Non-English websites may use those characters to pass data (for example, names) and web servers do accept such characters. Workaround: None.

CSCso22472—When you use class maps of type http loadbalance match-any to select a server farm and some of these class maps are empty, the ACE may make an incorrect load-balancing (LB) decision. This incorrect LB decision causes unexpected LB results. For example:

class-map type http loadbalance match-any A
  2 match source-address 192.168.1.1 255.255.255.255
class-map type http loadbalance match-any B <<< empty
class-map match-all VIP
  2 match virtual-address 192.168.1.10 tcp eq telnet
policy-map type loadbalance first-match LB
  class A
    serverfarm A
  class B
    serverfarm B
  class class-default
    serverfarm C    

Workaround: In the above configuration, you must add a dummy match statement under class map B. For example:

class-map type http loadbalance match-any B 
  2 match source-address 172.16.27.5 255.255.255.255 

CSCso38316—Following negative XML testing, a core dump occurred. The core dump did not cause the ACE module to reload, nor was there any negative impact to the ACE module. Workaround: None.

CSCso38327—While running SSL client authentication, the browser intermittently does not recognize that a certificate has been revoked. Instead, the browser indicates that the server has failed or could not connect. Workaround: None.

CSCso55673—Over time, the ACE can leak memory when it has a light continuous load of SSL client authentication traffic. The ACE will typically display a log message indicating this low memory condition before the CLI becomes unresponsive and the ACE possibly reloads. The ACE indicates that it is low on directly mapped memory by displaying the following message:

Available CP memory less than 1%: 8380416 bytes. Free high memory: 2093056 bytes

Workaround: None.

CSCso65486—With the SYN cookie feature configured on an ACE interface that is forwarding nonload-balanced traffic to a routed server, all legitimate traffic that is receiving a SYN cookie is being reset. A packet capture for failed connections shows that the ACE completes a three-way handshake with the client and then with the server before it resets the connection. This behavior may also be observed with load-balanced FTP traffic. Workaround: None.

CSCso73385—When you enter the inspect ftp command in a policy map, the ACE resets the FTP connection of the traffic that matches the policy after it sends an extended PASV (EPSV) command to the FTP server. Workaround: None.

CSCso79767—When DNS traffic matches a rule that contains the inspect dns command and the DNS response from the server contains a VIP address, the ACE drops the DNS response. Workaround: Disable the inspect dns command.

CSCso81191—The ACE module exits to the ROMMON prompt during an import into ANM when the configuration includes a Layer 7 SLB policy map that contains the drop or forward action. Workaround: None.

CSCso81172, CSCsv49518—An ACE shows dropped ICMP packets on servers that are tagged for a load-balancing VLAN. If you change the servers to a non-loadbalancing VLAN, the packet loss is not observed. Packet loss is also observed with just a bridged VLAN interface (BVI) group configured. Workaround: Reload the ACE.

CSCso85639—If you configure the passdetect interval command value for less than 30 seconds, the ACE sends overlapping probes that use additional management connections (resources). Workaround: Increase the passdetect interval command value to 45 seconds.

CSCso91403—You may observe connection resets when you modify a large configuration. These resets may occur even if you modify a service policy that is not assigned to an interface. Workaround: None.

CSCsq18476—In a RADIUS authentication configuration, if all of the RADIUS servers fail, the ACE falls back to the local database for authentication even if you change the default from local to the RADIUS servers. For example:

10.10.10.10 key 7 "abc" authentication accounting radius-server host
10.10.10.11 key 7 "abc" authentication accounting aaa group server
radius RADIUS_SERVERS
  server 10.10.10.10
  server 10.10.10.11
aaa authentication login default group RADIUS_SERVERS < not have local option
aaa authentication login console group RADIUS_SERVERS < not have local option

Workaround: None.

CSCsq23888—When you create a scripted probe that contains a VSH configuration command on the active ACE in a redundant configuration, the probe may fail with the "Internal error: Script error" error message on the standby ACE. The configuration commands are executed on the active ACE and then replicated on the standby ACE. If ft auto-sync running-config is disabled on the active ACE, the scripted probe executes properly on the active ACE but will fail on the standby ACE. Workaround: Enable ft-auto sync running-config on the active ACE.

CSCsq25300—When you configure fastpath logging to a syslog host in the ACE, the connection setup and teardown messages that are sent to the syslog server may contain an incorrect duration time stamp and may be formatted improperly. Workaround: None.

CSCsq28177—An ACE is present in the chassis, but while trying to perform an SNMP walk on the instance reported by the cefcModuleOperStatus MIB, a message states that the module is missing. Walking cefcModuleOperStatus(1.3.6.1.4.1.9.9.117.1.2.1.1.2) returns the complete value "SNMPv2-SMI::enterprises.9.9.117.1.2.1.1.2.1 = INTEGER: 2." While trying to walk "SNMPv2-SMI::enterprises.9.9.117.1.2.1.1.2.1," a message states that No Such Instance currently exists at this OID.

Workaround: Perform an SNMP walk on cefcModuleOperStatus(1.3.6.1.4.1.9.9.117.1.2.1.1.2).

CSCsq38934—The ACE may fail to respond to an ICMP Echo Request to the VIP address when a policy map is configured with the loadbalance vip icmp-reply active command and the same VIP address is configured in the class map with different IP ports and one of these VIP match statements is deleted.

Workarounds:

In a class map with the same VIP in multiple match statements, do not delete individual match statements. If you must make configuration changes, reboot the ACE.

If individual match statements for the same VIP need to be deleted, either reboot the ACE or delete the policy map and reconfigure it.

CSCsq45437—When you remove a probe that is associated with multiple real servers from one of the real servers, changes to the common probe parameters (for example, interval, passdetect interval, passdetect count, faildetect count, receive timeout, and so on) do not take effect and the probes continue to use the old values. Workaround: After you change the probe parameters, remove the probe association from one of the real servers and then reassociate the probe with the server.

CSCsq48296—If the persistence-rebalance command is enabled under an HTTP parameter map, the ACE may lose the MSS setting in the middle of a flow. Workaround: Configure the set tcp wan-optimization rtt 0 command under a connection parameter map.

CSCsq68949—When you use the CSM2ACE utility, a duplicate parameter map may be created when the utility converts a CSM vserver to the ACE equivalent class map. Workaround: Manually delete duplicate parameter maps and update the ACE configuration to use the consolidated parameter map.

CSCsq71893—When downloading an invalid Certificate Revocation List (CRL) while SSL termination traffic is enabled, the ACE may become unresponsive. It may also become unresponsive if there are delays from the server when sending the CRL data to the client. Workaround: Ensure that the CRL file that is referenced contains the valid and relevant CRL data and is in the proper format.

CSCsq71917—The ACE may become unresponsive if client authentication is enabled and the client certificate exceeds 16 KB.

Workarounds:

Use a client certificate that is smaller than 16 KB.

If possible, disable client authentication.

CSCsq75217—When the authentication-failure ignore command is configured in an SSL parameter map and a CRL is applied to the SSL proxy server on which the connection was received, a client connection may become unresponsive if the client uses an expired or an invalid certificate. The connection may stall while the ACE completes the revocation checks.

Workarounds: Perform one of the following:

Do not enable the authentication-failure ignore command.

Do not use a CRL with an SSL proxy.

Do not enable client authentication with an SSL proxy.

CSCsq81407—When you enter the show svclc module 3 traffic command on a Cisco Catalyst 6500 switch with an ACE module and a high traffic rate, after ten minutes, the byte counts are not correct in the output of the command. Workaround: None.

CSCsq87869—The show conn display 1000 detail command is part of the showtech script. This command should limit its output to the first 1000 connections, but there are always more connections in the output of the command if there are more than 1000 connections on the ACE. Workaround: None.

CSCsq91503—When a SIP call is closed simultaneously from both ends, the ACE may encounter a deadlock and become unresponsive. Workaround: None.

CSCsq92011—When a partial server farm failover is configured, or if all the real servers in the server farm become unavailable, and the backup server farm becomes active, the ACE may enter an unresponsive state while processing the failover message. This behavior may occur even with minimal network traffic. Workaround: None.

CSCsq92590—Prior to this release, the ACE allowed a maximum of 4096 Layer 7 match statements. With version A2(1.2), the ACE allows a maximum of 16,384 match statements. Workaround: Configure nonspecific regular expressions.

CSCsq93851—While using large certificates in an authgroup, the SSL process becomes unresponsive. Workaround: None.

CSCsr13505—If you attempt to authenticate users with a RADIUS server that does not support the authenticate-only service type, that RADIUS server does not pass role and domain information to the ACE during login authentication. Workaround: Use a RADIUS server that supports the authenticate-only service type.

CSCsr14070—SSL handshakes do not fail, but they should, when you change the SSL configuration parameters as follows:

Configure session reuse.

Configure an authgroup that verifies the server certificate.

Run traffic to create a client session cache entry.

Change the authgroup to one that does not verify the server certificate.

Open the connection again and the handshake does not fail.

Workaround: Clear the session cache. The connection in the last step above uses the client session cache entry to resume an SSL connection, which bypasses authgroup processing, because the previous step did not clear the session cache entry when the VIP changed. Always clear the session cache for the VIP when the VIP changes.

CSCsr14475—When the leastconns predictor is configured with the slowstart option, there is at least one real server with a backup server in the server farm, and you change the predictor to roundrobin and back to leastconns more than once while traffic is flowing, the ACE may become unresponsive. Workaround: None.

CSCsr40916—When the ACE attempts to download an expired CRL in an SSL configuration, the ACE may become unresponsive. Workaround: If possible, avoid the use of CRLs.

CSCsr41732—When the ACE is configured for SSL termination, client authentication, and a CRL, and SSL traffic is flowing, the ACE may become unresponsive while it is downloading a CRL and accessing that CRL for revocation checks at the same time.

Workarounds:

Do not use a CRL with an SSL proxy.

Do not enable client authentication with an SSL proxy.

CSCsr46740—If you configure FTP inspection and source NAT on the ACE, the ACE does not fix up the IP address of the original client when it forwards the PORT request to the backend server and the following conditions exist:

The ingress VLAN for client-to-VIP traffic is different from the egress VLAN for VIP-to-client traffic. This difference in VLANs can happen if the client is a routed client and the route back to the client points to a VLAN other than the ingress VLAN.

Source NAT is configured on the ingress VLAN

mac-sticky is configured on the ingress VLAN

Workaround: Add a static route to the client's IP subnet that sends egress traffic out the ingress VLAN.

CSCsr54155—The microengine dumper task may become unresponsive and not reload the ACE, which can result in the ACE running in a degraded manner for an unknown period of time. Workaround: None.

CSCsr57325—When stickiness is configured on a context and the maximum number of pending sticky requests exceeds 8192, the ACE may drop connections and the client may observe resets or pages not loading properly. If you enter the show np 1 | 2 me-stats -slb command, the Drop On Max Remote Stky counter keeps increasing. When this counter has a nonzero value, it means that there are too many sticky lookup requests in the pipeline and the ACE is dropping new requests. Workaround: In a redundant configuration, force a failover to the standby ACE, and then reload the ACE where you observed the problem.

CSCsr65064—When users log into the ACE with different roles, they behave like Network Monitor users and are not able to enter any configuration mode commands. Workaround: None.

CSCsr71638—Traffic that is destined to a real server may fail even if the real server state is OPERATIONAL. The number of current connections to the real server may be stalled or show a very high number of open connections in the output of the show rserver command. The "Total times rserver was unavailable" counter in the output of the show stats loadbalance command is incrementing. You may also see Layer 4 or Layer 7 rejections in the output of that command. Workaround: None.

CSCsr81611—The ACE may become unresponsive when you remove an access list configuration. Workaround: If possible, avoid removing any access list configuration.

CSCsr89311—An incompatibility exists between certain ACE software versions in the 3.0(0)A1.6.3x and A2.1x release trains. In a redundant configuration, the FT ACE pairs will not recognize each other and will report the following status as part of the show ft peer detail command output:

SRG Compatibility : INCOMPATIBLE

The following software version combinations that are marked with an "x" are incompatible:

A1(6.3x) Release
A2(1.0)
A2(1.0a)
A2(1.1)
A2(1.1a)
A2(1.2)

3.0(0)A1(6.3b)

x

 

x

x

 

3.0(0)A1(6.3c)

x

x

x

x

 

CSCsr96168—This caveat rescinds the fix for CSCsg80625, which was resolved in software version A2(1.1a).

Software Version A2(1.2) Open Caveats

The following open caveats apply to software version A2(1.2):

CSCsj68643—The following log messages may appear sporadically in the ACE log:

can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.

can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.

These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.

CSCsj74250—When you configure the TACACS+ server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, then there is a key mismatch when the ACE attempts to authenticate a user. Workaround: Paste the properly encrypted key into the running-configuration file.

CSCsj94366—When you attempt to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: "console configuration can only be done on console." Workaround: None.

CSCsl21191—When you enter the show module command on the supervisor engine for a running ACE, the command output may fail to display the software version information from the ACE. When this behavior occurs, the command output displays similarly to the following example output:

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
 4  0018.b9a6.88fc to 0018.b9a6.8903   1.1   8.6(0.252-En 8.6(0.252-En Ok  

This behavior rarely occurs, but once it does, the behavior will continue every time that you enter the show module command. The ACE continues to forward traffic normally. This is a display problem only. Workaround: Reboot the ACE.

CSCsl46334—When a high rate of Layer 7 load-balanced traffic is flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse is configured, the ACE may start dropping traffic after a few hours. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health checks. Workaround: None.

CSCsl75662—You may observe that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time that the probe is sent even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from the real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsm46044—ACL-MERGE-ERROR messages may be logged as follows:

ACL-MERGE-ERROR:cannot find ACL in acl_merge_rem_acl_from_list 
../security/acl/acl_merge.c:xxx

You may observe this behavior when you enable the debug access-list merge errors command in debug mode and then add new configurations to the ACE. Workaround: None.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform either of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, then log in again to access a different context than the original context with the configured packet capture function.

CSCso00234—After the ClientHello and the ServerHello, the ACE responds to the client with the Fatal, description:Bad Record MAC alert. Currently, the ACE cannot process non-minimally padded block ciphers, which is a TLS 1.0 feature. You employ non-minimally padded block ciphers in the following situations:

You use TLS version 1.0.

You negotiate a block cipher (AES256).

The Finished message is 256 bytes.

Workaround: If possible, restrict the SSL protocol version to SSL version 3. Alternatively, allow only stream ciphers, such as RC4.

CSCso12560—The show resource usage command may display a nonzero number for some resources that have their maximum value set to equal-to-min. Workaround: None.

CSCso19129—When the ACE is configured for load balancing or server-side NAT and RTSP inspection is enabled, the Windows Media Server may reject an RTSP session if you are using Windows Media player 10. If a real server IP address to VIP (or vice versa) translation is required, the ACE translates the IP address in the SDP part but does not update the content length in the header part. The message is then rejected by the server. This behavior does not occur if the real address length and VIP address length is the same. If the IP address length does not change between the VIP (mapped address) and the real server (real address), then this behavior does not occur. Workaround: None.

CSCso21587—When RTSP inspection is enabled and the ACE performs load balancing or destination NAT on an RTSP session, the Windows Media Server may reject the session. If the media stream is interleaved with the RTSP control connection (the transport type is RTP/TCP or RDT/TCP), then the ACE incorrectly unproxies the control connection as soon as it detects the transport type as TCP, causing the rest of the messages between the client and the server to pass through the ACE without any inspection. As a result, subsequent SETUP messages are not fixed up (NATed) and the server rejects the SETUP message with the VIP address (instead of the real server address) in it. Workaround: Use UDP instead of TCP as the transport mode for the media streams.

CSCso38853—After four consecutive Route Processor Redundancy (RPR) failovers in the Catalyst 6500 series switch, the primary and standby ACEs may enter the Active-Active state. This state is not resolved until you reload the primary ACE. Workaround: None.

CSCso47783—When you configure the ACE for NAT and you are using the NAT counters for troubleshooting, the NAT failure counter does not provide enough granularity for all cases that may cause the counter to increment. Workaround: None.

CSCso60304—When an invalid XML attribute is sent to the ACE, it does not respond as expected. Instead, the ACE displays a 500 Internal Server Error message. No negative impact to the ACE is observed. Workaround: None.

CSCso69044—With SYN cookie enabled, embryonic connections (incomplete TCP handshakes) may remain on the ACE after more than 24 hours. Workaround: None.

CSCso81785—If you are using TACACS+ and the Cisco Access Control Server (ACS) with an RSA authentication manager, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another network access server (NAS) to set your PIN.

CSCso81811—If you are using TACACS+ and the Cisco ACS with an RSA authentication manager and your account is in next token mode, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another NAS to enter the next token code and make your account accessible again.

CSCso93479—The Current Connections counter that is displayed in the output of the show serverfarm name command is not accurate. The output of the show service-policy command does have an accurate counter. Workaround: None.

CSCso95620—With long-lived HTTP, SSL, FTP and UDP traffic running on the ACE, you may observe a memory loss of approximately 333 KB in the ACE during an EtherChannel link (FT port channel) failure and recovery on the Catalyst 6500 series switch. Workaround: None.

CSCsq14440—The aclmerged process in the ACE may not complete or may exceed the available system resources. With very large configurations where there are many ACLs, NAT statements, and class maps, the processing of these elements can require a significant amount of time and internal resources. In some cases, the processing (as displayed by the show proc cpu | include aclmerged command) may become unresponsive and never complete. In other cases, the processing may complete, but the output could exceed the resources available on the ACE, which may cause the ACE to not function properly.

Workaround and recovery: Currently, there is no method to predict the aclmerged response. However, in most cases, the commands eventually complete and the ACE continues to function properly. The suggested workaround is to allow aclmerged to complete without any intervention, assuming that there is no external impact to traffic. If the process does not complete or if there is a significant disruption to traffic flow, then reboot your ACE. If you enter the write memory command prior to the reboot, then the ACE attempts to come up in the post-change configuration. This may allow the desired configuration to be applied properly after the reboot. If you do not enter the write memory command before rebooting the ACE, then the ACE should reload and continue to operate in the same manner as before the change.

CSCsq27062—After toggling the state of the FT port channel in the Catalyst 6500 series switch 110 times, the primary ACE module generated a core dump and reloaded. Workaround: None.

CSCsq87162—SSL transactions may not complete when the server-conn reuse command is enabled. Workaround: Disable the server-conn reuse command.

CSCsr00851—A user with the configured RBAC user role of Network-Monitor is allowed to delete other users directories on the ACE Flash memory. Workaround: Do not configure users with the Network-Monitor role.

CSCsr09129—When loadbalancing SIP with inspection enabled, the ACE should open a pinhole to the address in the Via header for the server response. However, the server responses remain in the data channel. Workaround: None.

CSCsr16179—When system logging is enabled with the logging fastpath command, IP addresses in the Built TCP Connection syslog messages may be incorrectly swapped. Workaround: None.

CSCsr16201—Built TCP Connection and Teardown TCP Connection syslog messages may continue to be sent to a syslog server even after disabling this functionality with the no logging fastpath command. Workaround: Set logging trap 4.

CSCsr22703—The ACE became unresponsive and generated a core dump while it was executing an OS kernel function. This behavior appears to have been a one-time event. Workaround: None.

CSCsr28182—When a class-map any command is combined with a class-map all command and more than ten header matches of the same type are used, the ACE CLI displays the following error message:

Error: Maximum 10 http header map is allowed per policy!

Workaround: Decrease the number of header matches to 10 or less by using regular expressions (regexes).

CSCsr38682—The ACE CLI allows you to configure overlapping IP addresses for both an alias IP address and a VIP address. When you add a service policy to an interface with the overlapping VIP and alias IP addresses, the ACE displays the following error message:

Error: vip address duplicates with an existing interface ip address!

However, if you remove the alias IP address and add the service policy to the interface, and then reconfigure the same alias IP address, the ACE allows the configuration. Also, when you reboot the ACE with this configuration, you receive the "*** cmd exec error ***" parser error and the ACE removes the service policy from the interface. Workaround: Ensure that the VIP address and the alias IP address are unique within the context.

CSCsr57510—When you configure a VIP with a subnet mask of 255.255.255.255 and you configure a policy map to forward traffic, the ACE may drop the packets because of a route lookup failure. Workaround: Configure a class map to match on the destination address.

CSCsr62027—When TCP normalization is disabled, the ACE places replicated TCP connections in the INIT state on the standby ACE. After the normal embryonic connection timeout occurs, the ACE removes the replicated connections from the standby. Workaround: Do not disable normalization.

CSCsr67565—When you create a Certificate Signing Request (CSR) parameter set on the ACE, the CLI does not allow special characters, for example, comma (,) and period (.), in the following CSR fields:

State

Locality

Organization-name

Organization-unit

Serial-number

This behavior occurs only in software releases starting with version A2(1.1). Previous releases allow these characters in the CSR fields. Workaround: Use a previous release to generate a CSR.

CSCsr68233—When server load balancing is configured on the ACE, the current connections counter of the show serverfarm command appears to be incorrect because it is greater than the show conn command counter. Workaround: None.

CSCsr72591—When you need to import many SSL keys and certificates, it may take a long time (approximately 30 minutes to import 1000 keys and certificates). You must import them one at a time; there is no bulk import feature available. Workaround: None.

CSCsr75832—When you modify the configuration of the ACE, the module may classify traffic using the wrong class map and, consequently, forward traffic to the wrong server farm. Workaround: Wait for a few seconds for the modified configuration to take effect.

CSCsu42225—When the ACE is configured with a Layer 4 load-balancing policy map and it receives a series of UDP requests with a payload of 3200 bytes that spans three non-fragmented packets, the ACE drops two packets from the first request. For subsequent requests, the ACE load balances all packets successfully. Workaround: None.

Software Version A2(1.2) Command Changes

Table 9 lists the commands and options that have been changed in software version A2(1.2).

Table 9 CLI Commands Changed in Version A2(1.2)  

Mode
Command and Syntax
Description

Exec

show service-policy [policy_name [class-map class_name]] [detail | summary]

Added the optional class-map class_name and summary keywords to this existing command. You can now specify summary statistics for server load-balancing policies. In addition, you can specify detailed or summary statistics for a particular policy with all its associated class maps or a particular class map associated with a particular policy. The output of the summary option in tabular format includes the following fields:

Service-policy—Unique identifier of the policy map.

Class—Name of the class map associated with the policy map.

VIP—Virtual IP address specified in the class map.

Protocol—Protocol specified in the class map.

Port—Port specified in the class map.

VLAN—VLAN ID of the interface to which the policy map has been applied.

State—Operational state of the VIP. Possible states are: IN-SRVC (in service) and OUT-SRVC (out of service).

Curr Conns—Number of active connections to the VIP.

Hit Count—Total number of requests for the VIP.

Dropped Conns—Number of requests for the VIP that were dropped.

Class map HTTP inspection

[line_number] match request-method {ext method | rfc method}

Added the following match statement HTTP inspection extension methods:

bcopy

bdelete

bmove

bpropfind

bproppatch

search

Policy map HTTP inspection

match request-method {ext method | rfc method}

Added the following inline match HTTP inspection extension methods:

bcopy

bdelete

bmove

bpropfind

bproppatch

search


Software Version A2(1.1a) Resolved Caveats and Open Caveats

The following sections contain the resolved and open caveats in software version A2(1.1a):

Software Version A2(1.1a) Resolved Caveats

Software Version A2(1.1a) Open Caveats

Software Version A2(1.1a) Resolved Caveats

The following resolved caveats apply to software version A2(1.1a):

CSCsg80625—When the ACE is configured with a Layer 4 load-balancing policy map and it receives a series of UDP requests with a payload of 3200 bytes that spans three nonfragmented packets, the ACE drops two packets from the first request. For subsequent requests, the ACE load balances all packets successfully. Workaround: None.

CSCso71629—When the ACE is configured with the failaction reassign command in a firewall load-balancing configuration, the inbound leg of a connection may not be reassigned, which causes traffic to continue to choose the failed inbound leg resulting in a traffic failure. Workaround: None.

CSCso80478—When you perform multiple parallel SNMP walks that last thirty seconds or longer on an ACE in a redundant configuration, you may observe response timeouts on both the active and the standby ACEs. You may also observe this behavior in multiple contexts. This behavior does not occur with SNMP walks of shorter durations. Workaround: None.

CSCsq24595—In a redundant configuration, the standby ACE becomes unresponsive when TCP reuse is enabled. This behavior is caused by logic related to the reused connections database on the standby. TCP reuse connections are not replicated to the standby. Other connections are replicated properly.

CSCsq40966—In a redundant configuration, the active ACE that is configured with a single context reboots and generates a core dump, causing the standby ACE to become active. Then, the new active ACE (previously standby) reboots and generates a core dump. The last boot reason on both ACEs is "NP 0 Failed : NP ME Hung". The ACEs are configured for load balancing and dynamic NAT without PAT and with multiple NAT pools and IP address ranges. Workaround: None.

CSCsq46553—The ACE incorrectly sends a null password to the Radius AAA server when it receives the encrypted session key from an SSHv1 client and upon receiving the Service_Request message from an SSHv2 client. When the client is prompted for the SSH password and enters it, the authentication succeeds. The initial null password that is sent to the RADIUS server may be logged as a Login Failure by the that server. Workaround: Use an alternative authentication method (for example, TACACS+).

CSCsq55950—An ACE shows dropped ICMP packets on servers that are tagged for a load-balancing VLAN. If you change the servers to a non-loadbalancing VLAN, the packet loss is not observed. Packet loss is also observed with just a bridged VLAN interface (BVI) group configured. Workaround: Reload the ACE.

CSCsq56133—While making changes to the bandwidth rate limit configuration on a real server in the ACE, the ACE generates a core dump. Workaround: None.

CSCsq59257—While running software version A2(1.0a) with load balancing and stickiness configured, the ACE becomes unresponsive. Workaround: None.

CSCsq69858—When the ACE is configured for TACACS+ authentication, a username that is defined on the TACACS server with a backslash (\) does not work because the ACE interprets the backslash character as a Tab character.

CSCsq96569—In an SSL configuration with the SSL session cache feature enabled, clicking the Reload button in the Firefox browser causes the ACE to reset the browser sessions if the server times out the connection. Workaround: Disable the SSL session cache feature.

CSCsr20681—While attempting to authenticate a remote user using a TACACS+ or a RADIUS server, the ANM or XML interface periodically fails. Workaround: Use a locally configured user with Admin privileges.

CSCsr40129—When the ACE is configured with an HTTP probe for multiple real servers the ACE may insert the wrong IP address in the header portion of an HTTP request when it sends the probe to the real servers. Workaround: None.

Software Version A2(1.1a) Open Caveats

The following open caveats apply to software version A2(1.1a):

CSCsi13378—If you configure certain commands in the ACE (for example, object-group, action-list, and so on) and you enable the xml-show command, the output of the show running-config command displays data outside the XML tags or incorrect XML tags. Workaround: None.

CSCsj68643—The following log messages may appear sporadically in the ACE log:

can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.

can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.

These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.

CSCsj74250—When you configure the TACACS+ server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, then there is a key mismatch when the ACE attempts to authenticate a user. Workaround: Paste the properly encrypted key into the running-configuration file.

CSCsj94366—When you attempt to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: "console configuration can only be done on console." Workaround: None.

CSCsl21191—When you enter the show module command on the supervisor engine for a running ACE, the command output may fail to display the software version information from the ACE. When this behavior occurs, the command output displays similarly to the following example output:

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
 4  0018.b9a6.88fc to 0018.b9a6.8903   1.1   8.6(0.252-En 8.6(0.252-En Ok  

This behavior rarely occurs, but once it does, the behavior will continue every time that you enter the show module command. The ACE continues to forward traffic normally. This is a display problem only. Workaround: Reboot the ACE.

CSCsl46334—When a high rate of Layer 7 load-balanced traffic is flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse is configured, the ACE may start dropping traffic after a few hours. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health checks. Workaround: None.

CSCsl75662—You may observe that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time that the probe is sent even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from the real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsm46044—ACL-MERGE-ERROR messages may be logged as follows:

ACL-MERGE-ERROR:cannot find ACL in acl_merge_rem_acl_from_list 
../security/acl/acl_merge.c:xxx

You may observe this behavior when you enable the debug access-list merge errors command in debug mode and then add new configurations to the ACE. Workaround: None.

CSCsm65534—You may find that sequential readings of the Client Byte Count and the Server Byte Count fields in the show service-policy command output increment or decrement by large values without the expected changes in network traffic. This behavior is a display-only issue and does not affect traffic forwarded by the ACE. You may encounter this behavior after the byte counters exceed the maximum of 4294967295 bytes. Workaround: None.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform either of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, then log in again to access a different context than the original context with the configured packet capture function.

CSCso00234—After the ClientHello and the ServerHello, the ACE responds to the client with the Fatal, description:Bad Record MAC alert. Currently, the ACE cannot process non-minimally padded block ciphers, which is a TLS 1.0 feature. You employ non-minimally padded block ciphers in the following situations:

You use TLS version 1.0.

You negotiate a block cipher (AES256).

The Finished message is 256 bytes.

Workaround: If possible, restrict the SSL protocol version to SSL version 3. Alternatively, allow only stream ciphers, such as RC4.

CSCso12560—The show resource usage command may display a nonzero number for some resources that have their maximum value set to equal-to-min. Workaround: None.

CSCso19129—When the ACE is configured for load balancing or server-side NAT and RTSP inspection is enabled, the Windows Media Server may reject an RTSP session if you are using Windows Media player 10. If a real server IP address to VIP (or vice versa) translation is required, the ACE translates the IP address in the SDP part but does not update the content length in the header part. The message is then rejected by the server. This behavior does not occur if the real address length and VIP address length is the same. If the IP address length does not change between the VIP (mapped address) and the real server (real address), then this behavior does not occur. Workaround: None.

CSCso21587—When RTSP inspection is enabled and the ACE performs load balancing or destination NAT on an RTSP session, the Windows Media Server may reject the session. If the media stream is interleaved with the RTSP control connection (the transport type is RTP/TCP or RDT/TCP), then the ACE incorrectly unproxies the control connection as soon as it detects the transport type as TCP, causing the rest of the messages between the client and the server to pass through the ACE without any inspection. As a result, subsequent SETUP messages are not fixed up (NATed) and the server rejects the SETUP message with the VIP address (instead of the real server address) in it. Workaround: Use UDP instead of TCP as the transport mode for the media streams.

CSCso22472—When you use class maps of type http loadbalance match-any to select a server farm and some of these class maps are empty, the ACE may make an incorrect load-balancing (LB) decision. This incorrect LB decision causes unexpected LB results. For example:

class-map type http loadbalance match-any A
  2 match source-address 192.168.1.1 255.255.255.255
class-map type http loadbalance match-any B <<< empty
class-map match-all VIP
  2 match virtual-address 192.168.1.10 tcp eq telnet

policy-map type loadbalance first-match LB
  class A
    serverfarm A
  class B
    serverfarm B
  class class-default
    serverfarm C    

Workaround: In the above configuration, you must add a dummy match statement under class map B. For example:

class-map type http loadbalance match-any B 
  2 match source-address 172.16.27.5 255.255.255.255

CSCso38316—Following negative XML testing, a core dump occurred. The core dump did not cause the ACE module to reload, nor was there any negative impact to the ACE module. Workaround: None.

CSCso38327—While running SSL client authentication, the browser intermittently does not recognize that a certificate has been revoked. Instead, the browser indicates that the server has failed or could not connect. Workaround: None.

CSCso38853—After four consecutive Route Processor Redundancy (RPR) failovers in the Catalyst 6500 series switch, the primary and standby ACEs may enter the Active-Active state. This state is not resolved until you reload the primary ACE. Workaround: None.

CSCso47783—When you configure the ACE for NAT and you are using the NAT counters for troubleshooting, the NAT failure counter does not provide enough granularity for all cases that may cause the counter to increment. Workaround: None.

CSCso55673—Over time, the ACE can leak memory when under a light continuous load of SSL client authentication traffic. The ACE will typically display a log message indicating this low memory condition before the CLI becomes unresponsive and the ACE possibly reloads. The ACE indicates that it is low on directly mapped memory by displaying the following message:

Available CP memory less than 1%: 8380416 bytes. Free high memory: 2093056 bytes. 

Workaround: None.

CSCso60304—When an invalid XML attribute is sent to the ACE, it does not respond as expected. Instead, the ACE displays a 500 Internal Server Error message. No negative impact to the ACE is observed. Workaround: None.

CSCso65486—With the SYN cookie feature configured on an ACE interface that is forwarding nonload-balanced traffic to a routed server, all legitimate traffic that is receiving a SYN cookie is being reset. A packet capture for failed connections shows that the ACE completes a three-way handshake with the client and then with the server before it resets the connection. This behavior may also be observed with load-balanced FTP traffic. Workaround: None.

CSCso69044—With SYN cookie enabled, embryonic connections (incomplete TCP handshakes) may remain on the ACE after more than 24 hours. Workaround: None.

CSCso73385—When you enter the inspect ftp command in a policy map, the ACE resets the FTP connection of the traffic that matches the policy after it sends an extended PASV (EPSV) command to the FTP server. Workaround: None.

CSCso79767—When DNS traffic matches a rule that contains the inspect dns command and the DNS response from the server contains a VIP address, the ACE drops the DNS response. Workaround: Disable the inspect dns command.

CSCso81785—If you are using TACACS+ and the Cisco Access Control Server (ACS) with an RSA authentication manager, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another network access server (NAS) to set your PIN.

CSCso81811—If you are using TACACS+ and the Cisco ACS with an RSA authentication manager and your account is in next token mode, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another NAS to enter the next token code and make your account accessible again.

CSCso85639—If you configure the passdetect interval command value for less than 30 seconds, the ACE sends overlapping probes that use additional management connections (resources). Workaround: Increase the passdetect interval command value to 45 seconds.

CSCso91403—You may observe connection resets when you modify a large configuration. These resets may occur even if you modify a service policy that is not assigned to an interface. Workaround: None.

CSCso93479—The Current Connections counter that is displayed in the output of the show serverfarm name command is not accurate. The output of the show service-policy command does have an accurate counter. Workaround: None.

CSCso95620—With long-lived HTTP, SSL, FTP and UDP traffic running on the ACE, you may observe a memory loss of approximately 333 KB in the ACE during an EtherChannel link (FT port channel) failure and recovery on the Catalyst 6500 series switch. Workaround: None.

CSCsq14440—The aclmerged process in the ACE may not complete or may exceed the available system resources. With very large configurations where there are many ACLs, NAT statements, and class maps, the processing of these elements can require a significant amount of time and internal resources. In some cases, the processing (as displayed by the show proc cpu | include aclmerged command) may become unresponsive and never complete. In other cases, the processing may complete, but the output could exceed the resources available on the ACE, which may cause the ACE to not function properly.

Workaround and recovery: Currently, there is no method to predict the aclmerged response. However, in most cases, the commands eventually complete and the ACE continues to function properly. The suggested workaround is to allow aclmerged to complete without any intervention, assuming that there is no external impact to traffic. If the process does not complete or if there is a significant disruption to traffic flow, then reboot your ACE. If you enter the write memory command prior to the reboot, then the ACE attempts to come up in the post-change configuration. This may allow the desired configuration to be applied properly after the reboot. If you do not enter the write memory command before rebooting the ACE, then the ACE should reload and continue to operate in the same manner as before the change.

CSCsq18476—In a RADIUS authentication configuration, if all of the RADIUS servers fail, the ACE falls back to the local database for authentication even if you change the default from local to the RADIUS servers. For example:

10.10.10.10 key 7 "abc" authentication accounting radius-server host
10.10.10.11 key 7 "abc" authentication accounting aaa group server
radius RADIUS_SERVERS
  server 10.10.10.10
  server 10.10.10.11
aaa authentication login default group RADIUS_SERVERS < not have local option
aaa authentication login console group RADIUS_SERVERS < not have local option

Workaround: None.

CSCsq23701—After an FT VLAN failure, which resulted in an Active/Active FT state, has been resolved, the ACE with the higher priority should take over as the active ACE (even though the preempt command is disabled), through the election process, but did not. Workaround: Enable the preempt command.

CSCsq23888—When you create a scripted probe that contains a VSH configuration command on the active ACE in a redundant configuration, the probe may fail with the "Internal error: Script error" error message on the standby ACE. The configuration commands are executed on the active ACE and then replicated on the standby ACE. If ft auto-sync running-config is disabled on the active ACE, the scripted probe executes properly on the active ACE but will fail on the standby ACE. Workaround: Enable ft-auto sync running-config on the active ACE.

CSCsq24595—While running software version A2(1.0a), the ACE may become unresponsive. No recent configuration changes can be identified. Workaround: Disable connection reuse.

CSCsq25300—When you configure fastpath logging to a syslog host in the ACE, the connection setup and teardown messages that are sent to the syslog server may contain an incorrect duration time stamp and may be formatted improperly. Workaround: None.

CSCsq27062—After toggling the state of the FT port channel in the Catalyst 6500 series switch 110 times, the primary ACE module generated a core dump and reloaded. Workaround: None.

CSCsq81407—When you enter the show svclc module 3 traffic command on a Cisco Catalyst 6500 switch with an ACE module and a high traffic rate, after ten minutes, the byte counts are not correct in the output of the command. Workaround: None.

CSCsq87162—SSL transactions may not complete when the server-conn reuse command is enabled. Workaround: Disable the server-conn reuse command.

CSCsq87869—The show conn display 1000 detail command is part of the showtech script. This command should limit its output to the first 1000 connections, but there are always more connections in the output of the command if there are more than 1000 connections on the ACE. Workaround: None.

CSCsq99448—An ACE was upgraded from version A1(6.3a) to A2(1.1) and experienced two episodes of unresponsiveness in the outbound connection manager (OCM) because of the deletion of an improper internal message. Workaround: None.

CSCsr00851—A user with the configured RBAC user role of Network-Monitor is allowed to delete other users directories on the ACE Flash memory. Workaround: Do not configure users with the Network-Monitor role.

CSCsr09129—When loadbalancing SIP with inspection enabled, the ACE should open a pinhole to the address in the Via header for the server response. However, the server responses remain in the data channel. Workaround: None.

CSCsr13505—If you attempt to authenticate users with a RADIUS server that does not support the authenticate-only service type, that RADIUS server does not pass role and domain information to the ACE during login authentication. Workaround: Use a RADIUS server that supports the authenticate-only service type.

CSCsr14898—XML output for the show serverfarm detail command is not valid XML. If the server farm does not have a configured probe, the generated XML output still contains a close tag </sf_probes> and does not have an open tag <sf_probes>. Workaround: Configure a probe in the serverfarm. If a probe is configured on the server farm, then there should be both an open tag and a close tag present in the XML output. If a probe is not configured on the server farm, then neither tag should be present.

CSCsr16179—When system logging is enabled with the logging fastpath command, IP addresses in the Built TCP Connection syslog messages may be incorrectly swapped. Workaround: None.

CSCsr16201—Built TCP Connection and Teardown TCP Connection syslog messages may continue to be sent to a syslog server even after disabling this functionality with the no logging fastpath command. Workaround: Set logging trap 4.

CSCsr18029—The ACE may reload after an SNMP query. Workaround: None.

CSCsr22703—The ACE became unresponsive and generated a core dump while it was executing an OS kernel function. This behavior appears to have been a one-time event. Workaround: None.

CSCsr28182—When a class-map any command is combined with a class-map all command and more than ten header matches of the same type are used, the ACE CLI displays the following error message:

Error: Maximum 10 http header map is allowed per policy!

Workaround: Decrease the number of header matches to 10 or less by using regular expressions (regexes).

CSCsr38682—The ACE CLI allows you to configure overlapping IP addresses for both an alias IP address and a VIP address. When you add a service policy to an interface with the overlapping VIP and alias IP addresses, the ACE displays the following error message:

Error: vip address duplicates with an existing interface ip address!

However, if you remove the alias IP address and add the service policy to the interface, and then reconfigure the same alias IP address, the ACE allows the configuration. Also, when you reboot the ACE with this configuration, you receive the "*** cmd exec error ***" parser error and the ACE removes the service policy from the interface. Workaround: Ensure that the VIP address and the alias IP address are unique within the context.

CSCsr46740—If you configure FTP inspection and source NAT on the ACE, the ACE does not fix up the IP address of the original client when it forwards the PORT request to the backend server. Workaround: If possible, avoid configuring FTP inspection and source NAT in the same context.

CSCsr89311—An incompatibility exists between certain ACE software versions in the 3.0(0)A1.6.3x and A2.1x release trains. In a redundant configuration, the FT ACE pairs will not recognize each other and will report the following status as part of the show ft peer detail command output:

SRG Compatibility : INCOMPATIBLE.

The following software version combinations that are marked with an "x" are incompatible:

A1(6.3x) Release
A2(1.0)
A2(1.0a)
A2(1.1)
A2(1.1a)

3.0(0)A1(6.3b)

x

 

x

x

3.0(0)A1(6.3c)

x

x

x

x


Software Version A2(1.1) Resolved Caveats, Open Caveats, and Command Changes

The following sections contain the resolved and open caveats in software version A2(1.0):

Software Version A2(1.1) Resolved Caveats

Software Version A2(1.1) Open Caveats

Software Version A2(1.1) Command Changes

Software Version A2(1.1) Resolved Caveats

The following resolved caveats apply to software version A2(1.1):

CSCsk36611—If you are using Internet Explorer (IE), an SSL rehandshake may fail if the total length of the SSL certificate chain is greater than 4024 bytes. When this condition exists, the ACE creates two SSL records. The first record has a total length field that indicates 4024 bytes, but actually contains a certificate item with a specified length that is greater than 4024 bytes. The second record contains a new SSL record header and the remaining portion of the previous SSL record. Workaround: Use Firefox or another browser.

CSCsk83277—When you enter the show run command, sometimes the ACE places two blank lines between the resource-class portion of the running-config and the TACACS+ portion of the running-config, but sometimes it inserts only one blank line. Workaround: None.

CSCsl33851—When you configure an action list to insert, rewrite, or delete HTTP headers, and then you enable HTTP persistence rebalance, you may encounter unexpected behavior when using the ACE to insert headers on both request and response for HTTP requests when there are a large number of header insert commands in the action list (for example 50 statements). Workaround: None.

CSCsl80651—When you configure a large number of scripted probes across multiple user contexts (for example, a total of 255 scripted probes running across 5 user contexts), you may find that the ACE stops sending probes to the real servers or that certain scripted probes remain in the INIT state. This behavior may occur when more than 200 scripted probe instances are running. When this behavior occurs, if you display configuration information and statistics for a probe by using the show probe detail command, the probe states for the probe instances appear to remain in the INIT state. Workaround: None.

CSCsm10896—Enabling the persistence rebalance command while a server farm is configured with any of the hash-based load-balancing predictors may cause the ACE to ignore the hash-based predictor after the first HTTP GET request. Instead, the ACE continues to load balance requests to the same real server that it previously selected if the requests match the same Layer 7 policy. Otherwise, the ACE load balances requests to a new real server in the server farm. Workaround: None.

CSCsm40004—When you are creating a certificate signing request (CSR) in the ACE, the CLI does not allow you to use the space character in the State name. Workaround: Use the state name abbreviation.

CSCsm50232—The ACE may truncate the output of the show service-policy command and display the following error message: "Unexpected header: 4." Workaround: Use the more specific show service-policy policy-name command instead.

CSCsm52480—All IPv6 multicast packets are dropped by the ACE even though the module is properly configured:

switch/Admin# access-list acl_1 ethertype permit ipv6 

This behavior is observed only with IPv6 multicast packets and does not occur with IPv6 unicast packets. Workaround: None.

CSCsm64646—If all or the majority of the configured real servers that receive health probes from the ACE become unreachable at the same time (for example, you shut down the interface on the Catalyst 6500 series switch), the probe states for all the instances do not immediately change to the FAILED state. With a significant number of active probe instances (for example, 8000), it may take a lengthy period of time, sometimes hours, for all the probes to move to the FAILED state. In this case, those probes appear to be stuck in their original state while the ACE generates numerous out-of-socket errors. Workaround: To reduce the severity of the issue, configure a large value for the passdetect interval with respect to the configured probe interval. In addition, configure a small value for the retry count for failed probes using the faildetect command. For example:

switch/Admin(config-probe-tcp)# passdetect interval 300
switch/Admin(config-probe-tcp)# probe interval 30
switch/Admin(config-probe-tcp)# faildetect 2

CSCsm70474—Occasionally, with any SSL service configured and a certain pattern of SSL traffic going to that service, the ACE runs out of packet buffers and stops servicing connections. This behavior depends on the internal timing that is affected by the traffic pattern. Workaround: None.

CSCsm71444—In a multi-level backup real server setup, the ACE may incorrectly place a real server into the OPERATIONAL state when you enter the no inservice standby command. For example, a multi-level backup configuration includes a server farm that is configured with three real servers (RS1, RS2, and RS3) in a chain (RS1 has RS2 as its backup real server and RS2 has RS3 as its backup real server). In this configuration, if you enter the no inservice standby command to take real server RS2 out of service, the ACE will incorrectly place real server RS2 into the OPERATIONAL state instead of placing it into the OUTOFSERVICE state without breaking the chain (RS1->RS2->RS3). As a result, the multi-level backup chain of RS1->RS2->RS3 is divided into two real server backup configurations: RS1 and RS2->RS3. If you enter the inservice standby command to place real server RS2 in the STANDBY state, the ACE does not retain the chain. Workaround: In a multi-level backup real server setup, do not enter the no inservice standby command or the no inservice command to remove the real server from the inservice state.

CSCsm77269—A portion of a policy-map configuration may disappear from the running-config even if you are not actively making any manual configuration changes at the time. During the time that the configuration information is missing, the ACE may incorrectly process the traffic. Workaround: Reboot the ACE to restore its normal operation.

CSCsm79292—When you attempt to use the vsh-conf-cmd TCL command to execute the command or set of commands specified in the preceding set command string (cmd_str) by invoking the Vegas shell (Vsh), the scripted probe may fail and the following error message appears:

Internal error: Script error

For example:

set cmd_str "rserver rs \n inservice"
vsh_conf_cmd $cmd_str

This behavior can occur when you create a scripted probe in which vsh-conf-cmd is specified on the active ACE. The configuration commands are executed on the active ACE and then replicated on the standby ACE. When this behavior occurs, the scripted probe executes properly on the active ACE but may fail on the standby ACE because configuration mode is disabled. Workaround: None.

CSCsm90293—With an SMTP probe configured on the ACE, a new mail server rejects the probe as syntactically invalid because of the use of an underscore (_) that breaks the new rules in RFC 2821 and causes the probed server to never become active.

This behavior is not observed with older mail servers that still adhere to or permit the SMTP command arguments accepted by RFC 821. Workaround: None.

CSCso00356—If you are configuring the SYN cookie feature in a bridged VLAN, you may find that some client connections may become unresponsive. The ACE can exhibits this behavior when all of the following conditions are true:

The VLAN interface is configured as part of a bridge group.

The traffic does not match any configured load-balancing service policy.

The SYN cookie feature is configured and is intercepting SYN packets, which means that the number of embryonic connections is equal to the configured SYN cookie embryonic connection threshold.

The client's ARP entry is not yet learned by the ACE.

Workaround: The unresponsive client connection behavior stops once the client's ARP entry has been populated in the ACE module ARP table.

CSCso02922—If the disk0: directory of the Admin context consumes all the ACE Flash memory space, an SSL certificate cannot be imported into a user context. As a result, some of the directories that are needed for a context to work cannot be created.

Workaround:

1. Free up some Flash memory space by deleting unnecessary files in the disk0: directory.

2. Remove and readd the context that was impacted.

CSCso12722—When you configure the ACE for SSL termination and a client sends a POST request, the request does not fully traverse the ACE to the real server. This behavior occurs with Layer 7 rules when the server MSS is lower than the client MSS and results in a timeout from the real server, which is waiting for the rest of the POST. Workaround: Configure a parameter map to set the ACE MSS to the same value as the server.

CSCso20415—After you import into an ACE context a certificate that contains the special character ampersand (&) in any of the fields of the certificate subject, the synchronization between the ACE and the ANM for that context fails. This behavior is caused by the ACE XML response to the show crypto certificate command because it does not translate the & to the corresponding escape character &amp;. The error causes the ANM to display the following error message:

Device discovery failed: Exception occurred for model:CryptoCertificateModel The 
entity name must immediately follow the '&' in the entity reference.

Workaround: None.

CSCso25654—When the UDP probe interval is set to 2 seconds, UDP probes take longer than expected to enter the failed state. Workaround: Use a time interval that is greater than or equal to 5 seconds for UDP probes.

CSCso28789—A syslog server that is receiving messages from the ACE fails to display messages that are formatted incorrectly. A few ACE syslog messages are missing a colon after the syslog message ID number. An example of an incorrectly formatted message is as follows:

%ACE-3-251008 Health probe failed for server....

The format should include a colon after the ID number as follows:

%ACE-3-251008: Health probe failed for server....

Workaround: View the local syslog buffer on the ACE by using the show logging history command.

CSCso28992—The ACE may fail to download the certificate revocation list (CRL). A sniffer trace shows that the CRL was received, but the ACE did not acknowledge (ACK) the CRL packets before it closed the connection. Workaround: None.

CSCso35927—When a CRL update occurs, the ACE replaces the existing CRL even if there is a download failure. Because the default behavior of the ACE is to allow expired (passed its update interval) CRLs to be used, the existing one should remain if there is a download failure. Workaround: None.

CSCso38903—After four consecutive Route Processor Redundancy (RPR) failovers in the Catalyst 6500 series switch, a core dump may occur on the standby ACE module. The core dump does not cause the ACE module to reload, nor is there any negative impact to the ACE module. Workaround: None.

CSCso65370—In an SSL initiation configuration, the back-end SSL connections may fail after the server has sent its certificate to the ACE, which, in this case, is acting as a client. The client is always responsible for verifying the certificate presented by the server. One of these checks is to make sure that the server certificate has not expired. If you analyze the connection, you should see a Certificate Expired alert. This behavior occurs because the back-end server is attempting to use a certificate that has expired. Workaround: Change the server certificate to one that has not expired.

CSCso45260—While attempting to troubleshoot a TACACS+ problem in a user context using debug logging, the debug log messages for both TACACS+ and AAA are not observed on either the console or in a Telnet session. The debug log messages are seen only in the Admin context. Workaround: Use the debug commands in the Admin context only.

CSCso60015—If you change a nonsticky server farm to a sticky server farm, the ACE may immediately become unresponsive when it receives a single connection. Workaround: Ensure that there is no traffic destined to the ACE for a minimum of 10 seconds after you modify the server farm.

CSCso60096—On rare occasions, the ACE generates a core dump and reloads when you configure the header name header-value value command under an HTTP probe configuration. This behavior occurs only when the header name string is at the end of the page in the memory. The ACE displays an error message similar to the following:

Apr 4 2008 08:28:24 Admin: %ACE-2-443001: System experienced fatal failure.Service name:cfgmgr(928) has terminated on receiving signal 11,reloading system Service name:cfgmgr(928) has terminated on receiving signal 11 6K-1_ACE2-1 login: dir coApr 4 2008 08:28:45 Admin: %ACE-2-443001: System experienced fatal failure.Service name:cfgmgr(928) crashed, last core saved,reloading system re: Password: Login incorrect 6K-1_ACE2-1 login: admApr 4 2008 08:28:56 Admin: %ACE-2-199006: Orderly reload started at Fri Apr 4 08:28:55 2008 by System. Reload reason: Service "cfgmgr"

Workaround: None.

CSCso60682—While toggling the state of interfaces and probes to force an FT tracking failover, the standby ACE may reboot. Workaround: None.

CSCso66799—If you configure the same IP address with different ports on multiple class maps and your application requires that the VIP is pingable when it is active, you must enter the loadbalance vip icmp-reply active command under all class maps that share that same VIP. If you have multiple rules with the same IP address and you enter the loadbalance vip icmp-reply active command only under some of the class maps in a policy map, the ACE may not respond even if the VIPs configured with the loadbalance vip icmp-reply active command are alive. Workaround: Enter the loadbalance vip icmp-reply active command under all class maps that have the same IP address in a policy map.

CSCso74209—When you enable the logging buffered command, you can observe the messages in the output of the show logging command, but eventually the messages stop. When the logging buffer is full and starts to wrap, the ACE stops logging the new messages. Workaround: Enter the clear logging command to clear the buffer and start logging again. The clear logging command will delete all existing syslog messages.

CSCso74865—If you make load-balancing-related configuration changes (for example, changing the server farm predictor or configuring real server connection limits), the load-balancing (LB) process may fail while it is processing a list that tracks real servers that are currently not used in the LB decision. You may also observe this behavior when a real server goes into or out of the MAXCONNS state. Workaround: None.

CSCso88288—A buffer leak may occur when you use an HTTPS probe if the connection requires a rehandshake. Workaround: Use a Layer 4 TCP probe instead.

CSCsq04822—The ACE becomes unresponsive when a real server reaches its configured maximum connection limit. Workaround: Increase the maximum connection value of the server so that the server never reaches the maximum connection limit or remove the conn-limit command from the real server configuration.

CSCsq30454—When SYN cookie is enabled, all routed TCP traffic passing through the ACE is reset. Workaround: None.

Software Version A2(1.1) Open Caveats

The following open caveats apply to software version A2(1.1):

CSCsi13378—If you configure certain commands in the ACE (for example, object-group, action-list, and so on) and you enable the xml-show command, the output of the show running-config command displays data outside the XML tags or incorrect XML tags. Workaround: None.

CSCsj68643—The following log messages may appear sporadically in the ACE log:

can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.

can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.

These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.

CSCsj74250—When you configure the TACACS+ server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, then there is a key mismatch when the ACE attempts to authenticate a user. Workaround: Paste the properly encrypted key into the running-configuration file.

CSCsj94366—When you attempt to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears:

console configuration can only be done on console

Workaround: None.

CSCsl21191—When you enter the show module command on the supervisor engine for a running ACE, the command output may fail to display the software version information from the ACE. When this behavior occurs, the command output displays similarly to the following example output:

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
 4  0018.b9a6.88fc to 0018.b9a6.8903   1.1   8.6(0.252-En 8.6(0.252-En Ok  

This behavior rarely occurs, but once it does, the behavior will continue every time that you enter the show module command. The ACE continues to forward traffic normally. This is a display problem only. Workaround: Reboot the ACE.

CSCsl46334—When a high rate of Layer 7 load-balanced traffic is flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse is configured, the ACE may start dropping traffic after a few hours. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health checks. Workaround: None.

CSCsl75662—You may observe that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time that the probe is sent even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from the real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsm46044—ACL-MERGE-ERROR messages may be logged as follows:

ACL-MERGE-ERROR:cannot find ACL in acl_merge_rem_acl_from_list 
../security/acl/acl_merge.c:xxx

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform either of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, then log in again to access a different context than the original context with the configured packet capture function.

CSCso00234—After the ClientHello and the ServerHello, the ACE responds to the client with the Fatal, description:Bad Record MAC alert. Currently, the ACE cannot process non-minimally padded block ciphers, which is a TLS 1.0 feature. You employ non-minimally padded block ciphers in the following situations:

You use TLS version 1.0.

You negotiate a block cipher (AES256).

The Finished message is 256 bytes.

Workaround: If possible, restrict the SSL protocol version to SSL version 3. Alternatively, allow only stream ciphers, such as RC4.

CSCso12560—The show resource usage command may display a nonzero number for some resources that have their maximum value set to equal-to-min. Workaround: None.

CSCso19129—When the ACE is configured for load balancing or server-side NAT and RTSP inspection is enabled, the Windows Media Server may reject an RTSP session if you are using Windows Media player 10. If a real server IP address to VIP (or vice versa) translation is required, the ACE translates the IP address in the SDP part but does not update the content length in the header part. The message is then rejected by the server. This behavior does not occur if the real address length and VIP address length is the same. If the IP address length does not change between the VIP (mapped address) and the real server (real address), then this behavior does not occur. Workaround: None.

CSCso21587—When RTSP inspection is enabled and the ACE performs load balancing or destination NAT on an RTSP session, the Windows Media Server may reject the session. If the media stream is interleaved with the RTSP control connection (the transport type is RTP/TCP or RDT/TCP), then the ACE incorrectly unproxies the control connection as soon as it detects the transport type as TCP, causing the rest of the messages between the client and the server to pass through the ACE without any inspection. As a result, subsequent SETUP messages are not fixed up (NATed) and the server rejects the SETUP message with the VIP address (instead of the real server address) in it. Workaround: Use UDP instead of TCP as the transport mode for the media streams.

CSCso22472—When you use class maps of type http loadbalance match-any to select a server farm and some of these class maps are empty, the ACE may make an incorrect load-balancing (LB) decision. This incorrect LB decision causes unexpected LB results. For example:

class-map type http loadbalance match-any A
  2 match source-address 192.168.1.1 255.255.255.255
class-map type http loadbalance match-any B <<< empty
class-map match-all VIP
  2 match virtual-address 192.168.1.10 tcp eq telnet

policy-map type loadbalance first-match LB
  class A
    serverfarm A
  class B
    serverfarm B
  class class-default
    serverfarm C    

Workaround: In the above configuration, you must add a dummy match statement under class map B. For example:

class-map type http loadbalance match-any B 
  2 match source-address 172.16.27.5 255.255.255.255 

CSCso38316—Following negative XML testing, a core dump occurred. The core dump did not cause the ACE module to reload, nor was there any negative impact to the ACE module. Workaround: None.

CSCso38327—While running SSL client authentication, the browser intermittently does not recognize that a certificate has been revoked. Instead, the browser indicates that the server has failed or could not connect. Workaround: None.

CSCso38853—After four consecutive Route Processor Redundancy (RPR) failovers in the Catalyst 6500 series switch, the primary and standby ACEs may enter the Active-Active state. This state is not resolved until you reload the primary ACE. Workaround: None.

CSCso47783—When you configure the ACE for NAT and you are using the NAT counters for troubleshooting, the NAT failure counter does not provide enough granularity for all cases that may cause the counter to increment. Workaround: None.

CSCso55673—Over time, the ACE can leak memory when under a light continuous load of SSL client authentication traffic. The ACE will typically display a log message indicating this low memory condition before the CLI becomes unresponsive and the ACE possibly reloads. The ACE indicates that it is low on directly mapped memory by displaying the following message:

Available CP memory less than 1%: 8380416 bytes. Free high memory: 2093056 bytes 

Workaround: None.

CSCso60304—When an invalid XML attribute is sent to the ACE, it does not respond as expected. Instead, the ACE displays a 500 Internal Server Error message. No negative impact to the ACE is observed. Workaround: None.

CSCso65486—With the SYN cookie feature configured on an ACE interface that is forwarding nonload-balanced traffic to a routed server, all legitimate traffic that is receiving a SYN cookie is being reset. A packet capture for failed connections shows that the ACE completes a three-way handshake with the client and then with the server before it resets the connection. This behavior may also be observed with load-balanced FTP traffic. Workaround: None.

CSCso69044—With SYN cookie enabled, embryonic connections (incomplete TCP handshakes) may remain on the ACE after more than 24 hours. Workaround: None.

CSCso73385—When you enter the inspect ftp command in a policy map, the ACE resets the FTP connection of the traffic that matches the policy after it sends an extended PASV (EPSV) command to the FTP server. Workaround: None.

CSCso79767—When DNS traffic matches a rule that contains the inspect dns command and the DNS response from the server contains a VIP address, the ACE drops the DNS response. Workaround: Disable the inspect dns command.

CSCso80478—When you perform multiple parallel SNMP walks that last thirty seconds or longer on an ACE in a redundant configuration, you may observe response timeouts on both the active and the standby ACEs. You may also observe this behavior in multiple contexts. This behavior does not occur with SNMP walks of shorter durations. Workaround: None.

CSCso81785—If you are using TACACS+ and the Cisco Access Control Server (ACS) with an RSA authentication manager, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another network access server (NAS) to set your PIN.

CSCso81811—If you are using TACACS+ and the Cisco ACS with an RSA authentication manager and your account is in next token mode, you may receive the Login Incorrect message when you try to log in to the ACE with an account that requires a new PIN even if you use the correct credentials. Workaround: Log in to another NAS to enter the next token code and make your account accessible again.

CSCso82971—If you are using a TACACS+ server that is an RSA server with TACACS+ continue authentication, authentication may fail to the configured server, but you still can log in using local authentication.

Use one of the following workarounds:

Use the Cisco ACS instead of the RSA server.

Do not configure local as the secondary authentication method.

CSCso85639—If you configure the passdetect interval command value for less than 30 seconds, the ACE sends overlapping probes that use additional management connections (resources). Workaround: Increase the passdetect interval command value to 45 seconds.

CSCso86485—When a client-side VLAN interface is brought up and down an excessive number of times on the active ACE under a light traffic load, the standby ACE may generate a core dump. Workaround: None.

CSCso91403—You may observe connection resets when you modify a large configuration. These resets may occur even if you modify a service policy that is not assigned to an interface. Workaround: None.

CSCso93479—The Current Connections counter that is displayed in the output of the show serverfarm name command is not accurate. The output of the show service-policy command does have an accurate counter. Workaround: None.

CSCso95457—When you enter the clear conn all command, the ACE sends an RST to close the connection only to the server and purges both the inbound and outbound connection entries from its connection database. As a result, the client connection is left open and any further requests arriving on that connection are not serviced. Workaround: None.

CSCso95620—With long-lived HTTP, SSL, FTP and UDP traffic running on the ACE, you may observe a memory loss of approximately 333 KB in the ACE during an EtherChannel link (FT port channel) failure and recovery on the Catalyst 6500 series switch. Workaround: None.

CSCsq14440—The aclmerged process in the ACE may not complete or may exceed the available system resources. With very large configurations where there are many ACLs, NAT statements, and class maps, the processing of these elements can require a significant amount of time and internal resources. In some cases, the processing (as displayed by the show proc cpu | include aclmerged command) may become unresponsive and never complete. In other cases, the processing may complete, but the output could exceed the resources available on the ACE, which may cause the ACE to not function properly.

Workaround and recovery: Currently, there is no method to predict the aclmerged response. However, in most cases, the commands eventually complete and the ACE continues to function properly. The suggested workaround is to allow aclmerged to complete without any intervention, assuming that there is no external impact to traffic. If the process does not complete or if there is a significant disruption to traffic flow, then reboot your ACE. If you enter the write memory command prior to the reboot, then the ACE attempts to come up in the post-change configuration. This may allow the desired configuration to be applied properly after the reboot. If you do not enter the write memory command before rebooting the ACE, then the ACE should reload and continue to operate in the same manner as before the change.

CSCsq18476—In a RADIUS authentication configuration, if all of the RADIUS servers fail, the ACE falls back to the local database for authentication even if you change the default from local to the RADIUS servers. For example:

10.10.10.10 key 7 "abc" authentication accounting radius-server host
10.10.10.11 key 7 "abc" authentication accounting aaa group server
radius RADIUS_SERVERS
  server 10.10.10.10
  server 10.10.10.11
aaa authentication login default group RADIUS_SERVERS < not have local option
aaa authentication login console group RADIUS_SERVERS < not have local option

Workaround: None.

CSCsq23701—After an FT VLAN failure, which resulted in an Active/Active FT state, has been resolved, the ACE with the higher priority should take over as the active ACE (even though the preempt command is disabled), through the election process, but did not. Workaround: Enable the preempt command.

CSCsq23888—When you create a scripted probe that contains a VSH configuration command on the active ACE in a redundant configuration, the probe may fail with the "Internal error: Script error" error message on the standby ACE. The configuration commands are executed on the active ACE and then replicated on the standby ACE. If ft auto-sync running-config is disabled on the active ACE, the scripted probe executes properly on the active ACE but will fail on the standby ACE. Workaround: Enable ft-auto sync running-config on the active ACE.

CSCsq24595—While running software version A2(1.0a), the ACE may become unresponsive. No recent configuration changes can be identified. Workaround: Disable connection reuse.

CSCsq25300—When you configure fastpath logging to a syslog host in the ACE, the connection setup and teardown messages that are sent to the syslog server may contain an incorrect duration time stamp and may be formatted improperly. Workaround: None.

CSCsq27062—After toggling the state of the FT port channel in the Catalyst 6500 series switch 110 times, the primary ACE module generated a core dump and reloaded. Workaround: None.

CSCsr89311—An incompatibility exists between certain ACE software versions in the 3.0(0)A1.6.3x and A2.1x release trains. In a redundant configuration, the FT ACE pairs will not recognize each other and will report the following status as part of the show ft peer detail command output:

SRG Compatibility : INCOMPATIBLE

The following software version combinations that are marked with an "x" are incompatible:

A1(6.3x) Release
A2(1.0)
A2(1.0a)
A2(1.1)
A2(1.1a)

3.0(0)A1(6.3b)

x

 

x

x

3.0(0)A1(6.3c)

x

x

x

x


Software Version A2(1.1) Command Changes

Table 10 lists the commands and options that have been changed in software version A2(1.1).

Table 10 New CLI Commands in Version A2(1.1)  

Mode
Command and Syntax
Description

SSL parameter map configuration

authentication-failure ignore

Allows the SSL connection even if the authentication fails. Possible reasons for the authentication failure include:

Certificate has expired

Certificate is not yet valid

Certificate has been revoked

General failure of receiving the certificate

This command and the failure reasons apply to both server certificates and client certificates.


Table 11 lists the commands and options that have been changed in software version A2(1.1).

Table 11 CLI Commands Changed in Version A2(1.1)  

Mode
Command and Syntax
Description

Exec

show sticky database

Duplicated information in the output of this command has been removed to simplify the output. The sticky group, type, timeout, timeout-activeconns, sticky-entry, rserver-instance, and time-to-expire flags fields are displayed only once for each sticky group, rather than duplicating those fields for multiple real servers in the same sticky group.

show sticky database group name

Duplicated information in the output of this command has been removed to simplify the output. The sticky group, type, timeout, timeout-activeconns, sticky-entry, rserver-instance, and time-to-expire flags fields are displayed only once for the specified sticky group rather than duplicating those fields for each real server in the specified sticky group.

Exec (cont.)

show system {internal {aaa {event-history {errors | msgs}} | mem-stats} | log {boot {kickstart | system} | install [details]} | mts {buffers [age seconds | details | node name | order | sap number | summary] | memory | opcode} | radius event-history {errors | msgs} | sysmgr {event-history {errors | msgs} | service {all [detail] | local [detail] | name service_name [dependencies | policies | seqnotbl] | not-running [details] | pid id [config | dependencies | log] | running [details] | uuid hex_id [config | dependencies]} | startup-config {locks | state} | state | time} | tacacs+ event-history {errors | msgs} | urifs | vshd {config-intro | feature-list | license-info | log {running-config | tree-table} | subtype-table | tree-table}}

The log {boot {kickstart | system} | install [details]} option has been removed.

show stats loadbalance rtsp | sip

Added the rtsp and sip options to display load-balancing statistics for the Real-Time Streaming Control Protocol (RTSP) or the Session Initiation Protocol (SIP), including:

Total sessions allocated

Total sessions failed

Total sticky entries added


Software Version A2(1.0a) Resolved and Open Caveats

The following sections contain the resolved and open caveats in software version A2(1.0a):

Software Version A2(1.0a) Resolved Caveats

Software Version A2(1.0a) Open Caveats

Software Version A2(1.0a) Resolved Caveats

The following resolved caveats apply to software version A2(1.0a):

CSCsj29467—UDP traceroute traffic and other traffic destined to an ACE VIP that is not part of a local subnet may cause forwarding loops until the time to live (TTL) expires. If you define an ACE VIP (class map) that is not part of a local subnet, you configure the class map for a specific TCP port or any TCP port, and you perform a UDP traceroute to this VIP, the ACE forwards this traffic to its default gateway. The default gateway, via a static route or a routing protocol, will attempt to forward the request back to the ACE, which may cause forwarding loops. This behavior is observed for VIPs that are configured with an IP address, a protocol, and a port and is not seen for VIPs that have only an IP address.

Perform one of the following workarounds:

Configure the ACE VIP with any protocol (match virtual-address 192.168.12.15 any)

Configure an additional match statement with UDP and a specific port (match virtual-address 192.168.12.15 udp eq sip)

Configure UDP and any port (match virtual-address 192.168.12.15 udp any)

Each of these workarounds cause the VIP to respond to the UDP traceroute with an ICMP Dest Port Unreachable message. If all real servers are down, there will be no response to the traceroute.

Alternatively, you can define an ACL to block all UDP traffic destined to the TCP VIP. This workaround causes the UDP traceroute to fail or to go unanswered.

CSCsm10702—In a redundant configuration the ACE may become unresponsive to network traffic. The FT peer of the ACE assumes mastership, but the affected ACE does not assume Standby status or respond to the network. However, you can access the affected ACE by sessioning in to the module as the default user from the Catalyst 6500 series switch CLI. The frequency of the unresponsive behavior may correlate with logging from the ACE to the Catalyst supervisor engine using the logging supervisor 7 command in configuration mode. Workaround: Decrease the logging level of the logging supervisor command, or avoid logging to the Catalyst supervisor engine. The ACE recovers upon reboot, again becoming responsive to the network and assuming its proper FT status.

CSCsm70574—With the UDP booster feature enabled, the ACE may route load-balanced connections incorrectly. When back-to-back UDP requests arrive at the ACE, the ACE may incorrectly treat the second request as an embryonic hit. Embryonic connection hits may modify the ACL node entry as though it was a dynamic entry, which removes the load-balancing feature for such connections. Workaround: None.

CSCso46209—When you attempt to configure an LDAP server, errors may result. For example, consider the following configuration:

switch/Admin(config)# ldap-server host 10.86.215.90 rootDN "cn=admin,dc=cisco,dc=com" 
password 7 abc0123 port 389 timeout 45 

When you display the configuration in the running-configuration, it appears incorrectly as follows:

switch/Admin# show running-config 
ldap-server host 10.86.215.90 rootDN "cn=admin" password dc=cisco dc=com port 7 
timeout abc0123 

LDAP functions as intended and the LDAP server configuration appears correctly as follows:

switch/Admin# show ldap
timeout : 5
port : 389
total number of servers : 1 
following LDAP servers are configured:
10.86.215.90:
timeout: 45   port: 389   rootDN: cn=admin,dc=cisco,dc=com

In a redundant configuration, configuration synchronization may not complete properly. After a reboot, the ACE may not be able to reapply the LDAP configuration and may produce an error similar to the following:

ldap-server host 10.86.215.90 rootDN "cn=admin" password dc=cisco dc=com port 7 
timeout abc0123`
*** Context 0: cmd parse error *** 

This behavior may result in the secondary ACE entering the STANDBY_COLD state.

To recover from this state, perform the following steps:

1. Enter the no ft auto-sync running-config command to disable auto synchronization.

2. Manually remove the LDAP configuration lines, described earlier.

3. Reenable auto synchronization by entering the ft auto-sync running-config command. The secondary ACE should enter the STANDBY_HOT state after the synchronization completes.

To avoid this behavior, use one of the following workarounds if possible:

Use a rootDN with only a single domain component.

Avoid using the LDAP feature in ACE software releases that exhibit this behavior.

Software Version A2(1.0a) Open Caveats

The following open caveats apply to software version A2(1.0a):

CSCsj41909—The ACE fails with the following message appearing in the syslog:

%ACE-5-199006: Orderly reload started at xxxxxxx by System. 

The reload reason is NP 1 Failed: NP Process Crashed. Workaround: None.

CSCsj64833—The network utility traceroute does not work for a configured ACE IP interface if the underlying protocol is UDP or TCP. Workaround: ICMP traceroute will work. The default of most traceroute utilities is UDP so a command line option might be necessary. For Linux, use the -I option.

CSCsj68643—The following log messages may appear sporadically in the ACE log:

can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.

can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.

These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.

CSCsj74250—When you configure the TACACS server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, there is a key mismatch when attempting to authenticate. Workaround: Paste the properly encrypted key into the running-configuration file.

CSCsj94366—While attempting to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: console configuration can only be done on console. Workaround: None.

CSCsk36611—If you are using Internet Explorer (IE), an SSL rehandshake may fail if the total length of the SSL certificate chain is greater than 4024 bytes. When this condition exists, the ACE creates two SSL records. The first record has a total length field indicating 4024 bytes, but containing a certificate item with a specified length greater than 4024 bytes. The second record contains a new SSL record header and the remaining portion of the previous SSL record. Workaround: Use Firefox or another browser.

CSCsk63774—Current connection statistics in the show serverfarm name command output may not coincide with the show serverfarm command output. This behavior may exist even with a minimal amount of traffic. Workaround: None.

CSCsl33851—With an action list configured to insert, rewrite, or delete HTTP headers, and HTTP persistence rebalance is enabled, you may encounter issues when using the ACE to insert headers on both request and response for HTTP requests when there are a large number of header insert command in action list (for example 50 statements). Workaround: None.

CSCsl46334—With a high rate of Layer 7 load-balanced traffic flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse configured, the ACE may start dropping traffic after a few hours. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health check(s). Workaround: None.

CSCsl75662—You may that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time the probe is fired even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to first expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

1. Remove the probe from real server and the server farm.

2. Modify the probe parameter that you want to change.

3. Readd the probe to the real server and the server farm.

For details, refer to the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsl80651—When you configure a large number of scripted probes across multiple user contexts (for example, a total of 255 scripted probes running across 5 user contexts), you may find that the ACE stops sending probes to the real servers or that certain scripted probes remain in the INIT state. This behavior may occur when more than 200 scripted probe instances are running. When this occurs, if you display configuration information and statistics for a probe by using the show probe detail command, the probe states for the probe instances appear to remain in the INIT state. Workaround: None.

CSCsm40004—When you are creating a Certificate Signing Request (CSR) and you attempt to define the state name parameter in the CSR parameter set by using the state command in CSR parameter configuration mode, the ACE does not allow you to use the space character in the state argument. Workaround: Use the abbreviation of the state.

CSCsm43541—The ACE is receiving SSL-related traffic and the module may reboot due to a buffer corruption issue on the crypto chip. Workaround: None.

CSCsm46044—ACL-MERGE-ERROR messages may be logged as follows:

ACL-MERGE-ERROR:cannot find ACL in acl_merge_rem_acl_from_list 
../security/acl/acl_merge.c:xxx

This behavior may be observed when you enable the debug access-list merge errors command in debug mode and then add new configurations to the ACE. Workaround: None.

CSCsm52480—All IPv6 multicast packets are dropped by the ACE even though the module is properly configured:

switch/Admin# access-list acl_1 ethertype permit ipv6 

This symptom is observed only with IPv6 multicast packets and does not occur with IPv6 unicast packets. Workaround: None.

CSCsm62263—The syslogd function in the ACE may become unresponsive and generate a core file. The symptom may occur when you attempt to shut down the ACE by entering the no power enable command in configuration mode at the Catalyst 6500 series switch CLI. Workaround: None.

CSCsm64646—If all or the majority of the configured real servers receiving health probes from the ACE become unreachable at the same time (for example, you shut down the interface on the Catalyst 6500 series switch), the probe states for all the instances do not immediately change to the FAILED state. With a significant number of active probe instances (for example, 8000), it may take a lengthy period of time, sometimes hours, for all the probes to move to the FAILED state. In this case, those probes appear to be stuck in their original state while the ACE generates numerous out-of-socket errors. Workaround: To reduce the severity of the issue, configure a large value for the passdetect interval as compared to the configured probe interval. In addition, configure a small value for the retry count for failed probes using the faildetect command.

For example:

switch/Admin(config-probe-tcp)# passdetect interval 300
switch/Admin(config-probe-tcp)# probe interval 30
switch/Admin(config-probe-tcp)# faildetect 2

CSCsm65534—You may find that sequential readings of the Client Byte Count and the Server Byte Count fields in the show service-policy command output increment or decrement by large values without the expected changes in network traffic. This behavior is a display-only issue and does not affect traffic forwarded by the ACE. You may encounter this behavior after the byte counters exceed the maximum of 4294967295 bytes. Workaround: None.

CSCsm67002—When operating in a redundant configuration, the standby ACE may reboot because one or more of the Micro Engines become unresponsive in the network processor on the ACE. After the standby ACE reboots, if you specify the show ft group command, the My State field in the show output displays the FT status "FSM_FT_STATE_INIT" and the active ACE displays the peer FT state "FSM_FT_STATE_UNKNOWN". You may find that a subsequent reboot of the ACE fails to clear the FT status. Workaround: None.

CSCsm71444—In a multi-level backup real server setup, the ACE may incorrectly place a real server into the OPERATIONAL state when you enter the no inservice standby command. For example, a multi-level backup configuration includes a server farm that is configured with three real servers (RS1, RS2, and RS3) in a chain, such that RS1 has RS2 as its backup real server and RS2 has RS3 as its backup real server. In this configuration, if you enter the no inservice standby command to take real server RS2 out of service, the ACE will incorrectly place real server RS2 into the OPERATIONAL state instead of placing it into the OUTOFSERVICE state without breaking the chain (RS1->RS2->RS3). As a result, the multi-level backup chain of RS1->RS2->RS3 is divided into two real server backup configurations: RS1 and RS2->RS3. If you enter the inservice standby command to place real server RS2 in the STANDBY state, the ACE does not retain the chain. Workaround: In a multi-level backup real server setup, do not enter the no inservice standby command or the no inservice command to remove the real server from the inservice state.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform one of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, then log in again to access a different context than the original context with the configured packet capture function.

CSCsm79292—When you attempt to use the vsh-conf-cmd TCL command to execute the command or set of commands specified in the preceding set command string (cmd_str) by invoking the Vegas shell (Vsh), the scripted probe may fail and the following error message appears: "Internal error: Script error."

For example:

set cmd_str "rserver rs \n inservice"
vsh_conf_cmd $cmd_str

This behavior can occur when you create a scripted probe in which there is vsh-conf-cmd specified on the active ACE. The configuration commands are executed on the active ACE and then replicated on the standby ACE. When this behavior occurs, the scripted probe executes properly on the active ACE, but may fail on the standby ACE because configuration mode is disabled. Workaround: None.

CSCsm90293—With an SMTP probe configured on the ACE, a new mail server rejects the probe as syntactically invalid because of the use of an underscore (_) that breaks the new rules in RFC 2821 and causes the probed server to never become active.

This behavior is not observed with older mail servers that still adhere to or permit the SMTP command arguments accepted by RFC 821. Workaround: None.

CSCso00234—After the ClientHello and the ServerHello, the ACE responds to the client with the Fatal, description:Bad Record MAC alert. Currently, the ACE cannot process non-minimally padded block ciphers, a TLS 1.0 feature. You employ non-minimally padded block ciphers when:

You use TLS version 1.0

You negotiate a block cipher (AES256)

The Finished message is 256 bytes

Workaround: If possible, restrict the SSL protocol version to SSL version 3. Alternatively, allow only stream ciphers, for example, RC4.

CSCso00356—If you are configuring the SYN cookie feature in a bridged VLAN, you may find that some client connections may become unresponsive. The ACE can exhibits this behavior when all of the following conditions are true:

The VLAN interface is configured as part of a bridge group.

Traffic does not match any configured load-balancing service policy.

SYN cookie is configured and is intercepting SYN packets, meaning that the number of embryonic connections is equal to the configured SYN cookie embryonic connection threshold.

The client's ARP entry is not yet learned by the ACE.

Workaround: The unresponsive client connection behavior stops once the client's ARP entry has been populated in the ACE module ARP table.

CSCso02922—If the disk0: directory of the Admin context consumes all the ACE Flash memory space, an SSL certificate cannot be imported into a user context. As a result, some of the directories that are needed for a context to work cannot be created.

Workaround:

1. Free up some Flash memory space by deleting unnecessary files in the disk0: directory.

2. Remove and readd the context that was impacted.

CSCso12722—When you configure the ACE for SSL termination and a client sends a POST request, the request does not fully traverse the ACE to the real server. This behavior occurs with Layer 7 rules when the server MSS is lower than the client MSS and results in a timeout from the real server, which is waiting for the rest of the POST. Workaround: Configure a parameter map to set the ACE MSS to the same value as the server.

CSCso18391—When you perform a checkpoint rollback and enter the no nat dynamic n vlan nnn CLI command, the ACE displays the following messages:

Generating configuration....
Errors while applying diff, please see log below.
Failed Command:
policy-map multi-match xxxxx
class xxxxxx
no nat dynamic 1 vlan 150
Failure Reason:
Error: Called API timed out

This behavior appears to be related to larger configurations with many regular expressions (regexes). Workaround: Enter the no nat dynamic command before you perform the rollback operation.

CSCso20415—After you import into an ACE context a certificate containing the special character ampersand (&) in any of the fields of the certificate subject, the synchronization between the ACE and ANM for that context fails. This behavior is caused by the ACE XML response to the show crypto certificate command's not translating the special reference "&" to the corresponding escape character "&amp;". The error causes ANM to display the following error message:

Device discovery failed: Exception occurred for model:CryptoCertificateModel The 
entity name must immediately follow the '&' in the entity reference.

Workaround: None.

CSCso22472—When you use class maps of type http loadbalance match-any to select a server farm and some of these class maps are empty, the ACE may make an incorrect load-balancing (LB) decision. This incorrect LB decision causes unexpected LB results. For example:

class-map type http loadbalance match-any A
  2 match source-address 192.168.1.1 255.255.255.255
class-map type http loadbalance match-any B <<< empty
class-map match-all VIP
  2 match virtual-address 192.168.1.10 tcp eq telnet

policy-map type loadbalance first-match LB
  class A
    serverfarm A
  class B
    serverfarm B
  class class-default
    serverfarm C    

Workaround: In the above configuration, you must add a dummy match statement under class map B. For example:

class-map type http loadbalance match-any B 
  2 match source-address 172.16.27.5 255.255.255.255 

CSCso25654—With the UDP probe interval set to 2 seconds, UDP probes take longer than expected to enter the failed state. Workaround: Use a time interval that is greater than or equal to 5 seconds for UDP probes.

CSCso66799—If you configure the same IP address with different ports on multiple class maps and your application requires that the VIP is pingable when it is active, you must configure the loadbalance vip icmp-reply active command under all class maps that share that same VIP. If you have multiple rules with the same IP address and you configure the loadbalance vip icmp-reply active command only under some of the class maps in a policy map, the ACE may not respond at all even if the VIPs configured with the loadbalance vip icmp-reply active command are alive. Workaround: Configure the loadbalance vip icmp-reply active command under all class maps that have the same IP address in a policy map.

CSCso73385—With inspect ftp configured on a policy map, the ACE resets the FTP connection of traffic that matches the policy after it sends an extended PASV (EPSV) command to the FTP server. Workaround: None.

CSCso74865—If you make load-balancing-related configuration changes (for example, changing the server farm predictor or configuring real server connection limits), the load-balancing (LB) process may fail while it is processing a list related to keeping track of real servers that are currently not used in the LB decision. You may also observe this behavior when a real server goes into or out of the MAXCONNS state. Workaround: None.

CSCsr89311—An incompatibility exists between certain ACE software versions in the 3.0(0)A1.6.3x and A2.1x release trains. In a redundant configuration, the FT ACE pairs will not recognize each other and will report the following status as part of the show ft peer detail command output:

SRG Compatibility : INCOMPATIBLE

The following software version combinations that are marked with an "x" are incompatible:

A1(6.3x) Release
A2(1.0)
A2(1.0a)
A2(1.1)
A2(1.1a)

3.0(0)A1(6.3b)

x

 

x

x

3.0(0)A1(6.3c)

x

x

x

x


Software Version A2(1.0) Resolved Caveats and Open Caveats

The following sections contain the resolved and open caveats in software version A2(1.0):

Software Version A2(1.0) Resolved Caveats

Software Version A2(1.0) Open Caveats

Software Version A2(1.0) Resolved Caveats

The following resolved caveats apply to software version A2(1.0):

CSCsh14278—A reset for an established connection was being accounted for as a connection failure. Workaround: None.

CSCsh70258—SSL initiation configured on an ACE may fail when it is used with Microsoft IIS 6.0 that is configured to accept client certificates. The additional SSL communication from the server to the ACE causes the ACE to experience an internal error. Workaround: In the IIS Web Server Properties > Directory Security > Edit menu, specify Ignore client certificates.

CSCsj38511—The ACE may reboot when there are a large number of scripted probes running across multiple user contexts (for example, a total of 255 scripted probes running across 5 contexts). Workaround: None.

CSCsj95429—Probe failures may be observed intermittently in a configuration with a large number of UDP or UDP-based probes running. To check this behavior, use the show probe detail command in Exec mode. You would observe that the counter associated with the No. Out of Sockets field increments. This behavior may occur when the probe time interval is set too small (for example, 5 seconds) and the number of probe instances is greater than 4000. Workaround: None.

CSCsk02170—If you configure a less-specific route before a more-specific route, ACE-originated traffic may not honor the more specific route. For example, if you configure the following routes in the order shown:

Admin/host(config)# ip route 1.0.0.0 255.0.0.0 10.0.0.1
Admin/host(config)# ip route 1.1.1.1 255.255.255.255 192.168.0.1

The output of the show ip route command displays the route for 1.1.1.1, but the ACE may continue to use the 10.0.0.1 next hop. This behavior is dependent on the order in which the routes are configured on the ACE and affects ACE-originated traffic only.

Workaround: To restore routing as expected from the show ip route command output, perform the following steps:

1. Configure a less-specific dummy route.

2. Remove the less-specific route.

This sequence has no net effect on the routing table, but it does cause a refresh of the cache information that corrects the routing problem.

CSCsk17904—You are using the packet capture function on all interfaces and the resulting packet capture file created is corrupted and cannot be read by applications such as the Ethereal network protocol analyzer. Workaround: None.

CSCsk20290You are using the packet capture function on all interfaces and there are incorrect packet details in the associated capture output. This behavior may occur due to packet corruption; the first packet is incorrectly prepared by the packet capture function. Workaround: None.

CSCsk33588—Health probes may remain in an Invalid state after you repeatedly create and delete user contexts that contain the health probes. Use the show probe detail command in Exec mode to display configuration information and statistics for a probe in a user context. Workaround: None.

CSCsk34767—CSCsh63231 added FTP inspection statistics to the show stats inspect command output, but the statistics cannot be cleared using the clear stats inspect command. Workaround: None.

CSCsk34843—The following messages may be observed on the console connection:

"Data bus error, epc == 2b5008e0, ra == 2b5008c8
badvaddr 0x00005f00 cause 0x8cd8fd18 status 0x00007fff<1>A_SCD_BUS_ERR_STATUS 
0x810c0000  
A_BUS_ERR_DATA_0 0x0  
A_BUS_ERR_DATA_1 0x0  
A_BUS_ERR_DATA_2 0xffffb12a00000000  
A_BUS_ERR_DATA_3 0xffff000000000000  
A_BUS_L2_ERRORS 0x0  
A_BUS_MEM_IO_ERRORS 0x20000 "

Workaround: None.

CSCsk40308—With NAT configured for a server in a server farm, the ACE may reboot because multiple Micro Engines become unresponsive in the network processor on the ACE. This behavior can be due to an invalid Layer 7 policy ID; the ACE could not successfully locate the correct mapped local ID because the ID Map table does contain the associated mapped entry. Workaround: None.

CSCsk40906—The show interface command output indicates that an associated VLAN was not assigned from the Catalyst supervisor engine to the ACE even though the show vlan command output displays the VLAN IDs on the ACE as downloaded from the supervisor engine. Workaround: None.

CSCsk44035—When persistence-rebalance is enabled and there is an extremely high load on the ACE, buffer allocation failures and slow end-user responses may occur, and the ACE may reboot. Workaround: None.

CSCsk53147—When you configure Session Initiation Protocol (SIP) inspection, if you set up SIP calls at approximately 125 connections per second (cps) and enable system logging with console logging, the ACE may become unresponsive. This behavior can also happen with other high-rate system message logs, such as those generated for HTTP application inspection. Workaround: Do not enable console logging when high-rate data plane syslogs are configured.

CSCsk59271—With 250 contexts configured, if you run a script that removes, adds, and reconfigures one context at a time, the ACE may reboot during the script run in the service config manager of the ACE. Workaround: None.

CSCsk62389—The ACE may reboot because the server load-balancing process is unresponsive. This behavior can occur when the ACE is under a stress situation and a large number of real servers are out of service. Workaround: Ensure that the real servers are online and operating properly, or remove those out-of-service real servers from your load-balancing configuration.

CSCsk61805—UDP probes may stay alive even when the ACE receives ICMP host unreachable messages. This behavior applies only to host unreachable messages. The UDP probe behavior is correct with port unreachable messages. Workaround: None.

CSCsk67947—Without limiting the rate at which the ACE generates system log messages in the syslog for a specific priority level, syslogs will be stopped after 4,000,000,000 messages have been logged by the ACE. Workaround: To limit the rate at which the ACE generates messages in the syslog, use the logging rate-limit configuration mode command as shown as follows:

switch/Admin# logging rate-limit 1000000000 1 level 0
switch/Admin# logging rate-limit 1000000000 1 level 1
switch/Admin# logging rate-limit 1000000000 1 level 2

CSCsk68396—In a firewall load-balancing configuration where the firewalls are positioned between two ACEs and you configure the mac-sticky command on the ACEs, an ICMP error packet from a router located between the second ACE and the real server is forwarded to a firewall that is different from the firewall that received the original IP packet that generated the ICMP error. Workaround: None.

CSCsk74766—While copying and pasting the commands necessary to configure sticky server farms for a load-balancing policy map, the ACE may display the following message: "Error: Add link failed! This behavior is intermittent." Workaround: Manually configure the commands needed to add the sticky server farm to the load-balancing policy map instead of copying and pasting the commands.

CSCsk76911—The ACE may reboot when you repetitively configure and unconfigure the destination IP address associated with an HTTP probe in quick succession. Workaround: None.

CSCsk81695—When operating in a redundancy (fault tolerance) configuration, the standby ACE module may reboot when you attempt to initiate a packet capture in a user (non-Admin). Workaround: None.

CSCsk82791—The output for the show np np_number me-stats ucdump_option diagnostic commands does not support the display of individual user contexts, including the Admin context, to determine whether the statistics displayed have been incremented in the specific user context or have been incremented in other user contexts configured on the ACE. The show np command output displays statistics with the designation "Context ALL Statistics" as shown as follows:

switch/Admin# show np 1 me-stats "-slb -C 0"
LB Perf stats at address 0x82e05000
LB Perf stats at address 0x82e05000
LB Statistics
--------------
(Context ALL Statistics)
.

Within the Admin context, the unmodified show np command (without the -C context-id qualifier) will continue to display the aggregate statistics for all contexts. The show np command can be modified within Admin context to display the per-context statistics of the context within any valid context specified by ID as shown as follows:

switch/Admin# show np 1 me-stats "-slb -C 0"
LB Perf stats at address 0x82e05000
LB Perf stats at address 0x82e05000
LB Statistics
--------------
(Context 0 Statistics) 
.

Within user contexts, the unmodified show np command displays the statistics for the associated context only. Any attempts to specify a context ID in a user context, even the context ID of the originating user context, can generate an error. Workaround: None.

CSCsk85890—The ACE may reboot when operating in a redundancy configuration after you repeatedly apply a series of configuration changes to the ACE. Workaround: None.

CSCsk86782—The ACE may reboot when you attempt to activate the packet capture feature while the ACE is processing TCP traffic. Workaround: None.

CSCsk91302—Redundancy between ACE modules may become unresponsive with sticky replication enabled. This behavior can occur when a large number of sticky entries are in use. Workaround: None.

CSCsk93578—ACE performance may significantly drop when using the hash URL predictor with a large number of configured real servers. Workaround: If possible, change the predictor to round-robin.

CSCsk99204—The ACE randomly returns the incorrect information during virtual context concurrent discovery; the ACE returns one context information for other context (context A sends a request and it receives context B information). This behavior can occur when the ACE attempts to send requests concurrently for different contexts. Workaround: None.

CSCsl04542—When you specify the xml-show on command in Exec mode, the XML tags for the show ft peer detail command may display null strings in tags. This behavior can be due to certain tags being missing for XML. Workaround: None.

CSCsl05105—In an end-to-end SSL setup, with the ACE running as an SSL client, the rehandshake request from the SSL server is not recognized by the ACE and the session is terminated. Workaround: None.

CSCsl05501—The ACE may reboot shortly after you enter the show ipcp clients command and enable an output modifier that filters the configured (and enabled) packet capture function. For example:

switch/Admin# show ipcp clients! begin pktcap 

Workaround: Do not specify the show ipcp client | begin pktcap command when the packet capture function is in progress.

CSCsl07437—When a packet capture is in progress in a management connection for a specific context, it is possible to remove the capture session from a second management connection for the same context. Workaround: Do not delete a packet capture session without first stopping the session.

CSCsl07449—The ACE is running multiple concurrent Layer 7 HTTP connections, with either TCP server reuse or HTTP persistence rebalance enabled. If the ACE reaches a point where queue full drops occur or the module runs out of proxy entries, some Layer 7 HTTP connections may not be cleaned up properly. Workaround: Limit the traffic connections to the ACE, or disable TCP server reuse or HTTP persistence rebalance.

CSCsl12798—When a high rate of RTSP traffic is sent through ACE with application inspection enabled and all the control connections are closed at approximately the same time, you may observe that the control connections are removed from the ACE but that the data channels linger. Workaround: None.

CSCsl15152—The ACE may reboot during a high rate of RTSP traffic (2000 connections per second) when RTSP inspection is enabled. This behavior can occur with or without a server load-balancing configuration. Workaround: None.

CSCsl15321—After reloading with an HTTP inspection configuration, the ACE drops HTTP packets even when the module is configured to allow them to pass. Workaround: Reapply the configuration.

CSCsl26094—Probe failures may be observed intermittently in a configuration with ICMP or UDP probes. This behavior may occur when the probe time interval is set too small (for example, 5 seconds) and the number of probe instances is greater than 4000. The total limit of management connections for the ACE is 10,0000, and each probe requires two management connections. The default timeout for ICMP probes is 2 seconds, and the default timeout for UDP probes is 120 seconds. This means that a UDP probe connection will not be deleted for 120 seconds. For example, if there are 1000 UDP probes, and the probe time interval is 5 seconds, it will take approximately (1000 * 2 * (120/5)) = 48,000 management connections over a period of two minutes. The number of probe failing because of management connections may become significant.

Workaround: Reconfigure the associated probe time interval so that it does not exceed the number of management connections available in the system.

CSCsl34397—When the ACE is running in bridge mode and SNMP notifications are enabled, the ACE may reboot with an SNMP daemon (snmpd) core dump. Workaround: None.

CSCsl35473—Because the DNS inspection hash table entry timeout is fixed at approximately 100 seconds, when DNS inspection is configured and there are many unanswered requests, the hash table may become full. When the table is full, the ACE stops inspecting new DNS traffic. Workaround: None.

CSCsl36877—The ACE may become unresponsive under the following conditions:

End-to-end SSL configuration

Traffic rate of 1000 TPS or more

Certificates and keys of any size

Variable page size of 1 KB to 8 KB (occurs more frequently) or a fixed page size of 8 KB

On the ACE console, you may see messages similar to the following:

n2_intr: ARB_UNIT_ERR 0x00000008

n2_intr: EFL_IND_RPTR 0x000327fc

n2_intr: EFL_IND_RPTR_TYPE 0x00000120

Workaround: None.

CSCsl38419—Modifying or changing the SSL proxy under a Layer 7 policy map does not take effect. Workaround: Delete the Layer 3 service policy from the interface and then readd it.

CSCsl44903, CSCsl42531—With a weight-based predictor configured in the server farm, and TCP server reuse or HTTP persistence rebalance enabled, the real server weight may become negative, which can result in an incorrect connection distribution. Workaround: None.

CSCsl48977—The ACE may reboot when performing Layer 7 server load balancing with TCP server- reuse is enabled. This behavior can occur because a network processor on the ACE is under a stress condition when receiving HTTP traffic. Workaround: Remove the server-conn reuse command in HTTP parameter-map configuration mode.

CSCsl49210—The ACE may reboot when TCP server- reuse is enabled, the module is receiving Layer 7 HTTP traffic, and many concurrent connections. This behavior can occur when many connections close simultaneously, causing the CM_CLOSE queue to become full, which results in a deadlock when the HTTP function is posting a reuse message. Workaround: Run less traffic to the ACE or remove the server-conn reuse command in HTTP parameter-map configuration mode.

CSCsl55152—The ACE displays the error "Error: Max 10 http header map is allowed per policy!" when you perform the following operational sequence:

1. Configure a Layer 7 SLB class map that contains ten different HTTP header entries.

2. Specify the Layer 7 SLB class with a Layer 7 SLB policy map and add to a Layer 3 and Layer 4 policy map.

3. Apply the Layer 3 and Layer 4 policy map to an interface VLAN.

4. Remove and then readd one of the HTTP header entries from the associated Layer 7 SLB class map.

In this case, the Layer 7 SLB class map can became unusable and fails to match HTTP headers after you delete and then readd the HTTP entry.

Workaround: Perform one of the following actions:

Remove the policy map from the interface.

Delete the policy map, update the class map, and then reconfigure the policy map.

CSCsl57828—With load-balanced TCP flows running through the ACE, the module may become unresponsive and may eventually reset if the client or the server advertises a maximum segment size (MSS) value that is less than 1460 bytes (the ACE default value) via the MSS option or if the client or the server does not send an MSS value. You may observe this behavior when the ACE configuration includes the following features:

Layer 4 load balancing and HTTP inspection

Layer 7 load balancing and the persistence rebalance command

Workaround: Configure the following Layer 4 connection parameter map and associate it with the Layer 4 multi-match policy map as follows:

parameter-map type connection TCP
  set tcp wan-optimization rtt 0

policy-map multi-match TEST
  class TEST
    connection advanced-options TCP

CSCsl60063—After you change an application inspection action in a policy map from one setting and then back to original value in quick succession (within 5 second intervals), traffic may stop flowing to the ACE. Workaround: To avoid this behavior, wait approximately 10 seconds between making application inspection action changes to prevent this condition. If this condition is already present, remove the current application inspection action, wait approximately 10 seconds, and then reapply the desired application inspection action.

CSCsl60440—When the predictor leastconns, server-conn reuse, and persistence-rebalance commands are configured, if successive GET requests result in the same policy being selected, then the ACE does not increment the connection counts for the real server and does not set up predictor weight values. However, when the server-side connection closes, the ACE incorrectly performs cleanup on every REUSE CLOSE message, which results in negative weights for the real servers and incorrect traffic distribution for real servers in the server farm. Workaround: Disable either the persistence-rebalance or the server-conn reuse command.

CSCsl60601The ACE module may reboot when you remove a VLAN interface, and then attempt to start a packet capture that is associated with the removed VLAN interface. When you attempt to start the packet capture, the ACE encounters an issue with the interface ID and prints the following error: "Error: Bad interface ID." The following command output illustrates this operational sequence:

switch/ctx1# capture cap1 all access-list mgmt_acl
switch/ctx1(config)# no interface vlan 500
switch/ctx1# capture cap1 start 
Error: Bad interface ID 

Workaround: Avoid removing an interface that you are using to capture packet information. If you encounter this issue, stop the packet capture, and then remove the session before removing the interface as follows:

switch/ctx1# capture cap1 stop 
switch/ctx1# capture cap1 remove 

CSCsl62013—Entering the show logging internal command may cause the syslog daemon (syslogd) to perform a core dump, which may cause the ACE module to reboot. Workaround: None.

CSCsl67151—Repeatedly entering the ft switchover command to cause a switchover can result in the ACE running out of buffers and becoming less responsive as the buffers get depleted until the Supervisor engine resets the ACE. Workaround: None.

CSCsl71315—The continuous reconfiguration of the inservice and no inservice states on an FT group may cause redundancy to enter an incorrect state on the standby ACE. The config sync received during each iteration of inservice and no inservice overwrites the configuration on the standby ACE. During multiple inservice and no inservice requests, the final status of the final config sync does not get recorded. Workaround: Remove and reconfigure the FT group to place it into the correct redundancy state.

CSCsl74755—ACE sockets may become depleted when you repeatedly (approximately 250 attempts) access the ACE using SSH. For example, entering the ssh admin@192.168.1.123 sh ver command may cause the ACE to generate the following message to the client:

Received disconnect from 192.168.1.123: 2: Could not create socket pairs: Too many 
open files in system.

In some cases, this condition generates the following message on the ACE console:

socket: Limiting socket usage, no more sockets, Max allowed is 512 and current_usage 
is 512

Workaround: Avoid excessive SSH logins. After the condition occurs, you must reboot the ACE to clear the resource depletion.

CSCsl77474—Configuring the shared-vlan-hostid 16 command causes any interfaces configured on shared VLANs using that ID to become unreachable. These interfaces are in the various user contexts on the ACE, and failing traffic includes ARPs, probes, and so on. In this case, the interface address becomes completely unreachable. Any interface created on a shared VLAN while the shared-vlan-hostid 16 command is enabled is also unreachable. The show arp command displays a MAC address of 00:12:43:dc:ab:00 or higher on such interfaces. Workaround: Do not configure the shared-vlan-hostid 16 command. You can use any of the other available host IDs (1 through 15) for the shared-vlan-hostid command.

The ACE randomly assigns a host ID to shared VLANs if you do not configure a host ID. The ACE never chooses a host ID of 16 when performing this random assignment. You will observe this behavior only if you configure the shared-vlan-hostid 16 command on your ACE.

CSCsl82712—When the ACE is configured for Layer 7 load balancing, an HTTP method request that exceeds the default (2000 bytes) or configured maximum HTTP header parse length is dropped and a TCP reset is sent to the client. Workaround: Increase the maximum HTTP header parse length in a parameter map using the set header-maxparse-length command.

CSCsl89772—With a UDP probe configured on port 69 (TFTP), the concurrent connections counter of the show resource usage command output keeps incrementing. Workaround: None.

CSCsl95565—When you are using SNMP with multiple contexts and redundancy is configured, the standby ACE may reboot with a last reboot reason of service snmpd. When the ACE comes back up, it may be in an odd state with some of the contexts not functioning properly. Workaround: None.

CSCsl98284—Traffic does not return to the real server in the primary server farm from the backup server farm even though the real server in the primary server farm is back online. This behavior can occur when two policies have server farms configured in reverse order. For example, policy A has server farm S1 as the primary and server farm S2 as the backup, and policy B has server farm S2 as primary and server farm S1 as primary. Workaround: Delete and readd the server farms to the configuration.

CSCsm01910—The ACE may reboot when you repeatedly delete and then readd a user context. Workaround: None.

CSCsm03018—When you enable logging for syslog message IDs 302022 to 302025, changing the logging level for syslog messages 302022 to 302025 from Level 6 to Level 4 also affects the logging level for messages 302028 to 3020311. Logging rate limiting does not work for message IDs 302028 to 302031, and you cannot change the logging level for messages 302028 to 302031 directly because these syslogs are not supported by the relevant CLI commands. Workaround: None.

CSCsm03874—Configuring and removing the inspect http command from a policy map prohibits the removal of match statements from the associated class map. When this behavior occurs, the following error appears:

Error: Cannot delete this object as this is referenced by inspect action

The class map under question must have multiple match statements, and the error is observed only when removing match statements other than the first match statement within that class map. Workaround: None.

CSCsm05168—You are unable to remove the match any statement in the Layer 3 and Layer 4 match-all class-map. When this behavior occurs, the following error appears:

Error: Match item does not exist

Workaround: Remove and readd the class map with the required match statements.

CSCsm09190—In an end-to-end SSL configuration with both a client and server VLAN configured, the ACE may reboot after you initiate a checkpoint rollback to a running-configuration that contains only a client VLAN in the configuration while SSL traffic is flowing. The rollback checkpoint should include both the client and server VLANs. Workaround: None.

CSCsm10390—In some instance the ACE may reboot and fail to save the associated core file in the core: directory for subsequent analysis. Workaround: None.

CSCsm11256—The ACE may reboot and reload to the ROMMON prompt when you have an ACE configuration that includes up to 250 contexts and significant number of probes. Workaround: None.

CSCsm12883—The ACE resets connections when the number of long-lived persistent HTTP connections is one less than the configured maximum value. Workaround: None.

CSCsm17099—In a large configuration with a significant number of active UDP probes, the UDP-related probes may take an unusually long time to check the state of the server and appear to be stuck in INIT state. This symptom can occur when you perform one of the following actions:

Delete and then readd a large number of UDP-related probes from the ACE configuration.

Remove a UDP-related probe from a real server or server farm and readd it back to the real server or server farm.

Workaround: None.

CSCsm17268—In a redundant configuration, when you add a single line to an existing SNMP configuration (for example, snmp community public ro) and the ACE performs a subsequent incremental synchronization, the configured command appears in the running-configuration on the peer, but the process seems to cause an internal failure which makes the peer transition to the STANDBY_COLD state. The running-configuration sync status is reported as Peer in Cold State. Incremental Sync Failure: snmp config sync to sby failed. Workaround: Avoid the use of the show snmp command on the peer.

CSCsm18052—An existing SSL connection may stall and fail to perform the handshake to complete the transaction. When this occurs, the ACE does not send a fatal alert indicating a user parameter change and the connection stalls. Workaround: After changing the configuration, use the clear conn all command to clear all connections that go through the ACE, originate with the ACE, or terminate with the ACE.


Note The clear conn all command clears all connections, not just SSL connections that exhibit potential problems.


CSCsm18824—The ACE may reboot when sending approximately 15,000 Layer 7 transactions per second (TPS) through the ACE. Sending this volume of traffic can result in the ACE running out of buffer particles, which causes the messaging layer to send truncated or corrupted buffers to the internal Load Balance module in the ACE. Workaround: If possible, do not put an excessive stress on the ACE to the point where all buffer particles are exhausted. In addition, do not make significant configuration changes while running a high volume of traffic through the ACE.

CSCsm18847—If you change the logging level for syslog messages 302028 to 302031 when they are enabled using the logging fastpath command, the ACE still uses level 6 to rate-limit these syslogs. Workaround: None.

CSCsm27280—RHI routes may be withdrawn when the redundancy state for the ACE standby module changes to the STANDBY_HOT state. When this occurs, both the active and the standby ACE module advertise the VIP for the time when the standby context is in STANDBY_BULK state, which can result in traffic entering a black hole (the incoming packet enters but does not emerge from a portion of the network).

CSCsm33730—The active ACE module running in a redundant configuration may reboot when operating at a high connection rate in a Layer 4 or Layer 7 SLB configuration. This behavior can occur in cases where the ACE becomes heavily loaded and all queues are full. The SLB redundancy message posted from the standby ACE module to the active ACE module results in a deadlock in the active ACE, which causes the active ACE to reboot. Workaround: None.

CSCsm34992—Valid HTTP connections may be dropped after you modify the action in a Layer 3 and Layer 4 policy map from specific application inspection action (SIP, RTSP, or Skinny) to an HTTP application inspection action. Workaround: None.

CSCsm35407—After rolling back a large ACE configuration to a blank context and entering the show running-config command, the ACE may perform a core dump. The core dump does not cause the ACE to reload, and there is no adverse impact to the module. Workaround: None.

CSCsm37228—Occasionally, when you are operating in a redundant configuration and you execute TACACS-related commands on the active, the TACACS process may become unresponsive on the standby ACE and the standby ACE may reboot. Workaround: None.

CSCsm43065—The ACE may reboot after executing the show serverfarm sfarm retcode command with traffic running. This behavior may be exhibited when a relatively large number of retcodes have been configured and exceeded on a server farm, resulting in the corruption of an internal buffer where the retcode data has been copied. Workaround: None.

CSCsm44254—The ACE may boot when you attempt to add a scripted probe to a server farm in a user context Workaround: None.

CSCsm46153—The ACE fails to forward IP fragments that are greater than 68 bytes in length if this fragment is not the last fragment. Workaround: None.

CSCsm46301—With both HTTP application inspection and packet capture enabled, the HTTP application inspection process may become unresponsive and then the ACE reboots after a high rate of HTTP traffic is sent through the ACE. Workaround: None.

CSCsm47121—The ACE may reboot without generating a core file after the following messages are printed on the console:

mts_acquire_q_space() failing - no space in sap 32
sap=13336 rq=0 lq=0 pq=0 nq=1 sq=0 buf_in_transit=2, bytes_in_transit=1776
mts_deliver_local_atomic:mts_acquire_q_space failed for opcode 17, src_sap = 500
mts_acquire_q_space() failing - no space in sap 32
sap=13336 rq=0 lq=0 pq=0 nq=1 sq=0 buf_in_transit=2, bytes_in_transit=1776

This behavior can occur when you use logging supervisor command to forward syslog messages to the supervisor engine in the Catalyst 6500 series switch or Cisco 7600 series router, and the volume of system message messages is very large.

Workaround: Do not use logging supervisor command if you expect a high volume of messages.

CSCsm52011—The standby ACE in a redundant configuration may become unresponsive upon booting. Workaround: None.

Software Version A2(1.0) Open Caveats

The following open caveats apply to software version A2(1.0):

CSCsj41909—The ACE fails with the following message appearing in the syslog:

%ACE-5-199006: Orderly reload started at xxxxxxx by System. 

The reload reason is NP 1 Failed: NP Process Crashed. Workaround: None.

CSCsj64833—The network utility traceroute does not work for a configured ACE IP interface if the underlying protocol is UDP or TCP. Workaround: ICMP traceroute will work. The default of most traceroute utilities is UDP so a command line option might be necessary. For Linux, use the -I option.

CSCsj68643—The following log messages may appear sporadically in the ACE log:

can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.

can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.

These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.

CSCsj74250—When you configure the TACACS server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, there is a key mismatch when attempting to authenticate. Workaround: Paste the properly encrypted key into the running-configuration file.

CSCsj94366—While attempting to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: console configuration can only be done on console. Workaround: None.

CSCsk36611—If you are using Internet Explorer (IE), an SSL rehandshake may fail if the total length of the SSL certificate chain is greater than 4024 bytes. When this condition exists, the ACE creates two SSL records. The first record has a total length field indicating 4024 bytes, but containing a certificate item with a specified length greater than 4024 bytes. The second record contains a new SSL record header and the remaining portion of the previous SSL record. Workaround: Use Firefox or another browser.

CSCsk63774—Current connection statistics in the show serverfarm name command output may not coincide with the show serverfarm command output. This behavior may exist even with a minimal amount of traffic. Workaround: None.

CSCsl33851—With an action list configured to insert, rewrite, or delete HTTP headers, and HTTP persistence rebalance is enabled, you may encounter issues when using the ACE to insert headers on both request and response for HTTP requests when there are a large number of header insert command in action list (for example 50 statements). Workaround: None.

CSCsl46334—With a high rate of Layer 7 load-balanced traffic flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse configured, the ACE may start dropping traffic after a few hours. Workaround: None.

CSCsl64911—The behavior of HTTPS probes in nonrouted mode is the same as that of the probes in routed mode (the inclusion of the routed option with the ip address command). For example:

probe https https1
   ip address 10.76.248.141
   interval 10
   passdetect interval 10

Workaround: None.

CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health check(s). Workaround: None.

CSCsl75662—You may that ACE health probes remain in the INIT state when you change a parameter that is associated with the probe; the configuration change takes effect only after the next time the probe is fired even though the configuration change is visible in the running-configuration file. This behavior may be most visible when you change a probe with a high time interval (for example, 65535 seconds) to a much lower interval (for example, 2 seconds). In this configuration, it may appear as if the probe fails to fire; the initial large time interval has to first expire before the new, smaller interval can take effect.

Workaround: For a probe parameter change to take immediate effect, perform the following procedure:

a. Remove the probe from real server and the server farm.

b. Modify the probe parameter that you want to change.

c. Readd the probe to the real server and the server farm.

For details, refer to the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide.

CSCsl80651—When you configure a large number of scripted probes across multiple user contexts (for example, a total of 255 scripted probes running across 5 user contexts), you may find that the ACE stops sending probes to the real servers or that certain scripted probes remain in the INIT state. This behavior may occur when more than 200 scripted probe instances are running. When this occurs, if you display configuration information and statistics for a probe by using the show probe detail command, the probe states for the probe instances appear to remain in the INIT state. Workaround: None.

CSCsm10702—In a redundancy configuration the ACE unexpectedly becomes unresponsive to network traffic. The FT peer of the ACE assumes mastership, but the affected ACE does not assume Standby status or respond to the network. However, you can access the affected ACE by sessioning in to the module as the default user from the Catalyst series switch CLI. The frequency of the unresponsive episodes may correlate with logging from the ACE to the Catalyst supervisor engine using the logging supervisor 7 command in configuration mode. Workaround: Decrease the log level of the logging supervisor command, or avoid logging to the Catalyst supervisor engine. The ACE recovers upon reboot, again becoming responsive to the network and assuming its proper FT status.

CSCsm40004—When you are creating a Certificate Signing Request (CSR) and you attempt to define the state name parameter in the CSR parameter set by using the state command in CSR parameter configuration mode, the ACE does not allow you to use the space character in the state argument. Workaround: Use the abbreviation of the state.

CSCsm43541—The ACE is receiving SSL-related traffic and the module may reboot due to a buffer corruption issue on the crypto chip. Workaround: None.

CSCsm46044—ACL-MERGE-ERROR messages may be logged as follows:

ACL-MERGE-ERROR:cannot find ACL in acl_merge_rem_acl_from_list 
../security/acl/acl_merge.c:xxx

This behavior may be observed when you enable the debug access-list merge errors command in debug mode and then add new configurations to the ACE. Workaround: None.

CSCsm52480—All IPv6 multicast packets are dropped by the ACE even though the module is properly configured:

switch/Admin# access-list acl_1 ethertype permit ipv6 

This symptom is observed only with IPv6 multicast packets and does not occur with IPv6 unicast packets. Workaround: None.

CSCsm62263—The syslogd function in the ACE may become unresponsive and generate a core file. The symptom may occur when you attempt to shut down the ACE by entering the no power enable command in configuration mode at the Catalyst 6500 series switch CLI. Workaround: None.

CSCsm64646—If all or the majority of the configured real servers receiving health probes from the ACE become unreachable at the same time (for example, you shut down the interface on the Catalyst 6500 series switch), the probe states for all the instances do not immediately change to the FAILED state. With a significant number of active probe instances (for example, 8000), it may take a lengthy period of time, sometimes hours, for all the probes to move to the FAILED state. In this case, those probes appear to be stuck in their original state while the ACE generates numerous out-of-socket errors. Workaround: To reduce the severity of the issue, configure a large value for the passdetect interval as compared to the configured probe interval. In addition, configure a small value for the retry count for failed probes using the faildetect command. For example:

switch/Admin(config-probe-tcp)# passdetect interval 300
switch/Admin(config-probe-tcp)# probe interval 30
switch/Admin(config-probe-tcp)# faildetect 2

CSCsm65534—You may find that sequential readings of the Client Byte Count and the Server Byte Count fields in the show service-policy command output increment or decrement by large values without the expected changes in network traffic. This behavior is a display-only issue and does not affect traffic forwarded by the ACE. You may encounter this behavior after the byte counters exceed the maximum of 4294967295 bytes. Workaround: None.

CSCsm67002—When operating in a redundant configuration, the standby ACE may reboot because one or more of the Micro Engines become unresponsive in the network processor on the ACE. After the standby ACE reboots, if you specify the show ft group command, the My State field in the show output displays the FT status "FSM_FT_STATE_INIT" and the active ACE displays the peer FT state "FSM_FT_STATE_UNKNOWN". You may find that a subsequent reboot of the ACE fails to clear the FT status. Workaround: None.

CSCsm71444—In a multi-level backup real server setup, the ACE may incorrectly place a real server into the OPERATIONAL state when you enter the no inservice standby command. For example, a multi-level backup configuration includes a server farm that is configured with three real servers (RS1, RS2, and RS3) in a chain, such that RS1 has RS2 as its backup real server and RS2 has RS3 as its backup real server. In this configuration, if you enter the no inservice standby command to take real server RS2 out of service, the ACE will incorrectly place real server RS2 into the OPERATIONAL state instead of placing it into the OUTOFSERVICE state without breaking the chain (RS1->RS2->RS3). As a result, the multi-level backup chain of RS1->RS2->RS3 is divided into two real server backup configurations: RS1 and RS2->RS3. If you enter the inservice standby command to place real server RS2 in the STANDBY state, the ACE does not retain the chain. Workaround: In a multi-level backup real server setup, do not enter the no inservice standby command or the no inservice command to remove the real server from the inservice state.

CSCsm72725—The packet capture output of one context may appear in other (different) user contexts. This behavior may occur when you use a terminal to configure the packet capture function in a context and then specify the changeto command to switch to a different context using the same terminal.

Workaround: Perform one of the following actions:

Stop the packet capture process before you enter the changeto command (the recommended workaround).

Log out of the terminal, then log in again to access a different context than the original context with the configured packet capture function.

CSCsm79292—When you attempt to use the vsh-conf-cmd TCL command to execute the command or set of commands specified in the preceding set command string (cmd_str) by invoking the Vegas shell (Vsh), the scripted probe may fail and the following error message appears:

Internal error: Script error

For example:

set cmd_str "rserver rs \n inservice"
vsh_conf_cmd $cmd_str

This behavior can occur when you create a scripted probe in which there is vsh-conf-cmd specified on the active ACE. The configuration commands are executed on the active ACE and then replicated on the standby ACE. When this behavior occurs, the scripted probe executes properly on the active ACE, but may fail on the standby ACE because configuration mode is disabled. Workaround: None.

CSCso00356—If you are configuring the SYN cookie feature in a bridged VLAN, you may find that some client connections may become unresponsive. The ACE can exhibits this behavior when all of the following conditions are true:

The VLAN interface is configured as part of a bridge group.

Traffic does not match any configured load-balancing service policy.

SYN cookie is configured and is intercepting SYN packets, meaning that the number of embryonic connections is equal to the configured SYN cookie embryonic connection threshold.

The client's ARP entry is not yet learned by the ACE.

Workaround: The unresponsive client connection behavior stops once the client's ARP entry has been populated in the ACE module ARP table.

CSCsr89311—An incompatibility exists between certain ACE software versions in the 3.0(0)A1.6.3x and A2.1x release trains. In a redundant configuration, the FT ACE pairs will not recognize each other and will report the following status as part of the show ft peer detail command output:

SRG Compatibility : INCOMPATIBLE

The following software version combinations that are marked with an "x" are incompatible:

A1(6.3x) Release
A2(1.0)
A2(1.0a)
A2(1.1)
A2(1.1a)

3.0(0)A1(6.3b)

x

 

x

x

3.0(0)A1(6.3c)

x

x

x

x


Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS Version 2.0.