Guest

Cisco Services Modules

Release Note vA2(3.x), Cisco ACE Application Control Engine Module

  • Viewing Options

  • PDF (1.3 MB)
  • Feedback
Release Note for the Cisco Application Control Engine Module

Table Of Contents

Release Note for the Cisco Application Control Engine Module

Contents

Supervisor Engine and Cisco IOS Support for the ACE Module

Virtual Switching System Support

ACE Module Troubleshooting Wiki

New Software Features in Version A2(3.6a)

Ability to Allow Send-data to take Carriage Return and Linefeed Characters

Modification to ACE Behavior When an Echo Probe (TCP or UDP) Does Not Match a Regex Value

Modifications to the show ip route Command Output

Modifications to the show cfgmgr Command Output

Modifications to the show ip fib Command

New Software Features in Version A2(3.5)

New Software Features in Version A2(3.4)

Monitoring and Displaying the Network Processor Buffer Usage

Displaying the NP Buffer Usage

Related Syslogs for Buffer Usage

Related SNMP Changes

Skipping a Malformed Cookie in an HTTP Flow

Bypassing Inspection during HTTP Transactions

Appending Nondefault Port Information in the HTTP Host Header

Reporting a Real Server MAXCONN State When One NP Reaches its Allocated Limit

New Software Features in Version A2(3.3)

Configuring SNMP Peer Engine ID for the Standby ACE

Enabling SSL Rehandshake on All Contexts

Ignoring Malformed Cookies in a Request

Bypassing HTTP Parsing After Processing a Connection Request

Accounting Logs Include Passphrase Commands

New Software Features in Version A2(3.2)

Probing a Redirect Server

Disabling Connection Replication

Configuring a Multicast MAC Address for a Host

Configuring Inactivity Timeout for Connections in Switch Mode

Configuring the Compilation Timeout for Regular Expressions

Accounting Logs Containing Sensitive Information

New Software Features in Version A2(3.1)

KAL-AP Node To Notify the GSS When Backup Server Farm is in Use

Secondary IP Address Enhancement

Admin Context is not Allowed to be Starved of all Resources

Configuring the ACE to Perform an SSL Rehandshake

New Software Features in Version A2(3.0)

ACE Operating Considerations

Software Version A2(3.6a) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.6a) Resolved Caveats

Software Version A2(3.6a) Open Caveats

Software Version A2(3.6a) Command Changes

Software Version A2(3.6a) System Log Messages

251010

322006

441003

Software Version A2(3.5) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.5) Resolved Caveats

Software Version A2(3.5) Open Caveats

Software Version A2(3.5) Command Changes

Software Version A2(3.5) System Log Messages

106029

111009

Software Version A2(3.4) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.4) Resolved Caveats

Software Version A2(3.4) Open Caveats

Software Version A2(3.4) Command Changes

Software Version A2(3.4) System Log Messages

251015

443002 through 443005

Software Version A2(3.3) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.3) Resolved Caveats

Software Version A2(3.3) Open Caveats

Command Changes in Software Version A2(3.3)

System Log Messages

901001

Software Version A2(3.2a) Resolved Caveats and Open Caveats

Software Version A2(3.2a) Resolved Caveats

Software Version A2(3.2a) Open Caveats

Software Version A2(3.2) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.2) Resolved Caveats

Software Version A2(3.2) Open Caveats

Command Changes in Software Version A2(3.2)

System Log Messages

251014

441003

442007

751001

901001

Software Version A2(3.1) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.1) Resolved Caveats

Software Version A2(3.1) Open Caveats

Command Changes in Software Version A2(3.1)

Commands Inherited from Software Version A2(2.3)

Commands Inherited from Software Version A2(1.6)

System Log Messages

New Syslog Messages

504003

Software Version A2(3.0) Resolved Caveats and Open Caveats

Software Version A2(3.0) Resolved Caveats

Software Version A2(3.0) Open Caveats

Available ACE Licenses

Ordering an Upgrade License and Generating a License Key

Upgrading Your ACE Software

Changing the Admin Password

Changing the www User Password

Checking Your Configuration for FT Priority and Preempt

Creating a Checkpoint

Updating Your Application Protocol Inspection Configurations

Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration

Before You Begin

Downgrade Procedure

Downgrading Effects on the kal-ap primary oos Command (Downgrade from version A2(3.1) to A2(3.0))

ACE Documentation Set

Obtaining Documentation and Submitting a Service Request


Release Note for the Cisco Application Control Engine Module


April 20, 2012


Note The most current Cisco documentation for released products is available on Cisco.com.


Contents

This release note applies to the following software versions for the Cisco Application Control Engine Module (ACE), models ACE10 (ACE10-6500-K9) and ACE20 (ACE20-MOD-K9):

A2(3.6a)

A2(3.5)

A2(3.4)

A2(3.3)

A2(3.2a)

A2(3.2)

A2(3.1)

A2(3.0)

For information on the ACE module features and configuration details, see the ACE documentation located at:

http://www.cisco.com/en/US/products/ps6906/tsd_products_support_model_home.html

This release note contains the following sections:

New Software Features in Version A2(3.6a)

New Software Features in Version A2(3.5)

New Software Features in Version A2(3.4)

New Software Features in Version A2(3.3)

New Software Features in Version A2(3.2)

New Software Features in Version A2(3.2)

New Software Features in Version A2(3.1)

New Software Features in Version A2(3.0)

ACE Operating Considerations

Software Version A2(3.6a) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.5) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.4) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.3) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.2a) Resolved Caveats and Open Caveats

Software Version A2(3.2) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.1) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

Software Version A2(3.0) Resolved Caveats and Open Caveats

Available ACE Licenses

Ordering an Upgrade License and Generating a License Key

Upgrading Your ACE Software

Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration

ACE Documentation Set

Obtaining Documentation and Submitting a Service Request

Supervisor Engine and Cisco IOS Support for the ACE Module

Table 1 and Table 2 summarize the supervisor engine model and Cisco IOS version support for the ACE module in the Catalyst 6500 series switch and the Cisco 7600 series router, respectively.

Table 1 Supervisor Engine and Cisco IOS Support for the ACE Module in a Catalyst 6500 Series Switch with a Multilayer Switch Feature Card (MSFC3)

Supervisor Engine Model
Minimum Required IOS Version
Other IOS Version Support

WS-SUP720

12.2(18)SXF4 (or later)

12.2(33)SXH (or later), 12.2(33)SXI1 (or later)

WS-SUP720-3B

WS-SUP720-3BXL

VS-S720-10G-3C

12.2(33)SXH (or later)

VS-S720-10G-3CXL

VS-S2T-10G2

12.2(50)SY (or later)

VS-S2T-10G-XL

1 Minimum required IOS version for VSS support. See the Virtual Switching System Support section.

2 Minimum required ACE20 module software version for Supervisor Engine 2T support is A2(3.4) or later. This ACE software version supports both supervisor engine models: VS-S2T-10G and VS-S2T-10G-XL.


Table 2 Supervisor Engine, Route Switch Processor (RSP), and Cisco IOS Support for the ACE Module in a Cisco 7600 Series Router with an MSFC3 

Supervisor Engine or RSP
Minimum Required IOS Version
Other IOS Version Support

WS-SUP720

12.2(18)SXF4 (or later)

12.2(33) SRB (or later)

Not supported: 12.2(33)SXH1

WS-SUP720-3B

WS-SUP720-3BXL

RSP720

12.2(33)SRC (or later)

None

RSP720-3C-10GE and RSP720-3CXL-10 GE

15.0(1)S (or later)

None

1 Cisco IOS release 12.2(33)SXH runs only on the Catalyst 6500 series switch. Therefore, the Supervisor 720-10GE engines are not supported in the Cisco 7600 series router.


For more information about Cisco IOS releases, see the Release Notes for Cisco IOS Release 12.2SXF and Rebuilds and the Release Notes for Cisco IOS Release 12.2(33)SXH and Later Releases.

Virtual Switching System Support

The ACE10 and the ACE20 running ACE software version A2(1.2) or later and installed in a Catalyst 6500 series switch running IOS software version 12.2(33)SXI or later support the Virtual Switching System (VSS). VSS is a system virtualization technology that allows the pooling of multiple Catalyst 6500 series switches into a single virtual switch for increased operational efficiency by simplifying the network. Inter-chassis Supervisor switchover (SSO) boosts nonstop communication. For more information about VSS, see the Cisco IOS Version 12.2(33)SXI Configuration Guide.

ACE Module Troubleshooting Wiki

The ACE documentation set now includes the ACE Module Troubleshooting Wiki. This wiki is a collaborative site that describes the basic procedures and methodology to assist you in troubleshooting the most common problems that you may encounter while you are operating your ACE.

As a registered user of Cisco.com, we strongly encourage you to add content to this site in the form of troubleshooting tips, procedures, or even entire sections. When you add content to the site, you should adhere to the format that has been established for the wiki. To access the ACE Module Troubleshooting Wiki on Cisco DocWiki, click the following URL:

http://docwiki.cisco.com/wiki/Cisco_Application_Control_Engine_(ACE)_Troubleshooting_Guide

New Software Features in Version A2(3.6a)

The A2(3.6a) software release provides the following new features:

Ability to Allow Send-data to take Carriage Return and Linefeed Characters

Modification to ACE Behavior When an Echo Probe (TCP or UDP) Does Not Match a Regex Value

Modifications to the show ip route Command Output

Modifications to the show cfgmgr Command Output

Modifications to the show ip fib Command

Ability to Allow Send-data to take Carriage Return and Linefeed Characters

Per CSCtq84947, the send-data under the TCP, ECHO, and UDP probes now allows the following combination of Carriage Return (CR) and Linefeed (LF) characters:

\r\n

\r\r

\n\n

multiples of these (such as \r\n\r\n)

An example would be "send-data GET / HTTP/1.0\r\n\r\n".

\r\n in send-data would be converted to CRLF while sending probe data to the server.

\r\r in send-data would be converted to CRCR while sending probe data to the server.

\n\n in send-data would be converted to LFLF while sending probe data to the server.

Separate entries such as \r and \n in send-data would not be converted to CR and LF. They will be sent as `\' followed by `r', and `\' followed by `n', respectively, similar to the process prior to the introduction of this enhancement in software version A2(3.6a).

Modification to ACE Behavior When an Echo Probe (TCP or UDP) Does Not Match a Regex Value

Per CSCth99982, when you configure an ECHO TCP or UDP probe on the ACE and the server sends a regex that does not match the configured send-data value, the probe fails and the ACE generates the following syslog message:

%ACE-3-251010: Health probe failed for server address on port number, Server response not 
matching with configured echo probe send-data
 
   

In addition, when the ECHO TCP or UDP probe does not match the regex value, the show probe detail command displays the following error message in the Last disconnect err field:

"Server response not matching with user configured send-data"
 
   

For the ECHO TCP and ECHO UDP probes, if the server responds with a regex value other than the configured ECHO probe send-data, the probe fails. The show probe detail command and health probe failure syslogs also have a new string associated for this failure.

Modifications to the show ip route Command Output

Per CSCti56893, you have to now use the reference count to keep track of the NAT pools that are applied on multiple interfaces. When NAT is applied on the first interface, both the routing information base (RIB) and forwarding information base (FIB) entries are created. Only one entry can exist in the RIB with the reference count incremented and decremented as and when applied or removed from the interfaces. FIB has one entry. The entries are removed both from the RIB and FIB when the NAT pool configuration is removed from the last interface.

Previously, the show ip route command displayed only the row and the value of the interface column was displayed as N/A.

Modifications to the show cfgmgr Command Output

Per CSCtw80706, the show cfgmgr internal sfarm detail command has been changed now to print additional diagnostic information for troubleshooting purposes.

Modifications to the show ip fib Command

Per CSCtu24904, the overflow (V) flag now displays the legend explanation in the show ip fib command as displayed in the following output:

switch/Admin# show ip ?
  dhcp       Show DHCP configurations
  fib        Display the FIB entries
  interface  IP interface status and configuration
  route      Display the route entries
  traffic    IP protocol statistics
switch/Admin# show ip fib
 
   
FIB for Context Admin (RouteId 0)
 
   
   Codes: H - host,   I - interface
          S - static,      N - nat
          A - need arp resolve,      E - ecmp
          V - virtual server
 
   
Destination         Interface         EncapId  Flags
------------------------------------------------------------------------
224.0.0.0/3         N/A                 DROP   N/A [0x100]
127.1.0.0/16        vlan1                  1   SI [0x18]
25.25.25.0/24       vlan200                0   IA [0x30]
25.25.25.49/32      vlan200                3   H [0x3]
127.1.0.0/32        N/A                 DROP   N/A [0x10]
127.1.0.1/32        vlan1                  1   I [0x10]
25.25.25.86/32      vlan200                4   H [0x3]
25.25.25.214/32     N/A                 DROP   V [0xc00]
127.1.255.255/32    N/A                 DROP   N/A [0x10]
25.25.25.0/32       N/A                 DROP   N/A [0x10]
25.25.25.99/32      N/A                 DROP   N/A [0x10]
25.25.25.255/32     N/A                 DROP   N/A [0x10]
25.25.25.11/32      vlan200                5   H [0x3]
25.25.25.13/32      vlan200                2   H [0x3]
Total route entries = 14

New Software Features in Version A2(3.5)

The A2(3.5) software release provides the following new features.

Per CSCtn90010, added the snmp-server bulk-request commands. The syntax of these two commands is as follows:

snmp-server bulk-request {max-oid number1 | max-repetition number2}

For the number1 or number2 argument, enter an integer from 0 to 2147483647. Both 0 and 2147483647 are default values and indicate that the command is disabled. These commands act as filters for incoming SNMP bulk requests and they allow you to limit the maximum number of OIDs that can be processed by the ACE and the maximum number of repetitions that can be returned in an answer. When the filter is triggered and the debug snmp trace command is enabled, the following messages appear on the console:

Limit Max Repetition to sysMaxRepetition (reqMaxRepetition)
Limit quantity OIDs to sysMaxOID
 
   

If you encounter any issues with the ACE being flooded with SNMP bulk requests, then you should set the argument values of these commands manually. For example:

host1/Admin(config)# snmp-server bulk-request max-oid 50
 
   
host1/Admin(config)# snmp-server bulk-request max-repetition 100
 
   

Per CSCtq31721, HTTP and HTTPS probes no longer display the default open timeout value in the running-configuration.

Per CSCto91867, added the peer password option for the auth and priv options of the snmp-server user command to enter a password for the peer user.

Per CSCth08113, added the cache option to the expect regex command for HTTP probe regex parsing. This option enables regex parsing in cached mode for HTTP and HTTPS probes which overcomes the issue with the regex parsing of a string at the end of a very long dynamic web page. Enter a cache value from 1 to 1000 bytes. The default is 1000 bytes.

Per CSCth08113, added the new regex cache-len field to the output of the show probe detail command to complement the cache option of the expect regex command.

Per CSCtj20245, changed the lower limit of the log threshold_number value and the remove threshold_number value of the retcode command to 2.

Per CSCtl50901, added the new np keyword to the show serverfarm name command to display the state of the real server on each network processor (NP).

Per CSCtn85846, the MTS warning message has been changed and clarified. VSH does not process MTS messages as it executes user commands. Because a user command may take a while to terminate, the ACE can receive many unprocessed MTS messages and run out of buffer space for the MTS queue. When this happens, a warning message is displayed as follows:

Warning:- MTS queue is full for opcode %d sap %d pid %d clear idle debug plugin sessions or telnet/ssh connections to recover

The new message is clearer and provides more information as follows:

Warning:- MTS queue is full for opcode %d sap %d pid %d. This warning can be ignored. If you want to recover - close all debug plugin sessions and terminate command execution in all telnet/ssh connections.

Per CSCtn26048, an SNMP OID to monitor ACE external buffer utilization was added as follows:

[root@GW-PC1 root]# /usr/bin/snmpget -v 2c -c public -m all -M /root/mibdir/MIBS 
25.25.25.106 .1.3.6.1.4.1.9.9.48 0.1.1.6.1.5.1 Did not find 'zeroDotZero' in module 
SNMPv2-SMI (/root/mibdir/MIBS/IP-MIB.my) 
enterprises.cisco.ciscoMgmt.ciscoL4L7moduleResourceLimitMIB.ciscoL4L7ResourceLimitMIBO
bjects.crlResource.ciscoL4L7BufferUtilizationTable.ciscoL4L7BufferUtilizationEntry.crl
ExternalBufferUsageValue.1 = Gauge32: 49815 buffers

New Software Features in Version A2(3.4)

The A2(3.4) software release provides the following new features:

Per CSCtn05967, the ACE20 running ACE software version A2(3.4) and installed in a Catalyst 6500 series switch running Cisco IOS software versions 12.2(50)SY and 12.2.(50)SZ1 supports the SUP2T.

Per CSCtd92176, this enhancement allows you to configure the ACE to reboot when the load-balancing process is stuck. The show np number me-stats command now includes the following load-balancing options:

lbabrt—Aborts the LB process

lboff—(Default) Disables the LB queue check

lbon—Enables the LB queue check

The syntax of this command is as follows:

show np number me-stats "-k lbabrt | lboff | lbon"

Per CSCtj84786, CSCtl57463, and CSCtj83501, the following buffer utilization changes were added to the ACE, as detailed in the "Monitoring and Displaying the Network Processor Buffer Usage" section:

The buffer threshold command in configuration mode allows you to enable the reboot of the standalone ACE when the buffer usage crosses the threshold. In redundant mode, the reboot of the active ACE occurs only when it reaches or crosses its threshold and the standby ACE is below its threshold.

The show np number buffer usage command in Exec mode allows you to display the buffer usage of each NP.

New buffer utilization syslog messages.

Per CSCtj83515, the three new SNMP OIDs were added to the ciscoL4L7BufferUtilizationTable for monitoring buffer utilization in the ACE through SNMP. See the "Related SNMP Changes" in the "Monitoring and Displaying the Network Processor Buffer Usage" section.

Per CSCtl94488, the following level-3 error syslog message is generated for scripted probe failures:

%ACE-3-251015: Scripted probe failed for server A.B.C.D, error message.
 
   

For more information, see the "Software Version A2(3.4) System Log Messages" section.

Per CSCtj65495 and CSCtl94225, the cookie-error-ignore command is deprecated and replaced by the parsing non-strict command in parameter map HTTP configuration mode. Also, the name of the cookie-error-ignore field displayed by the show parameter-map command was changed to parsing non-strict. For more information, see the "Skipping a Malformed Cookie in an HTTP Flow" section.

Per CSCtl74617, the new inspect non-persistence command in parameter map HTTP configuration mode allows you to configure the ACE to bypass connection persistence inspection during HTTP transactions for use with smooth streaming deployments. Also, the inspect non-persistence field is added to the show parameter-map command. For more information, see the "Bypassing Inspection during HTTP Transactions" section.

Per CSCtg07971, when you configure an FT track probe object without configuring an FT track host, the probe transitions to the DISABLED state as displayed by the show probe command. Previously, the probe transitions to the INVALID state.

Per CSCti76675, the new append-port-hosttag command allows you to configure the ACE to append the port information in the HTTP Host header when a nondefault port is used for an HTTP or HTTPS probe. By default, the ACE does not append the nondefault port. For more information, see the "Appending Nondefault Port Information in the HTTP Host Header" section.

Per CSCtj11142, the show running-config command has a new id option to filter the running-config file based on the ID. The syntax of the command is as follows:

show running-config [type [id]]

For example:

show run rserver rs1
show run serverfarm sf1
 
   

Per CSCsy91540 and CSCtn77149, the new system [no] watchdog hardware command in Exec mode allows you to enable the SiByte hardware watchdog. By default, the hardware watchdog is enabled.


Caution In some situations, this command causes the ACE module to become unresponsive and does not restart the ACE except in cases when SCP\LCP or some other emergency systems can handle the problem.

When SiByte hardware watchdog is enabled, it restarts the ACE when the following occurs:

The Linux kernel becomes unresponsive and cannot receive any IOCTL messages from uspace.

The CP uspace becomes unresponsive and the ACE is unable to fork new processes.

For example, to enable the hardware watchdog, enter the following command:

host/Admin# system watchdog hardware
 
   

To disable the hardware watchdog, enter the following command:

host/Admin# system no watchdog hardware
 
   

Per CSCtj65014, the new maxconn-one-np command in serverfarm host configuration mode allows the show commands to report that the real server is in the MAXCONN state when a single NP reports the real server reaches its limit. Thus, the global state of the real server can be MAXCONN before the configured limit is reached. If the MAXCONN limit on the other NP is not reached for this server, the server can still accept new connections, but never more than the global MAXCONN limit. For more information, see the "Reporting a Real Server MAXCONN State When One NP Reaches its Allocated Limit" section.

Per CSCtk33966, the connection limit of 4 million per real server has been removed. Previously, the ACE imposed a default connection limit of 4 million per real server.

Monitoring and Displaying the Network Processor Buffer Usage

When the ACE processes very heavy network traffic and the buffers of an IXP network processor (NP) reach their capacity, the ACE may become unresponsive and require a manual reboot. Per CSCtj84786 and CSCtl57463, the new buffer threshold command allows you to set threshold levels for the NP buffers for automatically rebooting the ACE. When you configure this command, the ACE checks the status of the NP buffer usage every five seconds.

When you configure this command in a standalone ACE and the buffer usage reaches or exceeds the configured threshold on either of its NPs, it reboots. In a redundant configuration, the configured thresholds for the active and standby ACEs determine whether the active ACE reboots and a switchover occurs, as follows:

The usage on either of its NPs in the active ACE reaches or exceeds the configured threshold.

The usage for all of the NPs in the standby ACE is below the configured threshold.

However, if the buffer usage in the standby ACE reaches or exceeds its configured threshold, the active ACE does not reboot and a switchover does not occur.

The buffer threshold command is available in configuration mode and the Admin context. The syntax of this command is as follows:

buffer threshold active number1% standby number2% action reload

The keywords and arguments are as follows:

active number1—Specifies the buffer threshold for the active redundant ACE or standalone ACE as a percentage. Enter 50, 75, 88, 95, or 100. There is no default value.

standby number2—Specifies the buffer threshold for the standby redundant ACE. Enter 10, 20, 30, 40, 50. There is no default value.

For example, to specify the active NP buffer utilization threshold as 88 percent and the standby NP buffer utilization threshold as 40 percent, enter the following command:

host1/Admin(config)# buffer threshold active 88% standby 40% action reload
 
   

To reset the default behavior, enter the following command:

host1/Admin(config)# no buffer threshold
 
   

Displaying the NP Buffer Usage

You can display the buffer usage of each NP by using the show np number buffer usage command in Exec mode. The syntax of this command is as follows:

show np number buffer usage

The number argument specifies the number of the NP for which you want to display buffer usage statistics.Enter an integer from 1 to 2.

Table 3 describes the fields in the show np number buffer usage command output when the buffer threshold command is configured.

Table 3 Output Fields of the show np number buffer usage Command

Field
Description

Total Internal Buffer

Total initial internal buffer space in bytes.

Internal buffer allocated

Amount of used internal buffer space in bytes.

Internal buffer usage

Amount of used internal buffer expressed as a percentage of the total initial buffer space.

Total External Buffer

Total initial external buffer space in bytes.

External buffer allocated

Amount of used external buffer space in bytes.

External buffer usage

Amount of used external buffer expressed as a percentage of the total initial buffer space.

Automatic reload

Status of the automatic reload feature:

Enabled—The buffer threshold command is configured.

Disabled—The buffer threshold command is not configured.

Active buffer threshold

Configured buffer usage threshold in the active ACE. This field is available only when the buffer threshold command is configured.

Standby buffer threshold

Configured buffer usage threshold in the standby ACE. This field is available only when the buffer threshold command is configured.


Related Syslogs for Buffer Usage

Per CSCtj83501, the following system log messages (syslogs) are generated when the internal or external buffer usage crosses 50 percent, 75 percent, 88 percent, 95 percent, and 100 percent.

The following warning syslog is generated when the buffer usage goes above the 50 percent threshold and falls below the 25 percent threshold:

ACE-4-443002:Available IXP 1 External Buffer reached above 75 percent threshold, Total 
buffer:65536, Available Buffer:60003
 
   

The following warning syslog is generated once when the buffer usage crosses the 50 percent threshold. The subsequent generation of this 50 percent syslog occurs only when the buffer usage goes below 25 percent and again crosses the 50 percent threshold.

ACE-4-443002:Available IXP 1 External Buffer reached below 50 percent threshold, Total 
buffer:65536, Available Buffer:31529
 
   

The following error syslogs are generated when the buffer usage crosses the 75 percent and 88 percent, respectively. The subsequent generation of these syslogs occurs once in five minutes if the same condition persists.

ACE-3-443003:Available IXP 2 Internal Buffer reached below 25 percent threshold, Total 
buffer:262144, Available Buffer:42102
ACE-3-443003:Available IXP 2 Internal Buffer reached below 12 percent threshold, Total 
buffer:262144, Available Buffer:22102
 
   

The following critical syslog is generated when the buffer usage crosses 95% and 100%, respectively. The subsequent generation of this syslog is once in 5 minutes if the same condition persists.

ACE-2-443004:Available IXP 2 Internal Buffer reached below 5 percent threshold, Total 
buffer:262144, Available Buffer:2102
 
   

An alert syslog is generated when the reload action occurs based on the configured buffer threshold command as follows:

ACE-1-443005:Available IXP2 Internal Buffer reached below number percent threshold, reload 
started.
 
   

The following critical syslog is generated when the NP buffer usage crosses 95% and 100%, respectively. The subsequent generation of this syslog is once in 5 minutes if the same condition persists.

%ACE-2-443005:Available NP 2 buffer reached below 5 percent threshold, Total 
buffer:155648, Available Buffer:7014
 
   

An alert syslog is generated when the reload action occurs based on the configured buffer threshold command as follows:

%ACE-1-443006:Available NP %d buffer reached below %d percent threshold, reload started
 
   

Related SNMP Changes

Per CSCtj83515, the ciscoL4L7BufferUtilizationTable was added to CISCO-L4L7MODULE-RESOURCE-LIMIT-MIB. The following SNMP OIDs in the ciscoL4L7BufferUtilizationTable display the NP buffer usage and percentage of buffer usage:

crlBufferUsageValue—Absolute buffer usage of the system

crlPercentageBufferUsage—Percentage of buffer usage in decimal format to allow historical information to be collected

crlPercentageBufferUsageDisplay—Percentage buffer usage in string format

Skipping a Malformed Cookie in an HTTP Flow


Note This feature was originally introduced in software version A2(3.3) with the cookie-error-ignore command. In software version A2(3.4) and later, the cookie-error-ignore command is deprecated. If you are upgrading from version A2(3.3) and have the cookie-error-ignore command in your configuration, you will receive a command exec error during the upgrade process.

In a redundant configuration, the standby ACE will remain in the WARM_COMPATIBLE state until you manually change the command configuration to the new syntax that is described below.

The functionality of this command has not changed; only the command name has changed. Also, the name of the cookie-error-ignore field displayed by the show parameter-map command was changed to parsing non-strict.


By default, when the ACE finds a malformed cookie in an HTTP flow, it stops parsing the remaining packets and drops the flow to Layer 4. You can use the parsing non-strict command in parameter map HTTP configuration mode to configure the ACE to ignore malformed cookies in a request and continue parsing the remaining packets in the flow. The syntax of this command is as follows:

parsing non-strict

For example, to configure the ACE to ignore a malformed cookie and continue parsing the packets in the flow, enter the following commands:

host1/Admin(config)# parameter-map http HTTP_PARAMMAP
host1/Admin(config-parammap-http)# parsing non-strict
 
   

To reset the ACE behavior to the default of stopping the parsing of packets in a flow when it finds a malformed cookie, enter the following command:

host1/Admin(config-parammap-http)# no parsing non-strict
 
   

Bypassing Inspection during HTTP Transactions

By default, when you configure an HTTP inspection policy, connection persistence inspection is enabled during HTTP transactions. However, this inspection can reduce the quality for video or MP4 content in streaming content deployments.

Per CSCtl74617, the inspect non-persistence command in parameter map HTTP configuration mode allows you to configure the ACE to bypass connection persistence inspection during HTTP transactions. Note that the ACE still inspects the initial packets (GET and response PDUs) in the same connections. The syntax of the command is as follows:

inspect non-persistence

For example, to configure this command, enter the following:

host1/Admin(config)# parameter-map http HTTP_PARAMMAP
host1/Admin(config-parammap-http)# inspect non-persistence
 
   

To reset the default behavior of enabling connection persistence inspection on an HTTP inspection policy, enter the following command:

host1/Admin(config-parammap-http)# no inspect non-persistence
 
   

To display whether the inspection persistence is enabled or disabled, see the inspect non-persistence field displayed by the show parameter-map command.

Appending Nondefault Port Information in the HTTP Host Header

By default, the ACE does not append port information in the HTTP Host header when you configure a nondefault destination port for an HTTP or HTTPS probe. Per CSCti76675, you can configure the ACE to append this information by using the append-port-hosttag command. The syntax of this command is as follows:

append-port-hosttag

For example, to configure the ACE to append port information in the HTTP Host header, enter the following command:

host1/Admin(config-probe-http)# append-port-hosttag
 
   

The following configuration is an example of this command:

probe http h1
port 8081
interval 10
passdetect interval 10
request method get url /index.html
expect status 200 200
append-port-hosttag
open 1
 
   

To reset the default behavior, enter the following command:

host1/Admin(config-probe-http)# no append-port-hosttag
 
   

Reporting a Real Server MAXCONN State When One NP Reaches its Allocated Limit

The ACE divides the total maximum connection (maxconn) limit for a real server between the two IXP network processors (NPs). Because each NP monitors the limit allocated to it, one NP may reach the limit before the other. Even though, one NP reaches the limit, the other NP continues to serve traffic to the server and the server continues to accept new connections. Only when both NPs reach their limit, the ACE reports that the global state of the server is MAXCONN as displayed by the show serverfarm or show rserver command, and the server stops receiving new connections.

Per CSCtj65014, the new maxconn-one-np command in server farm host configuration mode allows the show commands to report that the real server is in the MAXCONN state when a single NP reports the real server reaches its limit. Thus, the global state of the real server can be MAXCONN before the configured limit is reached. If the MAXCONN limit on the other NP is not reached for this server, the server can still accept new connections, but never more than the global MAXCONN limit.

The advantage of this new behavior is that if you configure a backup server farm, the backup activates as soon as one NP cannot handle new connections preventing drops.

The syntax of this command is as follows:

maxconn-one-np

For example, to configure this command, enter the following:

host1/Admin(config)# serverfarm SFARM1
host1/Admin(config-sfarm-host)# maxconn-one-np
 
   

To reset the default behavior, enter the following command:

host1/Admin(config-sfarm-host)# no maxconn-one-np
 
   

To display the state of the real server, see the state field displayed by the show serverfarm or show rserver command.

New Software Features in Version A2(3.3)

The A2(3.3) software release provides the following new features:

Configuring SNMP Peer Engine ID for the Standby ACE

Enabling SSL Rehandshake on All Contexts

Ignoring Malformed Cookies in a Request

Bypassing HTTP Parsing After Processing a Connection Request

Configuring SNMP Peer Engine ID for the Standby ACE

In prior releases, the ACE allowed you to configure an SNMP engine ID that applied to both the active and standby ACE. Per CSCth59753, you can configure a different engine ID for the standby ACE in a redundant configuration. The snmp-server engineid command in configuration mode includes the new peer engineid peer_value option. The syntax of this command is as follows:

snmp-server engineid local_value [peer engineid peer_value]

The local_value argument is the engine ID for the active ACE. If you do not enter the peer engineid value_2 option, the local_value argument applies to both the active and standby ACEs.

To change the value of an engine ID, you must change both values. Otherwise, the ACE displays the following error message:

Enter valid value for engineid/peer engineid
Either both should be same or both should change
 
   

To change the peer_value argument, you must also change the local_value argument, or visa versa, for example:

host/Admin(config)# snmp-server engineid 1234567892 peer engineid 2234567891
host/Admin(config)# snmp-server engineid 2134567892 peer engineid 2324567891
 
   

To change a configuration in which the active and standby engine IDs are different to a value that is the same value for both engine IDs, you must enter a value that is different for both IDs, for example:

host/Admin(config)# snmp-server engineid 2134567892 peer engineid 2324567891
host/Admin(config)# snmp-server engineid 4567892213
 
   

When synchronization occurs in a redundant configuration, consider the following:

When both the active and standby ACEs are running software version A2(3.3) and you configure different local and peer engine IDs on the active ACE, the active ACE sends the local engine ID as the peer ID to the standby ACE, and the peer engine ID as the local ID. For example, the running configuration on the ACEs will be similar to the following:

On the active ACE: snmp-server engineid 2134567892 peer engineid 2324567891

On the standby ACE: snmp-server engineid 2324567891 peer engineid 2134567892

When the active ACE is running software version A2(3.3) and standby ACE is running a software version less than A2(3.3) and you configure different local and peer engine IDs on the active ACE, the active ACE verifies that the software version on the standby ACE and sends only the peer engine ID as the local ID to the standby ACE. For example, the running configuration on the ACEs will be similar to the following:

On the active ACE: snmp-server engineid 2134567892 peer engineid 2324567891

On the standby ACE: snmp-server engineid 2324567891

When the active ACE is running a software version less than A2(3.3) and standby ACE is running software version A2(3.3) and since you can configure only one engine ID on the active ACE, the active ACE sends the engine ID to standby ACE. The local and peer engine IDs on the standby ACE will have the same value. For example, the running configuration on the ACEs will be similar to the following:

On the active ACE: snmp-server engineid 2134567892

On the standby ACE: snmp-server engineid 2134567892 peer engineid 2134567892

Use the no form of this command to delete the SNMP engine IDs. If you delete one engine ID, the other engine ID is also deleted.

Enabling SSL Rehandshake on All Contexts

By default, SSL rehandshake is disabled on the ACE. Previously, you could enable only SSL rehandshake at the SSL-proxy level by configuring the rehandshake enable command in the SSL parameter map and associate the parameter map with an SSL proxy server using the ssl advanced-options command.

Per CSCth85502, the ACE now allows you to enable SSL rehandshake for all contexts on the ACE by using the crypto rehandshake enabled command in configuration mode. This command is available in the Admin context. The syntax of this command is as follows:

[no] crypto rehandshake enabled

Use the no form of this command to reset the default behavior of disabling SSL rehandshake for all contexts on the ACE.

Ignoring Malformed Cookies in a Request

By default, when the ACE finds a malformed cookie in a flow, it stops parsing the remaining packets. Per CSCtj05814, the cookie-error-ignore command allows you to configure the ACE to ignore malformed cookies in a request and continue parsing the remaining cookies. This command is in parameter map HTTP configuration mode and the syntax is as follows:

[no] cookie-error-ignore

Use the no form of this command to reset the default behavior.

Bypassing HTTP Parsing After Processing a Connection Request

By default, the ACE performs HTTP parsing after it processes the CONNECT request. Per CSCti13494, the passthrough command configures the ACE to bypass HTTP parsing after it processes the CONNECT request. This command is available in policy map inspection HTTP class configuration mode and policy map inspection HTTP match configuration mode. It only works with a matching port misuse configuration and a CONNECT request.

The syntax of this command is as follows:

[no] passthrough [log]

The log option generates a log message for traffic that matches the class map when configured in policy map inspection HTTP class configuration mode or matches the inline match command when configured in policy map inspection HTTP match configuration mode.

The following configurations are examples of this feature:

When you configure a Layer 7 class map for tunneling protocols, the configuration is as follows:

(config)# class-map type http inspect match-any HTTP_INSPECT_L7CLASS
host1/Admin(config-cmap-http-insp)# 2 match port-misuse tunneling
host1/Admin(config)#
host1/Admin(config)# policy-map type inspect http all-match HTTP_INSPECT_L7POLICY
host1/Admin(config-pmap-ins-http)# class HTTP_INSPECT_L7CLASS
host1/Admin(config-pmap-ins-http-c)# passthrough log
 
   

When you configure a match statement for tunneling protocols, the configuration is as follows:

host1/Admin(config)# policy-map type inspect http all-match HTTP_INSPECT_L7POLICY
host1/Admin(config-pmap-ins-http)# match MATCH5 port-misuse tunneling
host1/Admin(config-pmap-ins-http-m)# passthrough log
 
   

Use the no form of this command to reset the default behavior.

Accounting Logs Include Passphrase Commands

Per CSCth55784, the ACE now masks the pass phrases for the following Exec mode commands in the accounting logs:

backup [all] pass-phrase text_string

crypto import [non-exportable] {bulk sftp [passphrase passphrase] ip_addr username remote_url}

crypto import [non-exportable] {ftp | sftp} [passphrase passphrase] ip_addr username remote_filename local_filename}

crypto import [non-exportable] {tftp [passphrase passphrase] ip_addr remote_filename local_filename}

crypto import [non-exportable] terminal local_filename [passphrase passphrase]

Previously, the ACE included these commands in the logs with their elocutionist in clear text.

With this behavior change, when the ACE includes any of these commands in the log, it masks the sensitive information with five stars. For example, when you enter the crypto import terminal local_filename passphrase passphrase command, the ACE logs the following:

crypto import terminal filename passphrase *****
 
   

New Software Features in Version A2(3.2)

The A2(3.2) software release provides the following new features:

Probing a Redirect Server

Disabling Connection Replication

Configuring a Multicast MAC Address for a Host

Configuring Inactivity Timeout for Connections in Switch Mode

Configuring the Compilation Timeout for Regular Expressions

Accounting Logs Containing Sensitive Information

Probing a Redirect Server

Per CSCtg31161, you can now add a probe to a redirect server. When you configure a probe under a redirect server, the ACE uses the state of the real server based on the probe result for load-balancing decisions.

You can configure only probes with an IP address in routed mode under a redirect server, real and server farm. You cannot associate a scripted probe to a redirect server.

The following configuration is an example of this feature:

probe tcp t1
  ip address 10.25.25.18 routed
  interval 10
  passdetect interval 10
  open 49
probe tcp t3
  ip address 10.5.55.5 routed
  interval 10
  passdetect interval 10
  open 1
probe tcp t4
  interval 10
  passdetect interval 10
  open 1
rserver redirect r1
  probe t3
  webhost-redirection http://3.111.1.100/index.html 302
  inservice
 
   
serverfarm redirect sf1
  probe t3
  rserver r1
    probe t1
    inservice
  rserver r2
    inservice
 
   

If you attempt to add a probe without an IP address in routed mode to a redirect server, the ACE displays the following error message:

Error: Only Probe in routed mode can be configured under a redirect server
 
   

If you try to remove the ip address ip_address routed option from a probe that is associated with a redirect server, the ACE displays the following error message:

Error: Cannot remove ip address option from a probe associated with redirect server
 
   

Caution We strongly recommend that you do not make any CLI changes when the ACE modules in a redundant configuration are running different software versions. Unexpected results may occur. Remove any new feature commands before performing a downgrade on the ACE.

Disabling Connection Replication

Before this release, connection replication was enabled by default and it could not be disabled. Starting with software version A2(3.2), you can disable connection replication by entering the following command in configuration mode:

ft connection-sync disable

If you are not concerned with state replication, you can disable connection replication and observe an increase in ACE performance. When you enable connection replication, it consumes CPU cycles that could be used for other purposes. When you disable connection replication, those CPU cycles are available for other processes within the ACE, which improves overall ACE performance.

To reenable connection replication after you have disabled it, enter the no ft connection-sync disable command.

To display the status of connection replication, enter either the show running-config command or the show ft group detail command.

Configuring a Multicast MAC Address for a Host

Per CSCtg31089, the static arp command in configuration mode now allows the configuration of the multicast MAC address for a host. The ACE uses this multicast MAC address while sending packets to the host. This enhancement allows the support of deployments that involve clustering (for example Checkpoint clustering). A host can be assigned a multicast MAC address with the arp command. The ACE does not learn the multicast MAC addresses for a host.

Configuring Inactivity Timeout for Connections in Switch Mode

Per CSCtf91257, the new switch-mode timeout command in configuration mode allows you to configure the inactivity timeout for TCP or UDP connections in Switch mode. The ACE forwards connections that do not match any VIP. In Switch mode, these connections have TCP normalization disabled and the inactivity timeout set to 2 hours and 15 minutes (8,100 seconds). Since UDP connections do not have a close protocol, this timeout defines their minimum lifetime. Therefore, this command was introduced to minimize the number of old connections, particularly UDP.

The syntax for this command is follows:

switch-mode timeout seconds

The seconds argument is the time period in seconds for idle connections after which the ACE disconnects the connection. Enter an integer from 1 to 65535. By default, the timeout is 8100 seconds.

For example, to configure a timeout of 10 seconds, enter the following command:

host/Admin(config)# switch-mode timeout 10
 
   

To reset the default timeout, enter the following command:

host/Admin(config)# no switch-mode timeout
 
   

Configuring the Compilation Timeout for Regular Expressions

Per CSCtg47919, the new regex compilation-timeout command in configuration mode allows you to configure the timeout for regular expression (regex) compilation. When you configure a regex and its compilation is longer than the configured timeout, the ACE stops the regex compilation. The syntax for this command is as follows:

regex compilation-timeout minutes

The minutes argument is the time period in minutes. Enter an integer from 1 to 500. The default timeout has been set to 60 minutes. This command is available only in the Admin context for an admin role and is applicable across all contexts.

For example, to configure a compilation timeout of 80 minutes, enter the following command:

host/Admin(config)# regex compilation-timeout 80
 
   

Accounting Logs Containing Sensitive Information

Per CSCtc87588, the ACE now includes the following configuration mode commands in the accounting logs:

[no] ldap-server host ip_address [port port_number] [timeout seconds] [rootDN "DN_string" [password bind_password]]

[no] radius-server key [0 | 7] shared_secret

[no] radius-server host ip_address key [0 | 7] shared_secret

[no] snmp-server community community_name

[no] snmp-server host ip_address [inform | traps] [version {1 | 2c} | {3 {auth | noauth | priv}}] community_ string_or_username

[no] snmp-server user user_name [group_name] [auth {md5 | sha} password1 [priv {password2 | aes-128 password2}] [localizedkey]]

[no] tacacs-server host ip_address key [0 | 7] shared_secret

[no] tacacs-server key [0 | 7] shared_secret

[no] username name1 [password [0 | 5] {password}]

Previously, the ACE omitted these commands from the logs because they contain sensitive information, such as a community name, shared secret, username, or password.

With this behavior change, when the ACE includes any of these commands in the log, it masks the sensitive information with five stars. For example, when you enter the snmp-server community community_name command, the ACE logs the following:

snmp-server community *****
 
   

Note The ACE logs the sensitive information for the following commands in plain text and does not mask it:

The backup pass-phrase command in Exec mode

The ip address command in KAL-AP UDP configuration mode

The credentials command in probe configuration mode


New Software Features in Version A2(3.1)

The A2(3.1) software release provides the following new features:

KAL-AP Node To Notify the GSS When Backup Server Farm is in Use

Secondary IP Address Enhancement

Admin Context is not Allowed to be Starved of all Resources

Configuring the ACE to Perform an SSL Rehandshake

KAL-AP Node To Notify the GSS When Backup Server Farm is in Use

Before this release, a redirect server farm was configured as the backup server farm. When the primary server farm failed, the redirect server farm redirected the client requests to other data centers.

With this release, when a backup server farm is in use, the ACE informs the GSS VIP and the new clients are redirected to another data center, while allowing the VIP to remain in the active state. When the primary server farm fails it informs the GSS. The GSS responds to the succeeding DNS requests with the IP address of the other data center.

The following KAL-AP command has been added to explicitly notify the GSS when the VIP's primary server is down:

kal-ap primary-oos

If you enter the kal-ap primary-oos command in policy map class configuration mode while the primary server farm is inactive and the VIP and the backup server farm are active, the load value for the VIP needs to be set to KALAP_OVERLOADED to ensure all the subsequent DNS requests are redirected. See the "Downgrading Effects on the kal-ap primary oos Command (Downgrade from version A2(3.1) to A2(3.0))" section.

Secondary IP Address Enhancement

The secondary IP address feature allows you to configure multiple IP subnets under the same VLAN. With this release, the maximum number of secondary IP addresses increases from 4 to 15 on a single interface.

Admin Context is not Allowed to be Starved of all Resources

When you configure resource allocations, it is possible to allocate hundred percent of resources to user contexts, which might result in Admin context being starved of resources, leading to denial of few of the important services like redundancy, Telnet, ICMP, SNMP, and SSH.

In order to make sure the Admin context is never denied of these important services, in case the user happens to allocate all the resources to user context, we will by default be reserving the following resources for Admin contexts so that the management or redundant services to the box are not denied.

For the redundancy to be functional, a small percentage of the resources need to be set to the values defined in Table 4.

Table 4 Resources

Resources
Value

Concurrent connections

100 connections

Management connections

100 connections

Throughput Rate

10 Mbps

Management Traffic rate

10 Mbps

Connection Rate

100 connections per second


Configuring the ACE to Perform an SSL Rehandshake

In prior releases, the ACE automatically performed an SSL rehandshake when necessary because rehandshake was enabled by default. Starting with this release, SSL rehandshake is disabled by default and a new CLI command has been added to explicitly enable this functionality. See CSCtd00816 in the "Software Version A2(3.1) Resolved Caveats" section for more details. The syntax of this command is as follows:

rehandshake enable

Configure this command under an SSL parameter map and associate the parameter map with an SSL proxy server using the ssl advanced-options command. To display the status of the rehandshake enable command, enter the show parameter-map command.

New Software Features in Version A2(3.0)

The A2(3.0) software release provides the following new features:

HTTP insert of SSL session, client, and server header information (see the Cisco Application Control Engine Module SSL Configuration Guide)

SSL redirect on SSL session setup failure (see the Cisco Application Control Engine Module SSL Configuration Guide)

Sample SSL certificate and key (see the Cisco Application Control Engine Module SSL Configuration Guide)

Backup and restore of configuration files, licenses, SSL certificates and keys, and checkpoints (see the Cisco Application Control Engine Module Administration Guide)

SNMP MIB and trap enhancements (see the Cisco Application Control Engine Module Administration Guide)

Failaction reassign across VLANs (see the Cisco Application Control Engine Module Server Load-Balancing Guide)

Secondary IP address support for multiple subnets on the same VLAN (see the Cisco Application Control Engine Module Routing and Bridging Configuration Guide)

Large configuration download optimization (see the Cisco Application Control Engine Module Administration Guide)

ACE Operating Considerations

When the SYN cookie limit/threshold is reached as seen in the output of the show syn-cookie command, POP or SMTP connections to a VIP do not work. This behavior is the same for any type of TCP-based application where the server sends data first after a TCP 3-way handshake, (for example, applications like FTP, Telnet, POP, and smtp). In this case, when a connection comes in to a configured Layer 4 TCP VIP with a Layer 7 generic load-balancing policy for POP, SMTP, FTP, or Telnet servers, the front-end connection stays open and the backend connection is never established by the ACE to the backend server. For example:

ace/context# show conn address 10.10.10.1 netmask 255.255.255.255
 
   
conn-id    np dir proto vlan source                destination           state
----------+--+---+-----+----+---------------------+---------------------+------+
185        2  in  TCP   204  10.10.10.1:56126    10.20.20.112:80       ESTAB
--         -  -   --    --   --                    --                    -- 
 
   

As a workaround, disable the SYN cookie feature (no syn-cookie) or reconfigure a Layer 7 load-balancing policy without the generic keyword.

When an ACE is configured to report the VIP status using KALAP and more than one VIP with the same IP address is used (the VIPs have the same IP address, but different ports), the ACE reports all VIPs as down (load of 255) if only one fails.

If you configure an ACL on an interface to block certain traffic and a management policy on that same interface allows that traffic, the management policy overrides the ACL and the ACE allows the traffic.

We strongly recommend that you do not make any CLI changes when the ACE modules in a redundant configuration are running different software versions. Unexpected results may occur. Remove any new feature commands before performing a downgrade on the ACE.

When the active ACE module is running software version A2(3.X) and the standby ACE is running a software version earlier than A2(3.0), do not make any IP address configuration changes including the peer and alias addresses on the active ACE. If you do and incremental synchronization occurs, the IP addresses on the standby ACE are lost.

In software release A2(3.0), the default UDP inactivity timeout was changed from 120 seconds to 10 seconds.

The ACE requires a route back to the client before it can forward a request to a server. If the route back to the client is not present, the ACE cannot establish a flow and drops the client request. Make sure that you configure the appropriate routing to the client network on the ACE VLAN where the client traffic enters the ACE module.

Software version A2(1.0) introduces hardware-assisted SSL (HTTPS) probes. For that reason, the ACE uses the all option for the default SSL version and uses the routing table (which may bypass the real server IP address) to direct HTTPS probes to their destination regardless of whether you specify the routed option in the ip address command. If you are using HTTPS probes in your A1(6.x) configuration with the default SSL version (SSLv3) or without the routed option, you may observe that your HTTPS probes behave differently with version A2(1.x) or higher. For more information about HTTPS probes, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

Additionally, hardware-assisted probes are subject to the same key-pair size limitations as SSL termination. The maximum size of a public key in a server SSL certificate that the ACE can process is 2048 bits. For more information about HTTPS probes, see the Cisco Application Control Engine Module Server Load-Balancing Guide.

By design, if you set the maximum resources for sticky to unlimited using the limit-resource command, the ACE ignores the setting and sets the maximum value to equal-to-min. In addition, the maximum resource value for sticky in the show resource usage command output displays as 0. This behavior occurs because the ACE does not allow sticky resources to become oversubscribed as with other configurable resources. Instead, when the sticky resource usage reaches the minimum value, the ACE ages out older sticky entries in the sticky table and reuses them for new sticky entries.

In software version A2(1.2), the maximum number of match statements per ACE has been increased from 4,096 to 16,384.

The Total Conn-failures counter in the show rserver detail command displays the total number of connection attempts that failed to establish a connection to the real server.

For Layer 4 traffic with normalization on, the count increments if the three-way handshake fails to be established for either of the following reasons:

- An RST comes from the client or the server after a SYN-ACK.

- The server does not reply to a SYN. The connection times out.

For Layer 4 traffic with normalization off, the count does not increment.

For Layer 7 traffic (normalization is always on), the count increments if the three-way handshake fails to be established for either of the following reasons:

- An RST comes from the server after the front-end connection is established.

- The server does not reply to a SYN. The connection times out.

In software version A2(2.0), the ACE supports a maximum of 3800 certificate-key pairs.

In software version A2(2.0), the ACE now supports an SSL filename with a maximum of 39 characters.

When you downgrade the ACE software, the features and commands of the higher release are lost because they are not supported by the lower release.

Per CSCsz87533, the outbound UDP connection may timeout shortly after the ACE receives a RADIUS request, but before it gets the response for this request from the server. This situation can cause the ACE to improperly forward subsequent RADIUS traffic. If the server is not expected to initiate connections through the ACE, we recommend that you apply an inbound ACL on the server interface to block these connections.

When redundant ACEs lose connectivity (for example, because of a network interruption) and they attempt to reestablish their connection, if you enter the show ft peer or show ft group command during this time, the response to this command may be delayed.

If you are using the Application Networking Manager (ANM) to manage an ACE module and you configure a named object at the ACE CLI, ANM does not support all of the special characters that the ACE CLI supports for a named object. If you use special characters that ANM does not support, you may not be able to import or manage the ACE using ANM.

When naming ACE objects (such as a real server, virtual server, parameter map, class map, health probe, and so on) for use with ANM, enter an alphanumeric string of 1 to 64 characters, which can include the following special characters: underscore (_), hyphen (-), dot (.), and asterisk (*). Spaces are not allowed.

In software release A2(3.2), the organization-name command in CSR parameters configuration mode now supports the ampersand (&) character. When you configure this command with the & character on an active ACE running software version A2(3.2) and then you perform an incremental synchronization to a standby ACE running software version A2(3.1) or earlier, the ACE allows the synchronization to occur. However, after you save the configuration on the standby ACE and perform a reboot, the ACE displays the following error message:

*** Context number: cmd exec error ***
 
   

In software version A2(3.2), the sample keys and certificate feature was available in the Admin context only. In prior releases, the feature was also available in the user context. If you upgrade from software version A2(3.0) or A2(3.1) to A2(3.2), the sample certificate and key for the existing context remain as they are. However, the sample certificate and key are not be seen in new contexts. If you upgrade from software version A2(2.X) or earlier, sample certificates and keys are present in the Admin context only.

When you remove a NAT pool configuration, wait more than five seconds before adding a NAT pool with the same ID.

The Account Expiry field for the show user-account command displays the date, if any, when the user account expires. This date is based on Coordinated Universal Time (UTC/GMT) which the ACE keeps internally. If you use the clock timezone command to configure a UTC offset, this field displays the UTC date and does not reflect the date with the offset as displayed by the show clock command.

When HTTP parsing for cookie is not configured on the ACE, including match cookie or cookie sticky, HTTP is less strict on the types of errors it raises during header value parsing. When cookie parsing is involved, parsing for a header becomes more strict.

In software version A2(3.4) per CSCtk33966, the connection limit of 4 million per real server has been removed. Previously, the ACE imposed a default connection limit of 4 million per real server.

In software version A2(3.4) per CSCsv84674, you cannot configure a space in the names of most object types. In prior software releases, the CLI allowed spaces; however, the ACE displays an error when you apply the configuration. Table 5 lists the object types, previous release behavior, and the behavior in software version A2(3.4).

Table 5 Spaces in Object Type Names 

SNo
Object Type
Whether the ACE allows spaces in the name
Release prior to A2(3.4) 1
A2(3.4)

1

context

no

no

2

classmap

yes

no

3

policymap

yes

no

4

kalap tag

yes

no

5

Probe

yes

no

6

acl

yes

no

7

l2_acl

yes

no

8

action_list

yes

no

9

if_zone

IF_ZONE config not supported

IF_ZONE config not supported

10

Object group

yes

no

11

ssl_proxy

yes

no

12

sticky_group

yes

no

13

serverfarm

yes

no

14

rserver

yes

no

15

access-list

yes

no

16

crypto(authgroup, chaingroup, crl, csr-param)

yes

no

17

hostname

yes

no

18

parametermap

yes

no

19

role

yes

no

20

resource-class

yes

no

21

username

no

no

22

aaa-group

yes

no

23

domain

yes

no

24

ip domain-name, ip domain-list

yes

no

25

script file, cert file, key file

yes

yes

26

ft trace

yes

no

27

peer host

yes

no

28

inline match

yes

no

29

snmp-server community

yes

no

30

Static Sticky

yes

yes

1 An error occurs on the ACE when you apply the configuration.


Software Version A2(3.6a) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

This release note includes the resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.6a) and changes to commands and system log messages:

Software Version A2(3.6a) Resolved Caveats

Software Version A2(3.6a) Open Caveats

Software Version A2(3.6a) Command Changes

Software Version A2(3.6a) System Log Messages

Software Version A2(3.6a) Resolved Caveats

The following resolved caveats apply to software version A2(3.6a):

CSCsv49606—QNX crash in the load balance module with back traces pointing to the sticky code after a period of time. The back traces show a corruption in the sticky database. Workaround: None.

CSCtc49976—When the configuration includes one or more Layer 4 policies, the show cfgmgr internal table slb-policy command shows all the entries as invalid which is incorrect. Workaround: None.

CSCth99982—When you configure an ECHO TCP probe with a send-data value and the server returns a different value, the ACE does not match what was sent and passes the probe by default.For the echo probe to pass, the response echoed back from the server must be same as the configured echo probe send-data. Workaround: Use a TCP probe with send-data and regex values as required instead of an ECHO TCP probe.

When you configure send data on a probe and the server returns something other than send data, the does not match that with what was sent and pass the probe by default. In an echo probe, the send data must be the same as what is echoed back from the server.

CSCth80666—When you apply the same object group to multiple ACLs, an ACL merge may fail. Workaround: Reapply the ACL.

CSCti68100—When SNMP v1 trap is sent with the agent address 0.0.0.x, if you select a bridged VLAN as the snmp-server trap-source, ace uses the bvi internal interface id to fill in the agent address instead of the bvi interface IP address. Workaround: Use a non-bridged VLAN.

CSCtj19438—In a bridged mode, a user on subnet A cannot ping a VIP IP on subnet B. Workaround: Create a VIP match type of "any" so that ICMP will be load balanced directly to the server.

CSCtl43296—Configuration of the SSL termination with a SSL key file which contains invalid prime numbers. Workaround: Use a different key.

CSCtl50549—RHI routes are not added in the new active SUP. Workaround: None.

CSCtn51752—When you remove and add reals in the serverfarm, the show process memory output shows a increase in the cfgmgr process memory. Workaround: None.

CSCtn96378—The retcode with check is removed, having reset timer set configured and conn-limit is enabled inside rserver. When the threshold is reached the rservers will be moved to MAXCONNS and RETCODE-FAILED state. But as RETCODE-FAILED is based on the response from server and this state is reflected on the output of sh serverfarm <name> detail command. And after the resume timer expires, the rservers should be moved to MAXCONNS state and not OPERATIONAL. Workaround: None.

CSCtn82912—When multiple contexts are configured, and the back-end SSL is configured in one context and traffic that enters the ACE at 1000 connections per second (cps) that hits both a server farm attached to the default class and the SSL policy, the first two GET requests are successful but subsequent requests fail. Eventually, the active ACE and then the standby ACE become unresponsive and generate multiple core dumps. Workaround: None.

CSCto52883—The ACE fails when SIP codenomicon suite is run. Workaround: None.

CSCto57801—When regex optimization is enabled, the configuration contains one multimatch policy with more then one VIP class, and an SSL proxy and an SSL parameter map are configured, if you make any changes in the SSL parameter map first and then in the server farm, a traffic failure may occur. Workaround: Reload the ACE.

CSCto74099—When stickiness is configured in multiple contexts, sticky entries in one context may point to real servers that are configured in another context. Workaround: Reboot the ACE. Another possible workaround is to enter the no sticky-serverfarm command followed by the sticky-serverfarm command on the affected policy map to cause a new download of the configuration data.

CSCto88742—SSL probes may fail intermittently even if the ACE is in standby mode. The issue is not seen on the FT peer ACE. Workaround: Reload the ACE.

CSCtq06460—When a real server is associated with a server farm and you change the IP address of the server to the subnet of the interface, the state of the server changes to ARP_FAILED. Workaround: None.

CSCtq07279—A rare race condition may cause the proxy free list on the IXP to become corrupt and the ACE drops connections due to a proxy resource limit. Workaround: Reload the ACE.

CSCtq92249—If a class has multiple ICMP inspects and you delete the inspect protocol, the following error occurs from the second time onwards. Error: This class does not have ICMP protocol.

CSCtq93957—ACL Merge fails to add ACE in the context admin, an error is displayed while processing the service-policy. Incomplete rule is currently applied on interface VLAN108. Configuration on this interface needs to be manually reverted. Also if you add the match statements with the line number and that line number has to match with other ACL downloaded line number. Workaround: Reload the ACE.

CSCtq99402—The ACE fails for some other reason and the core process gets interrupted due to a kernel watchdog timeout. Workaround: Disable the sibyte hardware Watchdog, be entering the system no watchdog hardware command.

CSCtr18423—When sticky resources are configured, the clear stats resource-usage command does not clear the sticky peak counter in the output of the show resource-usage command. Workaround: Reload the ACE.

CSCtr23640—After you enter the clear stats resource-usage command, the peak counter for syslogs is not cleared properly in the output of the show resource-usage command. Workaround: Reload the ACE to clear the peak counters.

CSCtr27089—When troubleshooting problems with ICMP ping requests to Virtual IP (VIP) addresses, the information contained in the output of the show cfgmgr internal table icmp-vip may be incorrect. Workaround: None.

CSCtr35832—VIP should go OOS when the real server moves from MAX_CONN to RETOCDE-FAILED. Workaround: Readding the virtual server to the layer 3 service-policy.

CSCtr46862—The ACE returns an error below and fails in downloading the CRL file. Workaround: None.

CSCtr52981—During a configuration change, the config manager may become unresponsive due to a memory corruption problem and the ACE reloads. Workaround: None.

CSCtr59788—Sometimes, on an ACE that has a redundant configuration and is running software version A2(3.4), the active and standby ACEs show the policy statements in the reverse order. Workaround: None.

CSCtr61540—If a specific memory location is corrupted, the ACE may not become unresponsive and reboot even if the microengine is detected in a hung state. Workaround: None.

CSCtr77869—When the configuration manager sends a message to TCP and the message has a proxy ID that is out of bounds, the network processor microengine (ME) becomes unresponsive and the ACE reloads with a last boot reason of "NP 1 Failed : NP ME Hung" or "NP 2 Failed : NP ME Hung". Workaround: None.

CSCtr98547—ACE IMAP probe fails with some IMAP servers that have configured the name of the mailbox from which the probe retrieves e-mail by using the "credentials mailbox" command. Workaround: 1) Do not use "credentials mailbox". 2) To use IMAP TCL scripted probe.

CSCts15700—When VIP is configured for server-conn reuse, persistence-rebalance strict and header modify per-request, TCP connections destined for SSL-Proxy VIP are stuck in the connection table in the CLSRST state beyond TCP IDLE timeout configured. Workaround: None.

CSCts20134—When you enter a command that contains a regex expression, the control plane (CP) management access stops working because the configuration manager is suspended while compiling the regex expression. Workaround: Reboot the ACE and manually edit the startup-config file to restore service.

CSCts31272—If any context has 'priority' as part of its name, it causes the admin context to go to the STANDBY COLD state. This issue is seen only when there is a Bulk sync of fault tolerance not during an incremental synchronization. Workaround: Delete the context that has 'priority' in its name and rebuild it with a new name.

CSCts34544—A regex download status shows as successful even when the regex download fails. Workaround: None.

CSCts35610—A denial of service vulnerability is found in the way the multiple overlapping ranges are handled by the apache HTTPD server. Multiple Cisco products could be affected by this vulnerability.

CSCts37668—When you configure ACE with large number of contexts, interfaces and ACLs, ACE ACL merge process hangs 10-15 mins after every boot which pushes the leaf parameter nodes and policy action nodes over the limit. Workaround: Check the show np 1 access-list resource after you boot, if the leaf parameter nodes go above 400K and policy action nodes goes above 200K. Remove ACLs and contexts until this threshold is not breached. Recommended values are 200K and 100K respectively.

CSCts42523—The show cde count command displays the undercut min pkt size counters. Workaround: None.

CSCts44196—When two access-lists are attached to the same object group and delete an access-list which is not attached to the interface, delete and readd the same host, readd the access-list back which will eventually cause an ACL Merge Error. Workaround: Attach the second access-list also to the interface.

CSCts50348—When the show service-policy detail or show service-policy url-summary commands are performed, the console hangs and does not return until <ctrl-c> is entered. Workaround: Break out of the command with <ctrl-C>.

CSCts50354—The ACE reboots with forced cores after you enter the show np 1 me-stats -udp command. Workaround: None, reload the ACE.

CSCts57337—When the user tries to install a temporary license for 20 virtual contexts and it fails with a invalid current count. Workaround: Reload the ACE and the license would install after that.

CSCts63455—Multiple requests of snmpget at the same time on the cpmProcessTable may cause SNMP timeouts. Workaround: Perform only sequential SNMP requests on the Cisco Process MIB.

CSCts74652—When the ACL configuration is modified sometimes an ACL merge error is reported on one or more of the interfaces where the ACL list is applied. This leaves the interface in an inconsistent state. Workaround: 1. Remove the offending lines one at a time until the ACL can be applied sucessfully. 2. Reload the ACE.

CSCts81413—%ACE-5-441003 syslog message is not logged even when server farm goes down. This issue is observed when a sticky server farm is configured on the policy map. Workaround: None.

CSCts85839—When the backend SSL traffic is running on ACE, ACE fails with cores. Workaround: None.

CSCts84756—The NAT pool xlate entries were in a dump state and only one of the processors was able to reset the connection via TCP. Workaround: None, reload the ACE.

CSCts85878—The config sync between the active and standby ACE fails with the standby ACE staying in the STANDBY_CONFIG state. After every four hours, the HA config sync eventually times out and the standby ACE changes to the STANDBY_COLD state. Workaround: Remove the banner from the running config and startup config of the context.

CSCts90705—The show service policy detail command does not have the correct XML tags. Workaround: None.

CSCts91300—You may notice a connection drop on the ACE and ACE service modules due to unavailable buffer. This issue is due to improper handling of specially crafted HTTP GET request to the ACE VIP. The issue is verified only if layer 7 inspection is configured. Workaround: Clear the connection which will clear all the stale connections and release the buffer.

CSCts91307—The ACE reloads with core dumps and it fails when it tries to free the same buffer for the second time. Workaround: None.

CSCts99918—Connections to the ACE are reset because the total buffer usage exceeds 75 percent. Workaround: None.

CSCtt04516—The following log messages appears:

%ACE-3-251006: Health probe failed for server x.x.x.x on port nnnnn, internal error: failed to setup a socket.

Workaround: None.

CSCtt46566—The server farm state in the sh cfgmgr internal table icmp-vip command output does not match with the output of show service-policy and show serverfarm details commands. Workaround: None.

CSCtu02155—When you use HTTP to HTTPS webhost redirection and uses special characters in the beginning of patch URL, such as double slash '//' causes the application error. Workaround: None.

CSCtu11517—When real servers move from MAX-CONN and send few more connections to move RETCODE_FAILED state. VIP moves to OSS, if resume-service configured in the serverfarm. VIP should go to the INSERVICE state after expiring the resume-time limit. Workaround: None.

CSCtu22841—When you add match statements in a single download cycle With SLB and inspect related configurations, addition of layer 3 rules causes an ACL merge error. Workaround: After you add each layer 3 rule a few seconds delay needs to be allowed for completion of the download thread.

CSCtu24904—The show ip fib output should include the explanation for the V flag in the legend. Workaround: None.

CSCtu33866—The command to look at the source of a script file is failing. The command is show script code <filename> and it fails saying as an invalid call. Workaround: Reboot the ACE.

CSCtu66133—The IP address, protocol, port, and match conditions from the show service-policy command output does not show up in the XML output. The server farm 'hit count' and 'dropped cons' output from the show service-policy command also do not show up in the XML format. Workaround: None.

CSCtu67809—Deleting and adding match statements in a single download cycle causes an ACL merge error. Workaround: After adding each layer 3 rule, a few seconds delay for completion of the download thread.

CSCtw49216—The show system resources command displays the free memory decrease on the ACE. show proc memory command displays a constant increase of the memory used by the SNMP process. Workaround: Reduce the frequency of the SNMP queries and reload the ACE to clear the memory usage.

CSCtw52452—The ACE does not forward the ICMP pings to some real servers over one network processor. Also, drops (redundant connections) are increasing in the show np X me-stats -socm output for the impacted NP. Workaround: None.

CSCtw64687—When ACE30 is configured in the bridging mode, a DHCP client on a VLAN behind the ACE30 is unable to obtain a lease from a DHCP server. Workaround: Use a different DHCP server where the reply is broadcasted instead of unicasted.

CSCtw70805—The ACE is configured as a FT pair with one side running on A2(1.4) and the other ACE being upgraded to A2(3.5). When performing a code upgrade on a fault tolerant (FT) pair, the ACE fails repeatedly and sends out a configuration manager core file. Workaround: Repeat the upgrade procedure again, isolate the ACE, and upgrade as a standalone.

CSCtw70879—If the load balancer queue monitoring logic is incorrect, it causes an invalid memory access and in turn the ACE would reboot and write out a crash file. Workaround: None.

CSCtw70891—When the load balancer xscale process fails when the xscale processor core files are zero in length. Workaround: None.

CSCtw73707—The ACE module does not populate the MAC address table for certain servers on secondary IP subnet. Workaround: Configure primary IP with the less specific subnet mask.

CSCtw78317—When accounting ON/OFF requests are pumped at high rates to the radius VIP, ACE module hits a low memory condition on the DP resulting in radius traffic failure. Workaround: None.

CSCtw80862—When account On/Off messages are sent at a high rate, total proxy mapper errors show up on show stats loadbalance radius output. Workaround: None.

CSCtw80868—HTTPS probe failures and SSL proxies are reset. This issue occurs mostly on a single network processor (NP) but can occur on all NPs. Workaround: Reload the ACE.

CSCtx23587—The ACE has static ARP entried even though no static ARPs are configured, if the static ARPs are configured in the past and removed, ACE sometimes fails to remove the entry. Workaround: Readd that entry in the ACE and then remove it to remove the static ARP from the ACE.

CSCtx30088—When a show command for an ACL larger than 91 lines is run using the ACE XML API, it creates additional XML tags after the 91st line. Workaround: Limit the ACL line numbers to 91lines or less.

CSCtx38552—The ACE timer list gets corrupted and causes the traffic to that network processor (NP) to stop. Workaround: Reload the ACE.

CSCtx67184—HTTP requests from the client hits incorrect layer 7 policy-map statement and is load-balanced to incorrect serverfarm. Workaround: None.

CSCtx82167—FT track priority does not work as expected after rebooting the ACE. This isuue occurs when multiple track priorities are configured on the FT track host configuration and some track states are displayed as TRACK_DOWN. Workaround: Reconfigure FT track priority such as no probe [probe_name] priority [priority]/probe [probe_name] priority [priority] or change all the track states from TRACK_DOWN to TRACK_UP.

CSCtx86297—The probe name and type under "Probe(s) :" output are not displayed when the show serverfarm detail command is entered. Workaround: Use show serverfarm [serverfarm] detail command instead of the show serverfarm detail command.

CSCtx98325—A file transfer gets terminated after 16 KB is transferred. Workaround: Configure an HTTP parameter map and set the content-maxparse-lengtlh and header-maxparse-length to larger values.

Software Version A2(3.6a) Open Caveats

The following open caveats apply to software version A2(3.6a):

CSCtn47103—During a time when the ACE was not sending probes, a probe failure occurs because of a server open timeout (no SYN ACK, as confirmed through tcpdump on the server). Workaround: Reboot the ACE.

CSCto00168—When you enter the show resource usage command, the sticky field displays 4294965431. Workaround: Reboot the ACE.

CSCto00198—Sticky entries are not inserted into the sticky table due to a lack of resources and the following syslog is displayed:

%ACE-LB_STICKY-3-728007: Internal communications error (sticky) -- type 4
 
   

This issue may be related to probe failures that cause extra downloads and a possible disruption in bank switch downloads. Workaround: Force a failover or reload the ACE.

CSCtr39136—An HTTP probe with a regex search string fails when the regex string is in the second packet. This issue causes the probes to pass and fail intermittently. Workaround: The server needs to send the 200 OK message and the regex search string in one packet.

CSCtu74012——When using an HTTP probe on the ACE, if the response (not the header) contains "content-length: 0", the ACE fails the probe with an "Unrecognized or invalid response" error even if the response is 200 ok from the server. Workaround: If you remove the "-" from the content-length and just use "contentlength", the ACE accepts the server response and will not fail the HTTP probe. Another alternative is to use a "head" instead of a "get" on the URL request method.

CSCtx00078—HTTP requests from the client hit incorrect layer 7 policy-map statements and are load balanced to the incorrect server farm. Workaround: None.

CSCtx04321—When the ACE downloads the CRL for the first time from the CRL location, the update is not attempted even after the next-update timer expires. The CRL download is not attempted by the ACE after the next-update timer expires. Workaround: None.

CSCtx53917—The ACE A235 module fails to reboot if it is run with 20 contexts and has traffic on all contexts. Workaround: None.

CSCtx92211—The network processor fails the loadBalance_g_ns on a standby ACE. Workaround: None.

CSCty04652—When the active ACE20 is reloaded and goes into the standby mode, there is a window of less than a second when an layer 2 loop occurs between the two ACE20s. The multicast HSRP packets from the new active ACE reach the standby ACE20 through the backend bridged VLAN, and the standby ACE20 bridges these multicast HSRP packets to the front-end VLAN. These packets can loop several thousand times in less than a second which causes the CPU on the Sup to spike. Workaround: If multicast traffic is normally bridged from VLAN A to VLAN B, apply an ACL on VLAN B that blocks this traffic in the reverse direction from VLAN B to VLAN A.

CSCty18004—The HTTP probe fails if the data has the control character (NULL) "\0" as part of the data. This issue is not seen in a normal UNIX server from which the file is fetched but is seen only with IXIA because it can manipulate the data. Workaround: The data/header portion of the reply should not have a NULL character.

CSCty37843—When you enter the show service-policy detail or show service-policy url-summary commands, the show service-policy command gets hung on ACE20 and only a part of the output is shown on the console. Workaround: Break the command with <ctrl-C>.

CSCty54552—When the ACE management IP or VIP is pinged by ICMP requests with data files size smaller than 18 bytes, ACE adds additional data to the ICMP replies to make the data field at least 18 bytes long. It happens starting from the second ICMP reply from ACE. Workaround: None.

CSCty74359—When two DNS queries are sent using the same 5-tuple to the ACE 20 running A2(3.5) and inspect DNS is configured, the second response from the DNS server is sometimes dropped. Workaround: If the two client requests are spaced out, the problem doesn't occur or disable inspect DNS.

CSCty74438—In this case the probe with the same name "SIPPROBE' exists on both HA systems. However it is a TCP probe on one and a UDP probe on the other. The probe is also attached to serverfarms on the both systems. Workaround: During the config synchronization, to bring the config in synchronization, the standby tries to remove and re-add the 'SIPPROBE', however since it is tied to a serverfarm, it cannot be removed resulting in a config synchronization failure.

Software Version A2(3.6a) Command Changes

Table 7 lists the new and changed commands in software version A2(3.6a).

Table 6 CLI Command Changes in Version A2(3.6a)  

Mode
Command and Syntax
Description

Exec

show probe detail

Per CSCth08113, added the new regex cache-len output field to complement the cache option of the expect regex command. See below in this table.

show running-config

Per CSCtq31721, HTTP and HTTPS probes no longer display the default open timeout value in the running-configuration.

show ip route

Per CSCti56893, when NAT is applied on an interface, the show ip route command now displays the interface name.

show ip

Per CSCtu24904, the overflow (V) flag now displays the legend explanation in the show ip fib command.


Software Version A2(3.6a) System Log Messages

Software version A2(3.6a) includes the following new system log (syslog) messages.

251010

Error Message    %ACE-3-251010: Health probe failed for server address on port number, 
Server response not matching with configured echo probe send-data

Explanation    Per CSCth99982, when you configure an ECHO TCP or UDP probe on the ACE and the server sends a regex that does not match the configured send-data value, the probe fails and the ACE generates a syslog message.

Recommended Action    Check the service running on the affected server.

322006

Error Message    %ACE-3-322006: Inconsistant arp entry for 25.25.25.99

Explanation    Per CSCtx23587, if an inconsistancy is detected in the ARP cache and the static_flag, a syslog message is displayed.

Recommended Action    None. This syslog is informational only.

441003

Error Message    %ACE-5-441003: Serverfarm (sf1) failed in policy_map (lb) --> 
class_map (#class_default_slb) without backup. Number of failovers = 1, number of 
times back in service = 2

Explanation    Per CSCts81413, a syslog message is generated when the sticky server farm goes down and the syslog message is the same as that of normal server farm failure.

Recommended Action    None. This syslog is informational only.

Software Version A2(3.5) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

This release note includes the resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.5), and changes to commands and system log messages:

Software Version A2(3.5) Resolved Caveats

Software Version A2(3.5) Open Caveats

Software Version A2(3.5) Command Changes

Software Version A2(3.5) System Log Messages

Software Version A2(3.5) Resolved Caveats

The following resolved caveats apply to software version A2(3.5):

CSCsr05599—The ACE considers the time between the last retransmission of the request and its response as the response time for the transaction. The response time should be the time between the first transmission of the request packet and the time the ACE received the response from the real server. Workaround: None.

CSCsz38667—When you remove the key and certificate from the SSL-proxy service, you cannot delete it from the policy map when it is invalid. Workaround: Add a dummy key and certificate to the previously configured SSL-proxy service.

CSCte05724—In a redundant configuration, the standby, with a minimum configuration in a user context and no traffic, becomes unresponsive after applying a configuration to it that was downloaded from a TFTP server under the following conditions:

Created a checkpoint for the minimum configuration

Performed a bulk sync import of SSL keys and certificates on both the active and the standby

Copied the downloaded configuration to the active

Created a checkpoint for the copied configuration

Observed an MTS error on the standby, but the ACE recovered.

Performed a checkpoint rollback from the copied configuration to the minimum configuration checkpoint

Performed a checkpoint rollback from the minimum configuration to the copied configuration

Workaround: None.

CSCtf38653—class-map type HTTP loadbalance matching source-address are ignored. Client that should match the source address does not match the class-map. Under the same policy-map type loadbalance, the following two classes are defined:

First class is defined nested so matches on source address (i.e. 10.0.0.0/24) and other (i.e. match http header).

Second class matches the source address of a wider subnet mask than the first one (i.e. 10.0.0.0/16), client hitting this policy map should be of subnet matching the first class map (i.e. 10.0.0.0/24) but not matching the http header.

Workaround: The subnets defined in the second class-map should match the same subnets defined in the first class-map.

CSCtf44818—The ACE module occasionally displays the incorrect value in the Unicast bytes input counter for the interface. This issue can cause problems for SNMP tracking the traffic, which in turn displays ~50Gbps flowing through the ACE. Workaround: Configure the SNMP application to ignore the counter increases above a certain value.

CSCtf86359—When you add a certificate to an existing chain group on the active ACE and the certificate does not exist on the standby ACE, the standby ACE should change to the FT COLD state but it does not. Workaround: To correct the state, perform one of the following.

On the FT group, enter the no inservice command followed by the inservice command.

In configuration mode, enter the no ft auto-sync running-config command followed by the ft auto-sync running-config command.

CSCtg09928—Replacing an existing load-balancing policy in a multi-match policy with a Layer 7 HTTP load-balancing policy with header insertion does not activate header insertion. Workaround: Remove and re-add the multi-match policy with a Layer 7 HTTP header insert policy already included in it.

CSCtg70913—When users whose accounts have expired attempt to log in to the ACE through SSH or Telnet, they succeed. Workaround: None.

CSCtg77964— If client authentication is performed with a client certificate that has an error (for example, an unknown signer), best-effort CRLs and the authentication-failure ignore command are configured, and the client certificate does not have any CRL distribution points (CDPs), the initial connection is rejected as expected (because of the unknown signer), but subsequent handshakes are allowed. Workaround: Disable the cdp-ignore command.

CSCth30569—When you apply a large multi-context configuration, the arp_mgr service in ACE becomes unresponsive. Workaround: None.

CSCth75674—When the SCP HW watchdog on the ACE fails to detect the timer expiry, the Catalyst 6500 supervisor power cycles the ACE with an SCP keepalive failure message. The watchdog may fail to detect the timer expiry when the internal counters overflow. The SCP HW watchdog mechanism detects when the ACE becomes unresponsive and collects the core files in error case scenarios which prevents the power cycling by the Catalyst 6500 supervisor. On rare occasions, the watchdog may fail to detect the timer expiry. Workaround: None.

CSCtj70903—When you configure an engine ID on the ACE and then configure users, if you change the engine ID to another value, the ACE deletes the users. However, if you reconfigure the original engine ID value, the ACE restores the deleted users. The ACE should permanently delete the users after you change the engine ID. Workaround: None.

CSCtj79482—When you configure a real server on the ACE, assign it an IP address, place it in service, and then delete it, the ACE generates an unnecessary trap. When the real server state changes from ARP-FAILED to OPERATIONAL, the ACE generates the CesRServerStateUp trap. Workaround: None.

CSCtj91891—When a configured TCP probe becomes active on the ACE and the server sends out-of-band data to the ACE, the ACE reboots and generates an hm_core file. Workaround: None.

CSCtl93050—In a redundant ACE configuration with SNMP polling in place, the ACE reboots in the snmp daemon. The network management server collects the following informations using SNMP from both ACE modules:

Once in four hours, sysName and sysDescr. These values were not read before the time when the ACE module spontaneously reboots.

Every minute, sysUptime and ciscoCpuUtil (1.3.6.1.2.1.1.3.0 and 1.3.6.1.4.1.9.9.109.1.1.1.1.7.1). Before the ACE spontaneously reboots, these variables could be read (sysUptime or ciscoCpuUtil).

Workaround: None.

CSCtl97127—Each time that the standby ACE reboots, a context on it transitions to the STANDBY_COLD state and the ACE displays the following error:

Error on Standby device when applying configuration file
 
   

It is a timing issue due to the configuration size and total number of contexts. This issue can lead to a lot of Configuration Manager (CFGMGR) download processing which can lead to a command failure. CSCtn50357 is tracking the issue of the actual failing command that is not properly placed in the error logs. Workaround: Perform either of the following:

On the FT group for the context in the STANDBY_COLD state, enter the no inservice command followed by the inservice command.

Change the context FT group ID in the FT group to a higher number so that the context with the largest configuration does the configuration synchronization last.

CSCtn14301—When a control store parity error occurs, the reason is not set properly. The me_dumper process needs to be fixed to identify the control store parity error. Workaround: None.

CSCtn26839—On a rare occasion when you remove a VIP from the configuration, the ACE continues to have an ARP entry for the VIP and it causes a traffic routing problem. Workaround: Reboot the ACE.

CSCtn52695—If you configure inconsistent netmasks similar to the following on the ACE, the CLI should prevent or notify you of the inconsistency through a syslog message:

access-list acl1 extended deny ip any 10.45.15.192 0.0.0.15
access-list acl1 extended deny ip any 10.45.15.192 0.0.9.0
 
   

Workaround: None.

CSCtn72817—When you upgrade an SSL certificate, the ACE sends an RST for existing connections. The ACE handles the subsequent new session requests for these connections with the new certificate. Workaround: Upgrade the SSL certificate in a maintenance window.

CSCtn89970—When you enter the show resource usage command, it displays the output for total bandwidth allocation that is unclear. The bandwidth should be the average of the throughput and the management traffic, but there was no throughput updating. This is a display issue only. Workaround: None.

CSCtn90010—When SNMP polling occurs to the local IP address of the ACE module and the ACE receives a malformed SNMP packet or heavy utilization of SNMP polls, its SNMP daemon may delay in responding by 10 to 15 minutes. Workaround: None.

CSCtn93329—When redundant ACEs generate SIP probes with the same Call-ID and From-Tag options, the SIP registrar servers interpret these probe messages as duplicates and do not reply to them causing SIP health probes to fail. Workaround: None.

CSCtn96791—When you enable FTP inspection on the ACE, the server responds to the client FIN but does not include its own FIN. The client sees the first FTP transaction succeed, but subsequent transactions fail. The connection is left half open between the ACE and the server. This issue causes problems for future client transactions. Workaround: Set the connection timeout for half-closed TCP connections to a low value. For example:

parameter-map type connection ftpftp
  set tcp timeout half-closed 1
 
   
policy-map multi-match VIP
  class VIP1
    loadbalance policy VIP1
    loadbalance vip icmp-reply
    connection advanced-options ftpftp
 
   

CSCto03171—When an SSL certificate or key is in use on the ACE, you can delete it. Workaround: Before removing the certificate or key, manually verify whether it is being referenced in the configuration.

CSCto05999—When you use a previously used IP address for a MAC address in an ARP entry for a real server, the ACE displays an ARP entry without a next scheduled ARP time. The entry does not time out and you cannot clear it by using the clear arp ip command. Workaround: Use a new IP address for the real server.

CSCto11694—With certain types of content, the ACE changes the UDP checksum, which causes the request to drop on the server. Workaround: None.

CSCto24128—Under rare circumstances, the ACE may experience an unexpected reboot with a core dump of the loadBalance_g_ns_core process. This event may occur when you are running high traffic with configuration changes on the control plane. For example:

Adding and deleting real servers

Taking a real server in and out of service

Adding and deleting the match statements in a Layer 7 class map

Adding and deleting a Layer 7 policy from a multimatch policy

Workaround: None.

CSCto34197—In the rare event that the communication between an ACE10 or ACE20 module and the supervisor engine in the chassis is interrupted, the ACE module may reload and generate a crashinfo file. This behavior has been changed so that the diagnostic information file is still generated, but the ACE does not reload. Now, the supervisor engine initiates the reload due to a keepalive polling failure. Workaround: None.

CSCto52883—When running SIP traffic with system logging enabled, the ACE may become unresponsive. Workaround: None.

CSCto64389—You cannot disable the driver of the SiByte hardware watchdog from the ACE CLI. You can only disable it in the debug plugin. Workaround: To disable the driver, run echo 1 > /proc/watchdog. To enable the driver, run echo 0 > /proc/watchdog. To display the status, run cat /proc/watchdog; dmesg | tail.

CSCto68363—When SSL session ID reuse and SSL client authentication are configured, they may place a load on the ACE CPU that can cause the ACE to reboot and generate a core file. Workaround: Remove SSL session ID reuse and SSL client authentication to help mitigate the CPU problems.

CSCto70197—When the last VIP address is removed and reapplied to a Layer 4 class map, the VIP address is not updated in the ARP cache table. Workaround: Remove and re-add the service policy under the client interface.

CSCto72289—When logging is enabled, the copy commands do not appear in the system accounting log. Workaround: None.

CSCto86720—In a redundant configuration, the snmp-server user command profile may be deleted on the standby ACE. This behavior can occur when all the following conditions are met:

The ACE pair is configured with a user profile that has automatically created an SNMP server user profile.

Each ACE module is configured with a unique SNMPv3 engine id

The Standby ACE reloads.

Workaround: None.

CSCto91867—In a redundant configuration, the standby ACE snmp-server user password auth md5 password command hash and the privacy password (if configured) synchronize with those of the active ACE. This action results in an SNMPv3 access failure to the standby ACE. This behavior can occur when all the following conditions are met:

The ACE pair is configured with an snmp-server user command profile with or without a privacy password.

Each ACE module is configured with a unique SNMPv3 engine ID

The standby ACE reloads.

Workaround: None.

CSCto93671—Under rare circumstances, multiple status changes of a tracked interface may cause the context to go into the STANDBY_HOT state on both modules. Workaround: N/A.

CSCtq31851—If you place a real server in the inservice standby mode and you have configured the leastconns, least-loaded, or response server farm predictor, configuration manager may become unresponsive and the ACE reloads. Workaround: Use the roundrobin predictor for the affected server farm.

CSCtq34823—In the rare event that the IPCP Hi priority queue becomes full and if there is a DP error, the SME can become stuck in the IPCP kernel code while it is trying to collect the core from the DP or while it is sending an ACK to the DP. Workaround: None.

CSCtq40973—If the ACE is configured for SSL termination and the ACE SSL function encounters an unexpected condition while processing an SSL connection, the ACE module may reload. Workaround: None. The ACE automatically reboots and clears the condition.

CSCtq44523—When multiple requests are sent over the same UDP connection in quick succession, the ACE intermittently fails to source NAT the traffic. The issue is not seen if the packets do not arrive in parallel. Workaround: If a linux device is initiating the connection, send the packets serially instead of in parallel. See the following URL:

https://bbs.archlinux.org/viewtopic.php?id=75770

CSCtq46504—When the ACE is configured with a context that performs RADIUS and user data load balancing (RLB) and another context that performs firewall loadbalancing (FWLB) using reverse sticky for return traffic, occasionally, after a config change in the RLB context, the FWLB context may perform destination NAT on the traffic sent to the firewalls that belong to a transparent server farm. The destination IP address that is used is the IP address of a real server configured on the other context performing RLB.

Workaround: Because the trigger for this issue is configuration change, the best workaround is that, after any change related to the RLB context and RADIUS load balancing (class map, server farm, real server, policy, sticky, and so on), perform a reload of the ACE to make sure that the issue is cleared. Other possible workarounds are:

Force an FT switchover of the RLB context to the standby ACE or keep the RLB context active on another ACE

Performing one additional config change in the RLB context may clear the issue.

CSCtq59957—The backup of a context may fail if the context contains a pass phrase-protected SSL key and both the key and the backup pass phrase are long. Workaround: Use a shorter key name or backup pass phrase or avoid protecting the key with a pass phrase.

CSCtq68743—When a server farm is configured for maxconns and all real servers in the server farm fail their probes while in the MAXCONNS state, the VIP state does not change to OUTOFSERVICE. Workaround: Remove the probe from the server farm and reapply it.

CSCtq91322—When traffic is bridged on the ACE and is one hop away from the ACE with a globally applied ACL, ACL merge fails for some of the VLANs that are in the bridge groups. Workaround: Reapply the global ACL or apply the ACL on the interface instead of globally on all interfaces.

CSCtr23173—When regex resources are in use, the clear stats resource-usage command does not clear the regexp peak counter in the output of the show resource-usage command. Workaround: Reload the ACE.

CSCtr23730—When end-to-end SSL is configured and a real server sends a TCP RST instead of a FIN, the ACE forwards the TCP RST to the client without sending an SSL CLOSE NOTIFY. Workaround: Configure SSL termination on the server and configure a Layer 4 VIP on the ACE.

CSCtr54076—When the ACE is servicing a high volume of traffic, has sticky configured, and the show tech command is executed continuously on the control plane, the ACE may become unresponsive. Workaround: None.

Software Version A2(3.5) Open Caveats

The following open caveats apply to software version A2(3.5):

CSCta92891—If you change the load-balance predictor from least conns to hash URL with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCth99982—When you configure an ECHO TCP probe with a send-data value and the server returns a different value, the ACE does not match what was sent and passes the probe by default. Workaround: Use a TCP probe with send-data and regex values as required instead of an ECHO TCP probe.

When you configure send data on a probe and the server returns something other than send data, we do not match that with what was sent and pass the probe by default. In an echo probe the send data must be the same as what is echoed back from the server.

CSCtj67085—When the ACE has Layer 7 traffic greater than 40K CPS and you enter a show command that displays more than 3 to 5 lines of output, the Telnet management connections to the CP may become unresponsive. Workaround: None.

CSCtn47103—During a time when the ACE was not sending probes, a probe failure occurs because of a server open timeout (no SYN ACK, as confirmed through tcpdump on the server). Workaround: Reboot the ACE.

CSCtn82912—With multiple contexts configured, back-end SSL configured in one context, and traffic entering the ACE at 1000 connections per second (cps) that is hitting both a server farm attached to the default class and the SSL policy, the first two GET requests are successful, but subsequent requests fail. Eventually, the active ACE and then the standby ACE become unresponsive and generate multiple core dumps. Workaround: None.

CSCto00168—When you enter the show resource usage command, the sticky field displays 4294965431. Workaround: Reboot the ACE.

CSCto00198—Sticky entries are not inserted into the sticky table due to a lack of resources and the following syslog is displayed:

%ACE-LB_STICKY-3-728007: Internal communications error (sticky) -- type 4
 
   

This issue may be related to probe failures causing extra downloads and a possible disruption in bank switch downloads. Workaround: Force a failover or reload the ACE.

CSCto04222—When you configure RADIUS traffic and maxconn limit on the real servers, the ACE does not correctly update some connection counters for the servers. The server farm and real servers remain in the MAXCONN state when there is no traffic. Workaround: None.

CSCto57801—When regex optimization is enabled, the configuration contains one multimatch policy with more then one VIP class, and an SSL proxy and an SSL parameter map are configured, if you make any changes in the SSL parameter map first and then in the server farm, a traffic failure may occur. Workaround: Reload the ACE.

CSCto74099—When stickiness is configured in multiple contexts, sticky entries in one context may point to real servers that are configured in another context. Workaround: Reboot the ACE. Another possible workaround is to enter the no sticky-serverfarm command followed by the sticky-serverfarm command on the affected policy map to cause a new download of the configuration data.

CSCto88742—SSL probes may fail intermittently even if the ACE is in standby mode. The issue is not seen on the FT peer ACE. Workaround: Reload the ACE.

CSCtq06460—When a real server is associated with a server farm and you change the IP address of the server to the subnet of the interface, the state of the server changes to ARP_FAILED. Workaround: None.

CSCtq07279—A rare race condition may cause the proxy free list on the IXP to become corrupt and the ACE drops connections due to a proxy resource limit. Workaround: Reload the ACE.

CSCtq87773—When there are many probe failures during a config sync, the tnrpc queue becomes full, and the config manager is not able to handle the messages. This issue causes the standby ACE to enter the STANDBY_COLD state. Workaround: None.

CSCtr18423—When sticky resources are configured, the clear stats resource-usage command does not clear the sticky peak counter in the output of the show resource-usage command. Workaround: Reload the ACE.

CSCtr23640—After you enter the clear stats resource-usage command, the peak counter for syslogs is not cleared properly in the output of the show resource-usage command. Workaround: Reload the ACE to clear the peak counters.

CSCtr27089—When troubleshooting problems with ICMP ping requests to Virtual IP (VIP) addresses, the information contained in the output of the show cfgmgr internal table icmp-vip may be incorrect. Workaround: None.

CSCtr33375—When you apply the same object group to multiple ACLs, ACL merge may fail. Workaround: Reapply the ACL.

CSCtr39136—An HTTP probe with a regex search string fails when the regex string is in the second packet. This issue causes the probes to pass and fail intermittently. Workaround: The server needs to send the 200 OK message and the regex search string in one packet.

splitting them into 2 or 3.

CSCtr52981—During a configuration change, the config manager may become unresponsive due to a memory corruption problem and the ACE reloads. Workaround: None.

CSCtr59788—Sometimes, on an ACE that has a redundant configuration and is running software version A2(3.4), the active and standby ACEs show the policy statements in the reverse order. Workaround: None.

CSCtr61540—If a specific memory location is corrupted, the ACE may not become unresponsive and reboot even if the microengine is detected in a hung state. Workaround: None.

CSCtr77869—When the configuration manager sends a message to TCP and the message has a proxy ID that is out of bounds, the network processor microengine (ME) becomes unresponsive and the ACE reloads with a last boot reason of "NP 1 Failed : NP ME Hung" or "NP 2 Failed : NP ME Hung". Workaround: None.

CSCts15816—If the ACE is fetching a file and you abruptly terminate an end-to-end SSL or SSL initiation (back-end SSL) connection using Ctrl-c, the ACE leaves the connection open as observed in the output of the show conn command. The ACE sends a CLOSE-NOTIFY alert and a FIN after the connection is closed on the client side. The result is a half-closed SSL connection on the back end. The server-side connection closes after the server sends a FIN or a RST. If the server becomes idle, the back-end connection closes when the idle timeout expires. Front-end SSL connections are not affected by this behavior. SSL connections that close gracefully do not have any issues. This behavior is not observed when HTTP connections (either proxied or unproxied) are closed abruptly. Workaround: None.

Software Version A2(3.5) Command Changes

Table 7 lists the new and changed commands in software version A2(3.5).

Table 7 CLI Command Changes in Version A2(3.5)  

Mode
Command and Syntax
Description

Exec

show probe detail

Per CSCth08113, added the new regex cache-len output field to complement the cache option of the expect regex command. See below in this table.

show running-config

Per CSCtq31721, HTTP and HTTPS probes no longer display the default open timeout value in the running-configuration.

Configuration

snmp-server bulk-request {max-oid number1 | max-repetition number2}

Per CSCtn90010, added the snmp-server bulk-request commands. For the number1 or number2 argument, enter an integer from 0 to 2147483647. Both 0 and 2147483647 are default values and indicate that the command is disabled. These commands act as filters for incoming SNMP bulk requests and they allow you to limit the maximum number of OIDs that can be processed by the ACE and the maximum number of repetitions that can be returned in an answer. When the filter is triggered and the debug snmp trace command is enabled, the following messages appear on the console:

Limit Max Repetition to sysMaxRepetition 
(reqMaxRepetition)
Limit quantity OIDs to sysMaxOID
 
        

If you encounter any issues with the ACE being flooded with SNMP bulk requests, then you should set the argument values of these commands manually. For example:

host1/Admin(config)# snmp-server bulk-request max-oid 
50
 
        
host1/Admin(config)# snmp-server bulk-request 
max-repetition 100

snmp-server user user_name [group_name] [auth {md5 | sha} password1 peer password2 [priv [aes-128] password3 peer password4] [localizedkey]]

Per CSCto91867, added the peer password option for the auth and priv options to enter a password for the peer user.

Probe configuration

expect regex value [cache [number]]

expect regex value offset number1 [cache [number2]]

Per CSCth08113, added the cache option for HTTP probe regex parsing. This option enables regex parsing in cached mode for HTTP and HTTPS probes which overcomes the issues with regex parsing and large web pages. Enter a cache value from 1 to 1000 bytes. The default is 1000 bytes.

Server farm host configuration

retcode number1 number2 check {count
| {
log threshold_number reset seconds1}
| {
remove threshold_number reset seconds1 [resume-service seconds2]}

Per CSCtj20245, changed the lower limit of the log threshold_number value and the remove threshold_number value to 2.

show serverfarm name np

Per CSCtl50901, added the new np keyword to display the state of the real server on each network processor (NP).


Software Version A2(3.5) System Log Messages

Software version A2(3.5) includes following new system log (syslog) messages.

106029

Error Message    %ACE-6-106029: ACL name configured with invalid netmask

Explanation    Per CSCtn52695, the ACE generates this syslog message when you configure a non-standard netmask for either a source or destination IP address in an ACL config.

Recommended Action    Configure a valid network mask for the IP address.

111009

Error Message    %ACE-7-111009: User username executed cmd: copy_command 

Explanation    Per CSCto72289, the ACE generates a syslog for all copy commands.

Recommended Action    None required. This syslog is informational only.

Software Version A2(3.4) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

This release note includes the resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.4), and changes to commands and system log messages:

Software Version A2(3.4) Resolved Caveats

Software Version A2(3.4) Open Caveats

Software Version A2(3.4) Command Changes

Software Version A2(3.4) System Log Messages

Software Version A2(3.4) Resolved Caveats

The following resolved caveats apply to software version A2(3.4):

CSCsv04459—When you configure several hundred action-list statements and create a checkpoint for this configuration, if you perform tests by rolling back between different checkpoint configurations, eventually you cannot configure any additional action lists and the ACE displays the following error:

Error: Action list not found.
 
   

You can configure only 4093 action-list statements on the ACE, even if you delete them and readd them. Workaround: Reboot the ACE.

CSCsv84674—When you create a real server, class map, policy map, KAL-AP tag, server farm, or context name that includes a space in it, the ACE redundant configuration can go out of synchronization. Workaround: Do not use spaces when naming an object on the ACE.

CSCsv95689—When you enable the logging persistent command, it allows the ACE to save a specified syslog to its flash memory. As expected, the ACE creates a "messages" file on disk0. However, after you delete this file, the logging persistent command does not work again until you remove and reconfigure the command. Workaround: Remove and reconfigure the logging persistent command.

CSCsy10000—When you enter the clear stats loadbalance radius command, the RADIUS framed-ip, username and calling-station sticky counts displayed by the show stats loadbalance radius command do not clear. Workaround: None.

CSCsy91540—When the supervisor engine detects that the ACE is not responding to keepalives, the ACE may silently reboot and not generate core dump files. Workaround: None.

CSCsz62556—When you apply connection limits by entering the conn-limit command at the real-server level and connection limits are already applied at the server-farm level, some real servers may remain in the stopped list and stop performing load balancing. Workaround: Reboot the ACE.

CSCta49917—When Telnet connections, SSH connections, or a debug session are active for a long time on the ACE, they do not close properly as indicated by one of the following:

After the execution of each changeto command, the MTS buffers increase as displayed by the show system internal mts buffers command.

The following error message occurs:

IPC queue full. Clear idle telnet/ssh connections or debug plugin sessions to 
recover err
 
   

Workaround: You can either Telnet to each context to make configuration changes, clear each Telnet or SSH session to the ACE with the clear line vty_name command, or reboot the ACE.

CSCtb03138—In a redundant configuration, if you configure SNMP traps on a VLAN that is not configured with an IP address or the peer IP address, the active ACE does not synchronize the SNMP traps to the standby ACE. The show ft group detail command displays the following error:

Error "Incremental Sync Failure: snmp config sync to sby." 
 
   

Workaround: Configure both an IP address and a peer IP address on the interface VLAN that you are using as the trap source.

CSCtb19070—If different sessions to an ACE issues the show np command at the same time, the displayed information may be incorrect. Also, the ACE may unexpectedly reboot and generate a core file. Workaround: Do not enter the show np command in different sessions to the ACE at the same time.

CSCtb77652—When you configure the failaction reassign or failaction across-interface command and you enter related show commands or attempt to ping from the ACE, the ICMP and ARP manager generate an RPC call failure message. Workaround: Do not configure the failaction reassign or failaction across-interface command.

CSCtd57287—When you copy the following configuration and apply on the ACE, the INVALID probe state is displayed by the show probe command:

serverfarm SF
    real RS
       no inservice
       no probe P1
       probe P2
       inservice
 
   

Workaround: Perform either of the following:

Change the sequence of commands in the configuration before applying it, as follows:

serverfarm SF
	real RS
	no inservice
	no probe P1
	inservice
	probe P2
 
   

Apply the configuration without the no inservice or inservice command:

         serverfarm SF
            real RS
            no probe P1
            probe P2
 
   

CSCtd92176—Per a request to check the health of all load-balancing threads, this enhancement allows you to configure the ACE to reboot when it detects that the LB process is stuck. Workaround: None.

CSCte68680—When remote AAA server are configured in multiple contexts and XML requests through HTTP are sent to multiple contexts, occasionally the ACE reboots when the AAA daemon becomes unresponsive. For this configuration, the structure for the session is getting freed. After freeing, the session.vcid element is used for printing causing the AAA daemon to become unresponsive. Also the other local variable is used for printing. Workaround: None.

CSCte84966—When the ACE performs the snmpwalk command on the cpmProcessTable, the show proc cpu command becomes unresponsive.The output of the show system internal mts buffers command displays an MTS leak. The output of the show system internal mts buffers details command confirms this leak. Also, the MTS sends error messages similar to the following:

mts_do_msg_input() failing since no space available in 91 (src_sap = 91, opc = 1376 
PID = 934) 2
 
   

Workaround: None.

CSCtg07971—When you configure an FT track host probe without an FT track host, the probe transitions to the INVALID state. Workaround: Configure an FT track host under the FT configuration.

CSCtg46241—During a high rate of SIP calls per second and during the initial processing of packets, if the SIP inspection engine encounters resource allocation failures (such as memory allocation, object allocation, and inspect config version mismatch failures), the ACE may reboot. Workaround: Disable the SIP inspection feature, if possible.

CSCtg74007—If a redundant configuration detects any component license mismatch between the active and standby ACE in the Admin context, the show ft group detail command displays the license mismatch and that the running synchronization is disabled. However, the running synchronization functionality is working correctly. Workaround: None.

CSCti68514—When RTSP traffic is running on the ACE, the SIP session allocated delta counter increments without SIP traffic. Workaround: None.

CSCti72201—After you correct a license mismatch on a standby ACE, the show ft group detail command displays the following:

Running cfg sync enabled: Disabled 
 
   

Workaround: Reboot the standby ACE.

CSCti76675—When you change the default destination port for an HTTP probe, the probe does not append the port to the Host tag in the HTTP request and the ACE receives an HTTP/1.1 404 Not Found error. Workaround: Configure the probe with the header Host header-value command to specify and append the destination port to the host in the HTTP request.

CSCtj18891—When you configure a primary VLAN interface as a server VLAN on the ACE, the ACE does not load balance to the real servers on the secondary VLAN. Workaround: After performing the configuration, reboot the ACE module for traffic to work properly.

CSCtj20335—When a real server is in the RETCODE FAILED state and you add a a probe that fails to the server, the server transitions to the OPERATIONAL state after the resume seconds elapses. Instead, the server should be in the DISABLED state on the data plane. Workaround: None.

CSCtj21592—When you configure two or more probes to a server farm, the probe instance is not created after removing one of the probes on the server farm and entering the inservice command on the real server. Workaround: None.

CSCtj35994—When you configure a user context on the ACE for KAL-AP, the ACE unexpectedly reboots and generates a gslb_proto_log.943.tar.gz core file. The last boot reason is Service "gslb_proto". Workaround: None.

CSCtj54534—Under normal operating conditions with logging configured, you cannot disable a specific syslog with kernel Id 901001. Workaround: Disable logging.

CSCtj61334—When you configure SNMP traps on the ACE, any reason (for example, an entire server farm goes down) generates a large number of traps. When the SNMP trap queue is full, the ACE displays error messages similar to the following:

snmpd[1027]: (ctx:9)send_notification: new: enqueueing notification........ 
snmpd[1027]: (ctx:9)ERROR: notif_enqueue_tail : Size of the notif queue is more than 
the MAX size 250 
 
   

You can also display error messages by enabling the debug snmp errors and debug snmp notif_trace commands. Workaround: Use syslogs instead of traps. If possible, decrease the number of SNMP traps and the rate that they are sent.

CSCtj90752—When you boot the standby ACE in a redundant ACE configuration with syslog enabled, the ACE displays the following syslog message:

%ACE-LB_HA-5-728028: Sticky mapping failed:. Invalid sticky group id 
 
   

Workaround: None.

CSCtj90760—In a redundant ACE20 configuration with approximately 600 server farms and 600 sticky groups, and when traffic is running on the standby ACE, if you remove the FT group and the context from the standby ACE, it reboots. When the sticky entries are replicated to the standby (in this case, outbound entries), the context ID is loaded from the FT group statistics. When you remove an FT group from the configuration, the ACE sets the context ID to invalid. Instead of passing the loaded context ID to the sticky insertion API with an integer, the ACE passes the value using the pointer inside the FT group statistics. There are no further verifications to validate the context ID. The result is the loading of the wrong sticky statistics group and the triggering of a segment fault. Workaround: None.

CSCtj90771—When a redundant switchover occurs, the mixed syslog messages from the actual HA state change message (ha_mgr) and the handling HA state change message (lb) may cause confusion. The LB_HA is for troubleshooting purposes only and should not be confused with the actual HA state change syslog message. For example, in a redundant configuration with syslog enabled, the first four messages at boot time are not related to actual HA state changes:

switch/Admin# ft switchover 
This command will cause card to switchover (yes/no)? [no] yes 
 
   
%ACE-LB_HA-6-728029: HA state for FtGroup 1 changed from 'StartPeriodicSync' to 
'StopSync' State 'handled' 
%ACE-LB_HA-6-728029: HA state for FtGroup 1 changed from 'StartPeriodicSync' to 
'StopSync' State 'handled' 
%ACE-LB_HA-6-728029: HA state for FtGroup 1 changed from 'StartPeriodicSync' to 
'StopSync' State 'handled' 
%ACE-LB_HA-6-728029: HA state for FtGroup 1 changed from 'StartPeriodicSync' to 
'StopSync' State 'handled' 
 
   
%ACE-2-727012: HA: FT Group 1 changed state to FSM_FT_STATE_STANDBY_REAP. Event: 
FSM_FT_EV_RELINQUISH 
 
   

Workaround: None.

CSCtj91335—When you configure a server farm with one real server and configure return code (retcode) using the maximum supported log and reset thresholds, the retcode feature does not work. The expectation is that the ACE sends retcode syslog messages when it observes 4294967295 retcode counts and 4294967295 seconds for the interval. When 1000 requests are sent in one minute, the retcode data on both DP and CP indicates that the current count is at 4 and the total is at 1000. The current count should be at 1000. When additional requests are sent, the current counter did not go above 4. Workaround: None.

CSCtj91674—When no captures were running across any of the contexts on the ACE, the tcpdump process consumes over 50 percent of the CPU usage and may become stuck. Workaround: To stop the tcpdump process, load the tcpdump_stop_nmicic_dplug.bin dplugin. Stop all captures when you load this dplugin.

CSCtj95951—When you enter the show script code script_name command, the display output terminates unexpectedly with an internal error and a VSH core file in the core: directory. The VSH core file does not reboot the ACE. Workaround: None.

CSCtj99706— When you enter the hw-module module number reset command on the supervisor for the ACE module, the ACE creates a Crashinfo file. Workaround: None.

CSCtk01422—When an improper TCP client requests data from the ACE and does not accept all of the data, and the connection on the ACE continuously probes the client TCP receive window (TCP.RCV_WND), traffic to the ACE may fail due to high network processor buffer utilization contained in a small number of extremely long-lived TCP connections. In some improper client TCP implementations, the client continues to send non-zero length segments while advertising a zero window. Another type of improper implementation might occur when the client sends FIN segments indefinitely to the ACE while advertising a zero window. In both the non-zero segment and the FIN cases, the ACE consumes one buffer for each packet until the connection is closed or the client advertises a nonzero window. Workaround: To identify the connections in the connection table, enter the show conn detail command and search for connections that are idle (for hours or more) on the outbound side but are not idle on the inbound side. To recover the buffers for an offending flow, clear the flow by entering the clear conn flow protocol source_ip source_port dest_ip dest_port command.

CSCtk06846—When the add compile time asserts for LB structures are out of bounds or are not cache aligned, some errors may occur at compilation time. Workaround: Disable compile time asserts. However if you do so, the ACE may not be able to boot if the values of the memory are out of bounds.

CSCtk07547—When you configure the ACE with connection reuse, if the server sends the last data packet and the client acknowledges (ACK) this packet with a new GET request, the ACE opens a new backend connection for the GET and does not forward the ACK to the server. The server retransmit the data packet and the client sends a single ACK. For a single client connection (not a proxy), there are seven backend connections and, for the end user, there is the impression of slowness due to the time lost opening the new connections. Workaround: Remove connection reuse or configure the wan rtt timeout 0 command.

CSCtk60666—When using the following sticky layer4-payload configuration for an SSL session ID, sticky works; however, the show sticky database layer4-payload session_ID command does not return a value even though there is an entry in the sticky database:

sticky layer4-payload SESSID-STICKY
  serverfarm SF1
  response sticky
  layer4-payload offset 43 length 32 begin-pattern 
"(\x20|\x00\xST)"
 
   

Workaround: None.

CSCtk61448—The device may be affected by an OpenSSL vulnerabilities described in CVE-2010-4180 and CVE-2010-4252.

Conditions: Device configured with any feature that uses SSL.

Workaround: Not available

PSIRT Evaluation: The Cisco PSIRT has assigned this bug the following CVSS version 2 score. The Base and Temporal CVSS scores as of the time of evaluation are 5.1/3.8:

https://intellishield.cisco.com/security/alertmanager/cvssCalculator.do?dispatch=1&version=2&vector=AV:N/AC:L/Au:N/C:C/I:C/A:C/E:H/RL:U/RC:C

CVE IDs CVE-2010-4180 and CVE-2010-4252 have been assigned to document this issue.

Additional information on Cisco's security vulnerability policy can be found at the following URL:

http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html

CSCtk64756—When you configure the retcode threshold in which each IXP receives a threshold of 1 and the IXP receives one error message in each reset interval, the real server may not transition to the RETCODE-FAILED state or transitions to the RETCODE-FAILED state with a delay. Workaround: None.

CSCtk64794—When ACE load-balanced traffic is flowing to multiple contexts simultaneously and you configure the return code under the server farm, some real servers may remain in the RETCODE-FAILED state. Workaround: To return the real server to OPERATIONAL state on all NPs, enter the no inservice command followed by the inservice command.

CSCtk65797—When you configure a server farm with the remove option on the retcode command, if you add and then delete a failing probe to the same farm, some of the real servers in it remain in the Disabled state on the dataplane and may cause the servers not to service any connections. Workaround: None.

CSCtk65817—When you configure a server farm with the remove option on the retcode command and it is in the ACTIVE state, the show serverfarm command may occasionally display the real servers in the OPERATIONAL state even though they are in the RETCODE-FAILED state on the dataplane. Workaround: None.

CSCtk84174—When the backup real server takes over from a server in a server farm after a probe failure, it uses the default weight instead of the configured weight for the standby real server. The issue occurs in the LB fabric code due to the fix of CSCtl09890, "Conns are not getting lb properly when backup rserver is configured." This fix avoids the calculating of the WeightFactor and does not allow the downloading the weight to the dataplane. Workaround: None.

CSCtk84177—When mixed RADIUS load-balancing and HTTP sticky traffic are running on the ACE, and outbound sticky leaks are high, the ACE tries to fetch the expired sticky entries to free the LRU entries in order to perform a new sticky insertion. Since the leak is around 30k, it becomes stuck in a loop for a long time and causes the ACE to reboot. Workaround: None.

CSCtk84181—When you configure the sticky radius framed-ip command with the username or calling-station-id option as part of framed-IP sticky, the ACE returns the sticky type as username or calling-station sticky. However, it does not create the framed-IP sticky. Workaround: None.

CSCtk84186—When you roll back a checkpoint from a round-robin predictor configuration to a leastconn predictor configuration with the conn-limit command, and RADIUS traffic is flowing through the ACE, the ACE reboots. Workaround: Avoid a predictor change when traffic is flowing through the ACE.

CSCtk84187— When you configure maximum connections with the conn-limit command and the ACE has no real server available for RADIUS end-user traffic configured with framed-IP sticky, the active ACE makes a FORWARD decision and sends a packet to the actual destination IP part of the HTTP packet from the client. When the standby ACE receives these connections, its Total Misc Errors counters increment because there are no valid real server ID or load-balanced real servers associated with this connection. Workaround: None.

CSCtk99627—When SSL causes a buffer leak on the ACE, it may cause an outage situation. Workaround: Reboot the ACE.

CSCtl02573—When you add and then delete the shared primary VLAN, traffic from the client or server in a secondary VLAN that needs to be routed or bridged through the ACE fails. Workaround: Reboot the ACE.

CSCtl09890—When you configure real servers with weights and the leastconns predictor, the ACE load balances the connections unequally. Some of the real servers do not receive any connections even though the weights are the same as the other servers that receive connections. Workaround: None.

CSCtl22458—When you change an ACL configuration for an object group, the following error messages occurs:

%ACE-1-106028: WARNING: ACL Merge failed to add ACE in context ContextName. Error 
while processing access-group. Incomplete rule is currently applied on interface 
vlan#.  Configuration on this interface needs to be manually reverted
 
   

Workaround: Avoid using object-group ACL configurations or reboot the ACE with a new ACL configuration that you applied and saved.

CSCtl41641—In software version A2(3.2) or A2(3.3) with three or more contexts, the show resource usage top number resource sticky command displays incorrect information or no results, and may log the following internal error:

Internal error during command execution
 
   

Workaround: Enter the total number of contexts including the Admin context for the top number argument. The command displays all contexts.

CSCtl48549—When you configure the ACE for SIP inspection and the server sends a SIP message that contains the content body and the last header without the content-length information, SIP calls fail to complete through the ACE. The load balance succeeds and the call is initiated with the backend server. However, the call does not fully complete and the server eventually terminates the call. Also, in this condition, the ACE may not forward the SIP message to the client in a timely manner, which causes the server to time out and close the connection. Workaround: None.

CSCtl57935—When you configure long names for a probe, server farm, and real server with a total length of more than 128 bytes, the ACE has difficulty parsing the ciscoSlbHealthMonMIBObjects MIB object. The SNMP query times out or does not contain all of the probe information. Workaround: Reduce the length of the probe, server farm, and real server names.

CSCtl63354—When you configure the following RDP load-balancing policy, the ACE fails to load balance RDP (Microsoft terminal service) connections:

policy-map type loadbalance rdp first-match rdp2
  class class-default
    serverfarm rdp 
 
   

Workaround: Use a Layer 4 load-balancing policy and configure a source IP sticky, similar to the following:

sticky ip-netmask 255.255.255.255 address source rdp-sticky
  replicate sticky
  serverfarm rdp
 
   
policy-map type loadbalance first-match rdp2
  class class-default
    sticky-serverfarm rdp-sticky
 
   

CSCtl71449—When an object group for a service is configured in a security ACL and a VIP is configured that fits within the network of the object group and also ends in a (multiple of 8) .7 and is the only VIP in that address range, the wrong virtual server may be hit when traffic is sent to that VIP. For example, the VIP ends in .7 and there are no other VIPs ending in the .1 to .6 range. Workaround: Add another VIP with an IP address that ends in a value which is within six numbers lower of any VIP that ends in a (multiple of 8) .7 and that has no other VIPs in that byte range. For example: If the VIP ends in .7 and has no other VIPs in the .1 to .6 range, then add a VIP in that range. If the VIP ends in .15, then add a VIP that ends in the .8 to .14 range, and so on.

CSCtl74617— When you configure a service on the ACE with HTTP inspection and a file download that contains video or mp4 content occurs through the service (VIP), the video quality is poor. Workaround: Remove the HTTP inspection policy from the Layer 3/Layer 4 server load-balancing policy.

CSCtl74730—The ACE reboots at LbUtil_StoppedServerListAdd and generates a qnx_1_loadBalance_g_ns_core_log.tar.gz file. Workaround: None.

CSCtl76856—With persistence rebalance, after receiving a HTTP HEAD request, ACE stops forwarding subsequent requests. Workaround: Disable persistence rebalance.

CSCtl80111—When you dynamically configure an ACL on an active ACE in a redundant configuration, it reboots unexpectedly and generates a Configuration Manager (CFGMGR) core file. When the commands are synchronized to the standby ACE, it reboots also causing an outage on the network. Workaround: Do the following:

Before making any changes, remove the service-policy input policy_name command from the interfaces. Make the changes and reapply it.

When making the dynamic changes, do not use the ACL line number.

The ACE does not reboot when removing a class map from the policy. Make the changes and reapply it.

CSCtl89041—If you configure multiple return codes under a server farm and the responses from the server are delayed, the real server may remain in the RETCODE-FAILED state longer than the configured resume seconds interval. Workaround: None.

CSCtl91344—When the ACE is performing end-to-end SSL or SSL initiation, the SSL connections between the ACE and the SSL server fail to negotiate. The SSL server resets the TCP connection immediately after it receives the SSL Client Hello packet. This problem occurs because the ACE sends an SSL Client Hello containing gmt_unix_time value that is incorrectly set to the SSL server. Most servers ignore this value, but some do require it to be correct. This issue does not affect SSL termination on the ACE. Workaround: None.

CSCtl94488—When you configure a server farm with a scripted probe for health monitoring and scripted probes fail, the ACE does not generate level 3 health probe failed error messages. If you configure SNMP traps, the SNMP device logs the probe failures but the ACE does not generate them in the system log. The expected level 3 message is similar to the following:

%ACE-3-251015 Scripted probe failed for server ip_address, error message. 
 
   

Workaround: None.

CSCtl95239—Due to the fix for CSCtk84177 allowing the access of NULL pointers, the ACE module reboots. Workaround: None.

CSCtl98081—When HTTP traffic greater than 1K cps is flowing through the ACE and you enable TCP server connection reuse, the ACE reboots. Workaround: Disable server connection reuse by using the no server-conn reuse command.

CSCtn05967—The ACE requires SUP2T support and new firmware. Workaround: None.

CSCtn08220— When you remove a peer probe, an FT track probe may become invalid. The following sequence of commands causes the probe for ft host track1 to become invalid:

ft track host track1
  track-host 10.10.171.14
  probe probe1 priority 200
  peer track-host 10.10.171.14
  peer probe probe1 priority 50
ft track host track2
  track-host 10.10.170.14
  probe probe1 priority 200
  peer track-host 10.10.170.14
  peer probe probe1 priority 50
 
   
ft track host track1
 no probe probe1 priority 200
 probe probe1 priority 200
 
   
ft track host track2
 no peer probe probe1 priority 50
 
   

Workaround: Remove and reconfigure the FT track object for the corresponding invalid probe.

CSCtn13034—When you use a script to remove a probe, add it to a server farm, and enter the no inservice and inservice commands for the real server under the same server farm, the probes are in an INVALID state and do start. This issue is a timing issue. When you apply the same configuration one command at a time, this issue does not happen. Workaround: Reboot the ACE.

CSCtn21787—In a redundant ACE configuration in which traffic is flowing, when you perform a configuration rollback from a configuration with the round-robin predictor to a configuration with the leastconn predictor and sticky, the ACE reboots. Workaround: Do not roll back the configuration when traffic is flowing on the ACE.

CSCtn27844—When you configure a Layer 4 payload sticky group with a static entry that has the prefix value \x and delete the entry from the configuration, the ACE does not remove it. The show sticky database static command still displays the entry, while the show running-config command does not. Workaround: None.

CSCtn49076—Occasionally, when the FT TCP channel may need to set up multiple times because it keeps getting torn down, its state eventually becomes TL_SETUP/FT_VLAN_DOWN or TL_ERROR/FT_VLAN_DOWN. Intermittent network outages or other conditions that create the need to set up the FT channel several times back-to-back may cause this issue. Workaround: Manually disable and enable the FT VLAN.

CSCto04141— When you configure a catch-all VIP on the ACE and a DoS attack using HTTP occurs, buffer utilization becomes very high causing the ACE to not serve new connections. Workaround: Instead of a catch-all VIP, restrict the configuration to a particular VIP or monitor the buffers and clear out these connections. You can also attempt to block these client IP addresses.

Software Version A2(3.4) Open Caveats

The following open caveats apply to software version A2(3.4):

CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, and then try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in an RSA format instead of an RSA1 format. For example, enter the following command:

host1/Admin# ssh key rsa 1024 force
 
   

CSCsv80430—When you configure RBAC on an ACE with a custom role and domain, any permit rule allows all show commands to be entered regardless of the configured permissions. Workaround: None.

CSCsx13061—When you perform a checkpoint rollback in a specific order or execute a match and no match statement under a class map, ACL memory is leaked and some entries configured in the ACL are not removed from the interface. Workaround: Remove the interface and readd it or do not perform a rollback in the specific order mentioned in the steps to reproduce of the bug description.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx55228—When you remove an entry with an object group from an ACL which is associated as global access group and then readd it, merge errors occur and disallowed traffic goes through the ACE. Workaround: Unconfigure and then reconfigure the access group.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: Configure multiple NAT IP addresses for PAT instead of limiting it to a single IP address.

CSCsy23268—The ACE may send probe traffic with the source IP address of the alias IP address instead of the local interface IP address. This issue occurs on the active ACE only. Workaround: None.

CSCsz19782—When you convert the configuration from a non-full proxy to a full proxy configuration for full proxied new connections and you add new VIPs for load balancing, traffic to these VIPs do not go through the ACE. Workaround: Reboot the ACE.

CSCsz22742—When you copy a large configuration to the running-configuration file, an API timeout error may occur. Workaround: None.

CSCsz38667—When you remove the key and certificate from the SSL-proxy service, you cannot delete it from the policy map when it is invalid. Workaround: Add a dummy key and certificate to the previously configured SSL-proxy service.

CSCsz85367—When you configure and unconfigure access lists in a loop, the ACE experiences a memory leak. Workaround: Do not configure and unconfigure access lists in a loop.

CSCsz92540—If the configuration contains inline match statements under a policy map, the check point rollback fails. For example:

policy-map type inspect http all-match http-match

match test strict-http

reset

Workaround: Remove all the inline match statements before doing the checkpoint rollback.

CSCta13446—When you remove and then reapply the inspect ftp command, the ACE drops connections. Workaround: None.

CSCta39372—When you perform repetitive checkpoint rollbacks, the ACE becomes unresponsive after five to six hours. Workaround: None.

CSCta73571—When you configure ft track for an interface that is constantly down and then attempt a checkpoint rollback from a large configuration to an empty configuration, the rollback ends prematurely, resulting in a partial rollback. The ACE, however, indicates that the rollback is complete. Workaround: Attempt the rollback once again. If it fails again, configure ft track with a greater difference between the active and standby priority settings.

CSCta92891—If you change the load-balance predictor from least conns to hash URL with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCtb30178—If you configure a RADIUS client Layer 7 policy map and continuously send accounting On/Off packets for 12 hours, the system fails. Workaround: None.

CSCtb55845—When a Virtual Switching System is configured on two Catalyst 6500 series switches, active-active redundancy is configured on the two ACEs in separate chassis, and you run stateless UDP traffic through the ACEs, some connections may fail. A trace shows that the successful flows use the ACE virtual MAC as the destination and the unsuccessful flows use the physical interface MAC of the standby ACE. A display of the default route and the svclc RHI routes shows two entries for the VIP in question. If you enter the show ip route command, the preferred route is the standby interface instead of the alias IP address. Workaround: None.

CSCtb72635—When you run a script for the show tech detail command on an ACE that has 4000 BVI and 4000 VLAN interfaces configured, the ACE may become unresponsive. Workaround: None.

CSCtd94085—You may observe an MTS memory leak for an invalid or a nonexistent process or PID. For a Vshell process, the MTS message queue limit is limited to a maximum of 4096 messages. Beyond that limit, any new message (for example, a changeto command is being executed), will get dropped and the following warning message is displayed on the console:

Warning:- MTS queue is full for opcode "<opcode value>" sap "<sad_id>" pid "<pid>" 
clear idle debug plugin sessions or telnet/ssh connections to recover. 
 
   

Sometimes, the PID that is displayed here may be invalid (no real process associated with it). Workaround: None.

CSCte26173—During periodic XML queries on ACE for show commands, such as the show ft group status command, the ACE places the bash core files in the core: directory. Some files are unpackaged and other files are mispackaged as VSH core files. Workaround: None.

CSCtf33100—If two or more probes associated with the server farm are in the failed state, at least one probe is in the passed state and the fail-on-all configuration is removed, the real server remains in the OPERATIONAL state and is not moved to the PROBE-FAILED state. Workaround: None.

CSCtf39655—If you configure the send-data option inside a finger probe with length greater than four characters, the probe fails. Workaround: Configure a send-data length with less than four characters.

CSCtf44818—The ACE module occasionally displays the incorrect value in the Unicast bytes input counter for the interface. This issue can cause problems for SNMP tracking the traffic, which in turn displays ~50Gbps flowing through the ACE. Workaround: Configure the SNMP application to ignore the counter increases above a certain value.

CSCtg70913—When users whose accounts have expired attempt to log in to the ACE through SSH or Telnet, they succeed. Workaround: None.

CSCtf86359—When you add a certificate to an existing chain group on the active ACE and the certificate does not exist on the standby ACE, the standby ACE should change to the FT COLD state but it does not. Workaround: To correct the state, perform one of the following.

On the FT group, enter the no inservice command followed by the inservice command.

In configuration mode, enter the no ft auto-sync running-config command followed by the ft auto-sync running-config command.

CSCtg93332—When you configure the mac-address autogenerate command on the client VIP interface in bridge mode, traffic to VIP starts failing. Workaround: Delete the client side interface and readd it.

CSCtg94333—When you create 10 contexts and allocate 10 percent of the resources to each context, ACL merge-list creation fails and management traffic to the VLANs fail. When you enable the debug access-list merge errors command and add or delete VLANs, the ACE displays merge-list errors. Also, attempts to ping and Telnet for the management access fail. Workaround: Remove the resource allocation from the contexts.

CSCth30569—When you apply a large multi-context configuration, the arp_mgr service in ACE becomes unresponsive. Workaround: None.

CSCth75674—When the SCP HW watchdog on the ACE fails to detect the timer expiry, the Catalyst 6500 supervisor power cycles the ACE with an SCP keepalive failure message. The watchdog may fail to detect the timer expiry when the internal counters overflow. The SCP HW watchdog mechanism detects when the ACE becomes unresponsive and collects the core files in error case scenarios which prevents the power cycling by the Catalyst 6500 supervisor. On rare occasions, the watchdog may fail to detect the timer expiry. Workaround: None.

CSCth85288—When a Layer-2 connected real server is in the ARP_FAILED state and you shutdown or delete the corresponding Layer-3 interface, the real server state does not transition from the ARP_FAILED state to OPERATIONAL or the probe failed state. Workaround: Remove the real server and reconfigure it.

CSCth99982—When you configure an ECHO TCP probe with a send-data value and the server returns a different value, the ACE does not match what was sent and passes the probe by default. Workaround: Use a TCP probe with send-data and regex values as required instead of an ECHO TCP probe.

When you configure send data on a probe and the server returns something other than send data, we do not match that with what was sent and pass the probe by default. In an echo probe the send data must be the same as what is echoed back from the server.

CSCti02008—When an ACE running configuration contains 64K ACLs and you roll it back to an empty checkpoint, the checkpoint rollback occurs in approximately 45 to 50 minutes. Workaround: None.

CSCti18687—When you perform various levels of end-to-end or back-end SSL traffic performance tests on the ACE, SSL-initiation and HTTPS probe connections remain in the CLSRST state until the inactivity timeout clears them. Workaround: Decrease the inactivity timeout or manually close the connections through the CLI commands.

CSCti26266—When the To CP traffic flows without any issues, the show cde health command displays the BRCM pull status as not pulling. The CDE FPGA asserts flow control to a different channel based on the buffer threshold. When the condition clears, the show command output does not update the pulling status. This is a display issue with no functionality impact. Workaround: None.

CSCtj03057—When the ACE has DNS traffic that has greater than 40K queries per second and is not configured with DNS inspection, the ACE may reboot. Workaround: None.

CSCtj20310—When you enable XML, the XML output is not seen for the show serverfarm name retcode details command. Workaround: None.

CSCtj21592—When you configure two or more probes to a server farm, the probe instance is not created after removing one of the probes on the server farm and entering the inservice command on the real server. Workaround: None.

CSCtj67085—When the ACE has Layer 7 traffic greater than 40K CPS and you enter a show command that displays more than 3 to 5 lines of output, the Telnet management connections to the CP may become unresponsive. Workaround: None.

CSCtj70903—When you configure an engine ID on the ACE and then configure users, if you change the engine ID to another value, the ACE deletes the users. However, if you reconfigure the original engine ID value, the ACE restores the deleted users. The ACE should permanently delete the users after you change the engine ID. Workaround: None.

CSCtj72215—When you configure SNMP local and peer engine IDs on the active ACE and you enter the show startup config|include snmp command on the standby ACE, the standby ACE does not display the exchanged value for the engine IDs (local ID as the peer, peer ID as the local). Both active and standby ACEs display the same output. Workaround: None.

CSCtj79482—When you configure a real server on the ACE, assign it an IP address, place it in service, and then delete it, the ACE generates an unnecessary trap. When the real server state changes from ARP-FAILED to OPERATIONAL, the ACE generates the CesRServerStateUp trap. Workaround: None.

CSCtj91891—When a configured TCP probe becomes active on the ACE and the server sends out-of-band data to the ACE, the ACE reboots and generates an hm_core file. Workaround: None.

CSCtl10565—If you configure Accept and Accept-Encoding headers by using the header command under an HTTP probe, the Accept header is not seen in the probe request on the server side. Workaround: Remove and readd the Accept header value in the HTTP probe configuration.

CSCtl75995—When the retcode resume seconds interval is in the default state of being an infinite interval and a real server transitions to the RETCODE-FAILED state, if you configure the resume interval with a finite value, the real server state may not recover from the RETCODE-FAILED to the OPERATIONAL state. Workaround: None.

CSCtl93050—In a redundant ACE configuration with SNMP polling in place, the ACE reboots in the snmp daemon. The network management server collects the following informations using SNMP from both ACE modules:

Once in four hours, sysName and sysDescr. These values were not read before the time when the ACE module spontaneously reboots.

Every minute, sysUptime and ciscoCpuUtil (1.3.6.1.2.1.1.3.0 and 1.3.6.1.4.1.9.9.109.1.1.1.1.7.1). Before the ACE spontaneously reboots, these variables could be read (sysUptime or ciscoCpuUtil).

Workaround: None.

CSCtl97127—Each time that the standby ACE reboots, a context on it transitions to the STANDBY_COLD state and the ACE displays the following error:

Error on Standby device when applying configuration file
 
   

It is a timing issue due to the configuration size and total number of contexts. This issue can lead to a lot of Configuration Manager (CFGMGR) download processing which can lead to a command failure. CSCtn50357 is tracking the issue of the actual failing command that is not properly placed in the error logs. Workaround: Perform either of the following:

On the FT group for the context in the STANDBY_COLD state, enter the no inservice command followed by the inservice command.

Change the context FT group ID in the FT group to a higher number so that the context with the largest configuration does the configuration synchronization last.

CSCtn01263—When you configure the tcp-options timestamp command on the ACE, the ACE use the wrong timestamp when it sends an ACK packet. With front-end encryption, this problem is evident on a client-side trace. The ACE acknowledges with a packet containing data, but uses the time stamp from a previous packet. Workaround: None.

CSCtn05910—When you configure a real server for a policy with the server connection reuse and persistence rebalance enabled, the ACE may not correctly update the connection count for the server. Workaround: None.

CSCtn14301—When a control store parity error occurs, the reason is not set properly. The me_dumper process needs to be fixed to identify the control store parity error. Workaround: None.

CSCtn18443—In Layer 7 mode, after a client transmits a packet, the ACE occasionally requests a retransmission of subsequent packets even though it has previously received the packets. Workaround: Enabling selective ACK (SACK) appears to help.

CSCtn25281—If the ACE sticky Layer 4 payload configuration does not include the layer4-payload offset number1 length number2 begin-pattern "expression" command and you enter the no response sticky command, the following error occurs:

Error: invalid input parameter
 
   

Workaround: None.

CSCtn26839—On a rare occasion when you remove a VIP from the configuration, the ACE continues to have an ARP entry for the VIP and it causes a traffic routing problem. Workaround: Reboot the ACE.

CSCtn47103—During a time when the ACE was not sending probes, a probe failure occurs because of a server open timeout (no SYN ACK, as confirmed through tcpdump on the server). Workaround: Reboot the ACE.

CSCtn52695—If you configure inconsistent netmasks similar to the following on the ACE, the CLI should prevent or notify you of the inconsistency through a syslog message:

access-list acl1 extended deny ip any 10.45.15.192 0.0.0.151
access-list acl1 extended deny ip any 10.45.15.192 0.0.9.0
 
   

Workaround: None.

CSCtn65581—When you configure Layer 7 persistence rebalance and server connection reuse, the ACE advertises an unscaled window size. This issue occurs for the packets from the second request onwards. Workaround: Remove server connection reuse from the configuration.

CSCtn72817—When you upgrade an SSL certificate, the ACE sends an RST for existing connections. The ACE handles the subsequent new session requests for these connections with the new certificate. Workaround: Upgrade the SSL certificate in a maintenance window.

CSCtn84524—When you enter the allocate-interface vlan command for one context in a redundant ACE configuration, it is missing in Admin even though the show ft group detail command displays the configuration in synchronization and the active and standby HOT status. Workaround: Manually readd the command.

CSCtn87820—When you configure the ACE with multiple class maps with the same IP but different ports and with KAL-AP UDP for KAL-AP by VIP to report the VIP-address status, the ACE reports the load of 255 for the VIP address if only one class goes out of service for any reason, such as a server farm failure due to a probe failure or real servers are taken out of service. Workaround: Use KALAP by tag instead of KAL-AP by VIP.

CSCtn89970—When you enter the show resource usage command, it displays the output for total bandwidth allocation that is unclear. This is a display issue. Workaround: None.

CSCtn90010—When SNMP polling occurs to the local IP address of the ACE module and the ACE receives a malformed SNMP packet or heavy utilization of SNMP polls, its SNMP daemon may delay in responding by 10 to 15 minutes. Workaround: None.

CSCtn93329—When redundant ACEs generate SIP probes with the same Call-ID and From-Tag options, the SIP registrar servers interpret these probe messages as duplicates and do not reply to them causing SIP health probes to fail. Workaround: None.

CSCtn96791—When you enable FTP inspection on the ACE, the server responds to the client FIN but does not include its own FIN. The client sees the first FTP transaction succeed, but subsequent transactions fail. The connection is left half open between the ACE and the server. This issue causes problems for future client transactions. Workaround: Set the connection timeout for half-closed TCP connections to a low value. For example:

parameter-map type connection ftpftp
  set tcp timeout half-closed 1
 
   
policy-map multi-match VIP
  class VIP1
    loadbalance policy VIP1
    loadbalance vip icmp-reply
    connection advanced-options ftpftp
 
   

CSCto00168—When you enter the show resource usage command, the sticky field displays 4294965431. Workaround: Reboot the ACE.

CSCto00198—When sticky is in use on the ACE, it stops working and the ACE displays the following log message:

%ACE-LB_STICKY-3-728007: Internal communications error (sticky) -- type 4
 
   

Workaround: Fail over or reboot the ACE.

CSCto03171—When an SSL certificate or key is in use on the ACE, you can delete it. Workaround: Before removing the certificate or key, manually verify whether it is being referenced in the configuration.

CSCto04222—When you configure RADIUS traffic and maxconn limit on the real servers, the ACE does not correctly update some connection counters for the servers. The server farm and real servers remain in the MAXCONN state when there is no traffic. Workaround: None.

CSCto05999—When you use a previously used IP address for a MAC address in an ARP entry for a real server, the ACE displays an ARP entry without a next scheduled ARP time. The entry does not time out and you cannot clear it by using the clear arp ip command. Workaround: Use a new IP address for the real server.

CSCto08690—When you configure 16K SMTP probes with 250 context, some probes may be errored out with the following error:

Last disconnect err:"Internal error: Out of sockets".
 
   

Workaround: None.

CSCto11694—With certain types of content, the ACE changes the UDP checksum, which causes the request to drop on the server. Workaround: None.

CSCto64389—You cannot disable the driver of the SiByte hardware watchdog from the ACE CLI. You can only disable it in the debug plugin. Workaround: To disable the driver, run echo 1 > /proc/watchdog. To enable the driver, run echo 0 > /proc/watchdog. To display the status, run cat /proc/watchdog; dmesg | tail.

Software Version A2(3.4) Command Changes

Table 8 lists the command changes in software version A2(3.4).

Table 8 CLI Command Changes in Version A2(3.4)  

Mode
Command and Syntax
Description

Exec

show np number buffer usage

Per CSCtj84786 and CSCtl57463, this new command allows you to display the buffer usage of each NP. For more information, see the "Monitoring and Displaying the Network Processor Buffer Usage" section.

Exec

show np number me-stats "-k lbabrt | lboff | lbon"

Per CSCtd92176, the show np number me-stats command includes the following load-balancing options:

lbabrt—Aborts the LB process

lboff—(Default) Disables the LB queue check

lbon—Enables the LB queue check

Exec

system [no] watchdog hardware

Per CSCsy91540 and CSCtn77149, this new command in Exec mode allows you to enable the Sibyte hardware watchdog. By default, the hardware watchdog is enabled.


Caution In some situations, this command causes the ACE module to become unresponsive and does not restart the ACE except in cases when SCP\LCP or some other emergency systems can handle the problem.

When Sibyte hardware watchdog is enabled, it restarts the ACE when the following occurs:

The Linux kernel becomes unresponsive and cannot receive any IOCTL messages from uspace.

The CP uspace becomes unresponsive and the ACE is unable to fork new processes.

For example, to enable the hardware watchdog, enter the following command:

host/Admin# system watchdog hardware
 
        

To disable the hardware watchdog, enter the following command:

host/Admin# system no watchdog hardware
 
        

Exec

show parameter-map name

Per CSCtj65495 and CSCtl94225, the cookie-error-ignore field is replaced by the parsing non-strict field for the new parsing non-strict command in parameter map HTTP configuration mode.

Per CSCtl74617, the inspect non-persistence field was added for the new inspect non-persistence command in parameter map HTTP configuration mode.

Exec

show running-config [type [id]]

Per CSCtj11142, the show running-config command has a new id option to filter the running-config file based on the ID. For example:

show run rserver rs1
show run serverfarm sf1

Configuration

buffer threshold active number1% standby number2% action reload

no buffer threshold

Per CSCtj84786 and CSCtl57463, this new command allows you to enable the reboot of a standalone ACE when the buffer usage reaches or exceeds its threshold on any NP. In redundant mode, the reboot of the active ACE occurs only when it reaches or exceeds its threshold on any NP and the standby ACE is below its threshold. For more information, see the "Monitoring and Displaying the Network Processor Buffer Usage" section.

FT track host

probe name priority number

Per CSCtg07971, when you configure an FT track probe without configuring an FT track host, the probe transitions to the DISABLED state as displayed by the show probe command. Previously, the probe transitions to the INVALID state.

Parameter map HTTP

[no] cookie-error-ignore

Per CSCtj65495 and CSCtl94225, the cookie-error-ignore command is deprecated and replaced by the parsing non-strict command. For more information, see the "Skipping a Malformed Cookie in an HTTP Flow" section.

Parameter map HTTP

[no] inspect non-persistence

Per CSCtl74617, this new command allows you to configure the ACE to bypass connection persistence inspection during HTTP transactions for use with smooth streaming deployments. For more information, see the "Bypassing Inspection during HTTP Transactions" section.

Parameter map HTTP

[no] parsing non-strict

Per CSCtj65495 and CSCtl94225, the parsing non-strict command replaces the cookie-error-ignore command. For more information, see the "Skipping a Malformed Cookie in an HTTP Flow" section.

Probe HTTP or HTTPS

[no] append-port-hosttag

Per CSCti76675, the new append-port-hosttag command allows you to configure the ACE to append the port information in the HTTP Host header when a nondefault port is used for an HTTP or HTTPS probe. By default, the ACE does not append the nondefault port. For more information, see the "Appending Nondefault Port Information in the HTTP Host Header" section.

Serverfarm host

[no] maxconn-one-np

Per CSCtj65014, the new maxconn-one-np command allows the show commands to report that the real server is in the MAXCONN state when a single NP reports the real server reaches its limit. Thus, the global state of the real server can be MAXCONN before the configured limit is reached. If the MAXCONN limit on the other NP is not reached for this server, the server can still accept new connections, but never more than the global MAXCONN limit.

The advantage of this new behavior is that if you configure a backup server farm, the backup activates as soon as one NP cannot handle new connections preventing drops.

For more information. see the "Reporting a Real Server MAXCONN State When One NP Reaches its Allocated Limit" section.


Software Version A2(3.4) System Log Messages

Software version A2(3.4) includes following new system log (syslog) messages.

251015

Error Message    %ACE-3-251015: Scripted probe failed for server A.B.C.D, error message.

Explanation    Per CSCtl94488, the ACE generates this syslog message for scripted probe failures. The possible values of the error message variable are as follows:

Probe error: Server did not respond as expected

Internal error: Fork failed for TCL script

Internal error: Script probe terminated due to timeout

Internal error: TCL interpreter PANIC

Internal error: Script error

Internal error: Script-file lookup failed or empty buffer

Internal error: Failed to allocate memory for tcl workerthread qnode

Internal error: Unknown script error

Internal error: Out of sockets for the TCL script

Internal error: Unable to read persistent variable table

Internal error: PData (probe data) pointer is null

443002 through 443005

Per CSCtj83501, the ACE generates the system log messages (syslogs) from 443002 through 443005 when the IXP network processor buffer usage crosses 50 percent, 75 percent, 88 percent, 95 percent, and 100 percent. For more information, see the "Related Syslogs for Buffer Usage" section.

Software Version A2(3.3) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

This release note includes the resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.3), and changes to commands and system log messages:

Software Version A2(3.3) Resolved Caveats

Software Version A2(3.3) Open Caveats

Command Changes in Software Version A2(3.3)

System Log Messages

Software Version A2(3.3) Resolved Caveats

The following resolved caveats apply to software version A2(3.3):

CSCsv82779—The ACE treats the deny function inside a management policy or class map as a SKIP. The ACE does not deny the traffic. Instead, it skips the class map and tries to match another one. Workaround: None.

CSCsx14845—When you configure a Session Initiation Protocol (SIP) probe for health monitoring (HM), the ACE may incorrectly display the probe as down due to the ACE using the same Call ID for multiple probe instances to different configured real servers. Workaround: Configure the ACE with a different probe type.

CSCsx41539—The ACE module may reboot and generate the following core files:

last boot reason: NP 0 Failed : NP Process Crashed
	182284 Feb 1 15:53:45 2009 qnx_1_mecore_log.999.tar.gz
	687601 Feb 1 15:53:41 2009 qnx_1_io-net_core_log.114693.tar.gz
	113726 Feb 1 15:53:47 2009 ixp1_crash.txt
 
   

Workaround: None.

CSCsz37412—When the software and license on the ACE are compatible, ANM does not display their compatibility status. The XML show ft peer 1 detail command on the ACE is not correct. Workaround: None.

CSCta40969—When you configure end-to-end SSL on the ACE and enable session ID reuse, the ACE reboots. Workaround: Disable session ID reuse.

CSCtb08821—When you enter an SSL crypto show command, the ACE generates a VSH core file. The VSH core file does not cause the ACE to reboot. Workaround: None.

CSCtb21313—When you configure persistence rebalance in a configuration with two server farms containing the same real server with different port numbers and attached to two different Layer 7 policy maps, the ACE drops connections intermittently after a rebalance occurs to a different Layer 7 policy. Workaround: None.

CSCtd27259—When the ACE is running cron with the logrotate to rotate the Apache and wtmp logs, the ACE reboots with /proc/meminfo corruption in the SNMP, SYSMGR, HM and SCRIPTED HM core files. Workaround: Disable cron, and the Apache and wtmp logs.

CSCtd74175—When you configure client authentication on the ACE, the ACE may not be able to converge to the correct cache entry and the statistic for client authentication cache hits does not increment. Workaround: None.

CSCtd80111—When the ACE is running RADIUS load balancing at high rate of more than 500 requests per second with a single client connection, a buffer leak occurs on the dataplane (DP) Workaround: None.

CSCte03073—When you configure HTTPS probes on the ACE for an IIS server that is configured with the Accept client certificates option, the probes fail. Workaround: None.

CSCte81257—When you perform dynamic configurations of usernames in multiple contexts and enter the no username name command in a user context, the ACE module unexpectedly reboots and generates an SNMP core file. Workaround: None.

CSCtf22462—When you configure a probe on a host real server and the probe state changes from FAILED to SUCCESS, the ACE generates a cesRserverStateUp trap. However, the ACE should generate a cesRserverStateChange trap. Workaround: None.

CSCtg18442—When the ACE is running a software version earlier than software release A2(3.3) and configured for SSL termination, if an incoming SSL client sends a encrypted stream of traffic during the SSL handshake phase and the ACE decrypts it, the ACE may reboot. The reboot occurs in the ACE SSL Nitrox chip that performs the decryption. Workaround: None.

CSCtg22592—After you make a change to a large ACE configuration and enter show commands, the CLI becomes unresponsive for a period of time. In this case, the show processes cpu | include cfgmgr command displays one of the configuration manager (cfgmgr) processes consuming CPU resources. After you apply the configuration change, the cfgmgr CPU usage drops to zero, and the CLI becomes unresponsive. Workaround: Wait until the cfgmgr process completes its previous operation before entering the show command.

CSCtg35291—The ACE may unexpectedly reboot and display a crashinfo file only in the dir core: output. This is a kernel crash of the CP. Workaround: None.

CSCtg45108—When the rate of each flow is approximately 600 K packets per second and packets arrive at burst, fastpath queues fill up causing the dropping of packets. Workaround: None.

CSCth34050—Under normal operating conditions with logging enabled on the ACE module, the ACE unexpectedly reboots and generates a syslogd crash file. Workaround: None.

CSCth36358—Under normal operating conditions, the ACE unexpectedly reboots and generates a Linux kernel crashinfo file. Workaround: None.

CSCth41583, CSCth21361—When the ACE receives a cookie string that contains many cookies and encounters a space character in the cookie value, it stops processing the cookies. Spaces are not permitted in the cookie name or cookie value. Persistence or stickiness fail. Workaround: None.

CSCth44555—Configuring udp eq sip causes the standby ACE to enter the STANDBY_COLD state. UDP port 5060 is not a standard port in the ACE. Workaround: Configure the tcp-udp eq 5060 instead.

CSCth54951—On a rare occasion, the hash value displayed by the show acl detail command does not match the hash value in the ACL-merge output. Workaround: None.

CSCth59753—When an SNMP agent polls active and standby ACEs with the same SNMP engine ID, it becomes confused. The SNMP engine ID should be unique for each ACE. Workaround: The existing snmp-server engineid value command now includes the peer engineid option as follows, snmp-server engineid value peer engineid value command. This command allows you to change the value for both engine IDs or enter the old value for both. This ensures that SNMP users are deleted from both engine ID when the ID changes to avoid a mismatch in the configuration. This command also allows you to enter only the engine ID value assigning the same value to both the active and standby ACEs. The peer configuration is not mandatory.

CSCth66757—When you configure many servers with active/active NIC teaming, the ACE arp_mgr service may consume 100% of the CPU due to the ARP flood caused by teaming mode. Workaround: Reduce ARP traffic. Always use active/standby NIC teaming.

CSCth69747—When you manage the ACE with SNMPv3 in CiscoWorks LAN Management Solution (LMS)/CiscoWorks Resource Manager Essentials (RME), the ACE intermittently reports false usmStatsUnknownUserNames (1.3.6.1.6.3.15.1.1.3.0) during LMS/RME inventory collection, and the RME inventory collection may fail occasionally. Workaround: Manage the ACE with SNMPv2 in LMS/RME.

CSCth69782—When you configure a VIP on the ACE, the ARP entry is inconsistent but the connections are working. Workaround: None.

CSCth73392—When you configure the ACE with SSL termination and the CP crash detection process detects that the Nitrox-II chip is unresponsive, the ACE reboots and generates a nitrox_core.tar.gz core file in the core: directory. This occurrence was a one-time event. Workaround: None.

CSCth74249, CSCth75242—When the ACE is using SSL client authentication and is oversubscribed beyond capacity, HTTPS probes continue to fail after traffic has failed over to the standby ACE. The connections become stuck. Workaround: Do not allow the ACE to be oversubscribed. Clear all the connections and allow the connections to continue.

CSCth75707—If the client or server retransmits a packet and the remote end exceeds the acceptable window size, the ACE incorrectly drops the retransmission packet and increments the [Drops] fp TCP window left edge counter. Workaround: Disable normalization or correct the client or server to honor the window sizes.

CSCth77963—When you upgrade ACE to software version A2(2.4), the ACE logs the following message after the reboot message:

%ACE-4-901001 kernel: Cannot find mapfile.
 
   

Workaround: None.

CSCth80972—On a rare occasion, the ACE reboots and generates a crashinfo file. Workaround: None.

CSCti03213—After upgrading the ACE to software version A2(3.1), FT auto-sync may not work and active-to-active scenario may occur causing an outage. The banner command had special characters, like a quote ("). The banner command may be deleted on one context. Workaround: Disable auto-sync, manually reconfigure the banner command on both ACEs context, and reenable auto-sync.

CSCti03626—When you apply the same NAT pool to multiple VLAN interfaces, the show ip route command displays the pool of NAT addresses for only one VLAN interface. Workaround: None.

CSCti13494—When the ACE load balances clients toward the HTTP proxies, the ACE resets proxied SSL connection by performing an RST on the Client Hello. This issue may be associated with HTTP/1.1 in the CONNECT request or response. Workaround: You can configure HTTP/1.0 on the client and server. Do not inspect the HTTP connections.

CSCti13660—When the ACE has high CPU usage in the SME processor with an PCI TX Q full error and attempts to generate QNX core dump, it may become unresponsive. Workaround: None.

CSCti15939—When you configure PAT and heavy traffic flows on the ACE, the show xlate command displays the "Got no reply" message. Workaround: None.

CSCti27209—The sample keys and certificate feature is now available in the Admin context only. If you upgrade from software version A2(3.0) or A2(3.1) to A2(3.2), the sample certificate and key for the existing context remain as they are. However, the sample certificate and key are not be seen in new contexts. If you upgrade from software version A2(2.X) or earlier, sample certificates and keys are present in the Admin context only. You can generate a key named cisco-sample-key but cannot delete it. Workaround: None.

CSCti34239—When a client sends a SYN on an existing connection in a Layer 7 connection, the ACE responds to a TCP SYN with an ACK, and an incorrect ACK sequence number. Workaround: None.

CSCti34245—The ACE does not reset a SYN for an existing Layer 7 connection if the sequence number is within the receive window. Workaround: None.

CSCti37783—When traffic is flowing through the ACE and you change a real server configuration, on a rare occasion the weight bucket may become corrupted and the ACE may reboot. Workaround: None.

CSCti39738— When you apply any changes to a Layer 4 policy map, the ACE resets the Layer 4 connections. Workaround: Remove and attach the service policy under the interface.

CSCti47017— When ANM or a script at bootup time issues the show resource usage all command, the standby ACE console displays command parse errors and the context transitions to the STANDBY_COLD state. Workaround: After an ACE boot has completed, resynchronize the configuration.

CSCti48586—When you attempt to delete a server farm with the no serverfarm host command, the ACE displays the following error message:

Error: serverfarm `serverfarm_name' is in use. Cannot delete! 
 
   

The configuration manager thinks the server farm is still applied to the load-balance policy. Workaround: None.

CSCti59680—When the static host route and ARP entry are for the same host, the ACE reboots due to the itasca_route_mgr. Workaround: None.

CSCti61160, CSCth52830—If the %EARL-SWITCH_BUS_IDLE error occurs in the chassis, the supervisor declares the ACE as MajFail and the LCPFW process stops responding. The show proc command does not display the LCPFW process. The reload command on the ACE does not work. Workaround: None.

CSCti64333—When you configure RHI on the ACE, the routes inserted through RHI are duplicated and inconsistent. Workaround: None.

CSCti64341—When you enter the clear arp command, it also deletes any existing static host route. Workaround: Reconfigure the static ARP through the CLI.

CSCti77476—When you configure multiple real servers in multiple server farms in multiple contexts and traffic is hitting all the contexts, some real servers in the RETCODE_FAILED state recover more slowly than the time configured for the reset timer. Workaround: None.

CSCti77979— When the Layer 7 TCP path is overutilized that causes the Timer Freelist Empty to be hit several times, the ACE reboots because of the Timer Freelist corruption. Workaround: Reduce the work load of the Layer 7 TCP path.

CSCtj00241—When you incorrectly configure a sticky server farm through the CLI or XML, the configuration may fail. Workaround: Reboot the ACE and the configuration succeed.

CSCtj04145—When a manual or automatic reboot occurs and as the ACE goes down, the ACE may forward HSRP multicast traffic for a few seconds. Workaround: Configure an input ACL to deny multicast traffic. For example, apply the following configuration to the ACE VLAN interfaces:

access-list deny_mcast line 10 extended deny ip any 224.0.0.0 240.0.0.0   
access-list deny_mcast line 20 extended permit icmp any any   
access-list deny_mcast line 30 extended permit ip any any 
 
   

CSCtj04650—Under normal operating conditions, the ACE running software version A2(2.4) reboots unexpectedly and generates incomplete core files. Workaround: None.

CSCtj04660—When you configure a match source address in a class map and the source address sends traffic, the ACE may become unresponsive because of an error in the load-balancing function. Workaround: None.

CSCtj11617—When the ACE parses an SNMP query, SNMP causes the ACE to reboot and generate the cfgmgr_log.954.tar.gz core file. Workaround: None.

CSCtj14869—When you configure an interface as a management interface, but do not apply an access list to it, you cannot ping the alias address. Workaround: Configure an access list and access group on the interface.

CSCtj17579—After you perform multiple deletions or additions of access group or service policy commands under an interface in parallel, leaf node and action node leaks occur as displayed by the show np 1 access-list resource command. Workaround: Delete the interface and readd it.

CSCtj22893—When you use the XML interface to configure an action list or SSL URL rewrite on the ACE, the XML response may unexpectedly fail. Workaround: Configure the action-list and ssl url rewrite commands from the CLI.

CSCtj30474—When an incomplete response to an SNMP query for cippfIpFilterConfig, the ACE cannot allocate an MTS buffer. Workaround: None.

CSCtj34012—When you configure an object group with a description field that contains a single-quote character ('), ANM does not poll the object-group configuration from an ACE. Workaround: Remove the single-quote character (') from the description.

CSCtj41469—When you use the show system resources command, the command displays the free memory value decreasing but never increasing. Normally, this value should be approximately 150 to 200 MB after you reboot the ACE and slowly decrease approximately 0.5 to 1 MB per day. Workaround: None.

CSCtj54507—When you configure a large number of probes on the ACE, the kernel CPU usage becomes high and the ACE-4-901001 syslog message overruns the log file or the syslogd server. Workaround: Disable logging.

CSCtj67840—When you configure switch mode on the ACE and failover occurs, idle replicated connections become stuck. Workaround: To clear these connections, change the switch-mode timeout value.

CSCtj87838—When you configure the ACE with access lists, objects group and DHCP, and apply the configuration to an interface, an ACL Merge failure occurs. This issue can cause the configuration to be incomplete and you must manually backed it out. Workaround: When you configure the ip dhcp relay enable command, do not configure a duplicate configuration. If you initially configure the command, the ACE does not display an error. However, when you configure the command again, the ACE displays an error.

Software Version A2(3.3) Open Caveats

The following open caveats apply to software version A2(3.3):

CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, and then try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in an RSA format instead of an RSA1 format. For example, enter the following command:

host1/Admin# ssh key rsa 1024 force
 
   

CSCsr76812—When you configure the ACE with Layer 7 load balancing, TCP connection may be disrupted. Packets arrive at the client in reverse order or packets are forced to be resent. Workaround: None.

CSCsv80430—When you configure RBAC on an ACE with a custom role and domain, any permit rule allows all show commands to be entered regardless of the configured permissions. Workaround: None.

CSCsx13061—When you perform a checkpoint rollback in a specific order or execute a match and no match statement under a class map, ACL memory is leaked and some entries configured in the ACL are not removed from the interface. Workaround: Remove the interface and readd it or do not perform a rollback in the specific order mentioned in the steps to reproduce of the bug description.

CSCsx28587—When the maximum aclmerge instance limit of 8191 is reached and then freed, ACL merge will not occur. Also, after reaching the maximum limit of instances, if you remove the outbound ACL from the interface, the policy action nodes are not released. Workaround: None.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx55228—When you remove an entry with an object group from an ACL which is associated as global access group and then readd it, merge errors occur and nonallowed traffic goes through the ACE. Workaround: Unconfigure and then reconfigure the access group.

CSCsx62330—When SSL is configured in one or more contexts and a large number of certificates and keys (approximately 2000 or more) are configured on the ACE, HTTPS probes may fail if you reload the module. The ACE appears to send the HTTPS probes, but they are not successful. You will not see this problem if you do not reload the module after the configuration. Workaround: If possible, reduce the number of certificates and keys to below 2000, and then reload the ACE.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: None.

CSCsy23268—The ACE may send probe traffic with the source IP address of the alias IP address instead of the local interface IP address. This issue occurs on the active ACE only. Workaround: None.

CSCsy31553—When traffic traverses the ACE module with the same source and destination port and dynamic NAT for that traffic is enabled, the ACE performs an implicit PAT. This behavior interrupts some sessions. This problem does not occur when NAT is not involved. Workaround: If possible, disable dynamic NAT.

CSCsy91540—When the supervisor engine detects that the ACE is not responding to keepalives, the ACE may silently reboot and not generate core dump files. Workaround: None.

CSCsz19782—When you convert the configuration from a non-full proxy to a full proxy configuration for full proxied new connections and you add new VIPs for load balancing, traffic to these VIPs do not go through the ACE. Workaround: Reboot the ACE.

CSCsz22742—When you copy a large configuration to the running-configuration file, an API timeout error may occur. Workaround: None.

CSCsz62556—When you apply connection limits by entering the conn-limit command at the real-server level and connection limits are already applied at the server-farm level, some real servers may become stuck in the stopped list forever and not perform load balancing. Workaround: Reload the ACE.

CSCsz67761—When a network error, such as a network interface going down, occurs during the bulk importing of crypto files, the temporary storage space for imported crypto files is not gracefully released. Some of the temporary files remain in the temporary storage area until the system is reloaded. Bulk import procedures currently do not perceive network failures or inactivity if the transfer of the files has begun. Workaround: None.

CSCsz85367—When you configure and unconfigure access lists in a loop, the ACE experiences a memory leak. Workaround: Do not configure and unconfigure access lists in a loop.

CSCsz92540—If the configuration contains inline match statements under a policy map, the check point rollback fails. For example:

policy-map type inspect http all-match http-match

match test strict-http

reset

Workaround: Remove all the inline match statements before doing the checkpoint rollback.

CSCta13446—When you remove and then reapply the inspect ftp command, the ACE drops connections. Workaround: None.

CSCta39372—When you perform repetitive checkpoint rollbacks, the ACE becomes unresponsive after five to six hours. Workaround: None.

CSCta49917—When Telnet connections, SSH connections, or a debug session are active for a long time on the ACE, they do not close properly as indicated by one of the following:

After the execution of each changeto command, the MTS buffers increase as displayed by the show system internal mts buffers command.

The following error message occurs:

IPC queue full. Clear idle telnet/ssh connections or debug plugin sessions to 
recover err
 
   

Workaround: You can either Telnet to each context to make configuration changes, clear each Telnet or SSH session to the ACE with the clear line vty_name command, or reboot the ACE.

CSCta73571—When you configure ft track for an interface that is constantly down and then attempt a checkpoint rollback from a large configuration to an empty configuration, the rollback ends prematurely, resulting in a partial rollback. The ACE, however, indicates that the rollback is complete. Workaround: Attempt the rollback once again. If it fails again, configure ft track with a greater difference between the active and standby priority settings.

CSCta92891—If you change the load-balance predictor from least conns to hash URL with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCta99792—When you are making configuration changes to an ACE that has 30 contexts with traffic running, the control plane configuration manager process may become unresponsive while it is processing a configuration download or configuration changes. Workaround: None.

CSCtb00726—If the VIP address conflicts with the shared interface address across contexts, the standby ACE goes into the cold state with the show ft config-error command displaying the following error message:

interface vlan number 
Error: Global Policy applied, conflicts with VIP, NAT or Interface IP in shared 
interface!
 
   

Workaround: Do not configure a VIP address with the same address as the shared interface IP address on which the service policy is configured.

CSCtb03138—If you configure SNMP traps on a VLAN that has either the IP address or the peer IP address missing and redundancy is enabled, then the active ACE does not synchronize the SNMP traps to the standby ACE. The show ft group detail command displays the following error:

Error "Incremental Sync Failure: snmp config sync to sby." 
 
   

Workaround: Configure both an IP address and a peer IP address on the interface VLAN that you are using as the trap source.

CSCtb28077—When you add the nat dynamic pool id vlan vlan-id command to a Layer-3 rule (combination of Layer-3 policy map and Layer-3 class map), which already has one dynamic NAT pool configured. For example:

policy-map multi-match pm1
class vip1
nat dynamic 1 vlan 731
 
   

This configuration already contains one dynamic NAT statement. If you add another statement for NAT dynamic, that configuration will not be downloaded. Dynamic NAT configuration is not downloaded to Data Plane and dynamic NAT does not work. Workaround: Remove and add the service policy under the client interface.

CSCtb30178—If you configure a RADIUS client Layer 7 policy map and continuously send accounting On/Off packets for 12 hours, the system fails. Workaround: None.

CSCtb32537—The ip name-server command is seen in the standby mode even after removing it in active mode. This issue happens in redundant configuration. Workaround: None.

CSCtb55526—With HTTP and SMTP traffic flowing and approximately 140,000 concurrent connections, the ACE module may exhibit CP slowness and eventually reboot with no core dump files. Workaround: None.

CSCtb55845—When a Virtual Switching System is configured on two Catalyst 6500 series switches, active-active redundancy is configured on the two ACEs in separate chassis, and you run stateless UDP traffic through the ACEs, some connections may fail. A trace shows that the successful flows use the ACE virtual MAC as the destination and the unsuccessful flows use the physical interface MAC of the standby ACE. A display of the default route and the svclc RHI routes shows two entries for the VIP in question. If you enter the show ip route command, the preferred route is the standby interface instead of the alias IP address. Workaround: None.

CSCtb66309—When you add a set of hosts and later delete the same under the network type object group, you will observe a policy action nodes leak for an object group. Workaround: None.

CSCtb72635—When you run a script for the show tech detail command on an ACE that has 4000 BVI and 4000 VLAN interfaces configured, the ACE may become unresponsive. Workaround: None.

CSCtb77652—When you configure the failaction reassign or failaction across-interface command and you enter related show commands or attempt to ping from the ACE, the ICMP and ARP manager generates an rpc call failure message. Workaround: Do not configure the failaction reassign or failaction across-interface command.

CSCtb82146—When you configure a global service policy and add a new interface, the ACE drops packets to the existing interface for a short duration. Workaround: Add a service policy where you add the new interface if the configuration is dynamic.

CSCtd94085—You may observe an MTS memory leak for an invalid or a nonexistent process or PID. For a Vshell process, the MTS message queue limit is limited to a maximum of 4096 messages. Beyond that limit, any new message (for example, a changeto command is being executed), will get dropped and the following warning message is displayed on the console:

Warning:- MTS queue is full for opcode "<opcode value>" sap "<sad_id>" pid "<pid>" 
clear idle debug plugin sessions or telnet/ssh connections to recover. 
 
   

Sometimes, the PID that is displayed here may be invalid (no real process associated with it). Workaround: None.

CSCte26173—During periodic XML queries on ACE for show commands, such as the show ft group status command, the ACE places the bash core files in the core: directory. Some files are unpackaged and other files are mispackaged as VSH core files. Workaround: None.

CSCte68680—During a supervisor switchover in a VSS system that contains an ACE module in each Catalyst 6500 chassis, the ACE may reboot due to the AAA daemon. Workaround: None.

CSCtf33100—If two or more probes associated with the server farm are in the failed state, at least one probe is in the passed state and the fail-on-all configuration is removed, the real server remains in the OPERATIONAL state and is not moved to the PROBE-FAILED state. Workaround: None.

CSCtf36703—When the device undergoes stress or excess load, the performance of the generic protocol parsing, HTTP Layer 7 load balancing enabled with SYN-COOKIE or HEADER-INSERT decreases by 7 to10 percent. Workaround: None.

CSCtf38995—After you reboot the ACE, you cannot remotely log in to the ACE using RADIUS authentication. Workaround: Perform a ping between the server and ACE before using authentication.

CSCtf39655—If you configure the send-data option inside a finger probe with length greater than four characters, the probe fails. Workaround: Configure a send-data length with less than four characters.

CSCtf44818—The ACE module occasionally displays the incorrect value in the Unicast bytes input counter for the interface. This issue can cause problems for SNMP tracking the traffic, which in turn displays ~50Gbps flowing through the ACE. Workaround: Configure the SNMP application to ignore the counter increases above a certain value.

CSCtg46241—During a high rate of SIP calls per second and during the initial processing of packets, if the SIP inspection engine encounters resource allocation failures such as memory allocation, object allocation, inspect config version mismatch failures, the ACE may reboot. Workaround: Disable the SIP inspection feature, if possible.

CSCtg70913—When users whose accounts have expired attempt to log in to the ACE through SSH or Telnet, they succeed. Workaround: None.

CSCtg93332—When you configure the mac-address autogenerate command on the client VIP interface in bridge mode, traffic to VIP starts failing. Workaround: Delete the client side interface and readd it.

CSCtg94333—When you create 10 contexts and allocate 10 percent of the resources to each context, ACL merge-list creation fails and management traffic to the VLANs fail. When you enable the debug access-list merge errors command and add or delete VLANs, the ACE displays merge-list errors. Also, attempts to ping and Telnet for the management access fail. Workaround: Remove the resource allocation from the contexts.

CSCth02932—When you enter the show np 1 me-stats | memory | status or show tech-support commands in the user context, the ACE displays an error message. Workaround: Enter these commands in the Admin context.

CSCth15050—When you place a VIP in a Layer-3 policy map out of service, the ACE does not remove the VSERVER-related ARP entries from the ARP cache. Workaround: Clear ARP to clear all ARP entries.

CSCth30569—When you apply a large multi-context configuration, the arp_mgr service in ACE becomes unresponsive. Workaround: None.

CSCth52802—When you configure Radius Layer 7 load balancing with framed-IP sticky on the ACE for HTTP traffic and the ACE executes multiple scripts in parallel to change the load-balancing configuration while another script executes the show conn command in a loop, the ACE reboots and may generate incomplete core files. Workaround: None.

CSCth75674—When the SCP HW watchdog on the ACE fails to detect the timer expiry, the Catalyst 6500 supervisor power cycles the ACE with an SCP keepalive failure message. The watchdog may fail to detect the timer expiry when the internal counters overflow. The SCP HW watchdog mechanism detects when the ACE becomes unresponsive and collects the core files in error case scenarios which prevents the power cycling by the Catalyst 6500 supervisor. On rare occasions, the watchdog may fail to detect the timer expiry. Workaround: None.

CSCth85288—When a Layer-2 connected real server is in the ARP_FAILED state and you shutdown or delete the corresponding Layer-3 interface, the real server state does not transition from the ARP_FAILED state to Operational or the probe failed state. Workaround: Remove the real server and reconfigure it.

CSCth94715—When you configure multiple contexts in an FT configuration and configure probes for each context but you configure one context with an FT track probe, if you remove these contexts from the FT configuration and delete them, health monitoring may become unresponsive. Workaround: None.

CSCth99982—When you configure an ECHO TCP probe with send-data and regular expression (regex) values, the probe always passes even if the server sends a regex that does not match the sent-data value. Workaround: You can use a TCP probe with send-data and regex values as required instead of an ECHO TCP probe.

CSCti02008—When an ACE running configuration contains 64K ACLs and you roll it back to an empty checkpoint, the checkpoint rollback occurs in approximately 45 to 50 minutes. Workaround: None.

CSCti18687—When you perform various levels of end-to-end or back-end SSL traffic performance tests on the ACE, SSL-initiation and HTTPS probe connections remain in the CLSRST state until the inactivity timeout clears them. Workaround: Decrease the inactivity timeout or manually close the connections through the CLI commands.

CSCti26266—When the To CP traffic flows without any issues, the show cde health command displays the BRCM pull status as not pulling. The CDE FPGA asserts flow control to a different channel based on the buffer threshold. When the condition clears, the show command output does not update the pulling status. This is a display issue with no functionality impact. Workaround: None.

CSCtj03057—When the ACE has DNS traffic that has greater than 40K queries per second and is not configured with DNS inspection, the ACE may reboot. Workaround: None.

CSCtj18891—When you configure a primary VLAN interface as a server VLAN on the ACE, the ACE does not load balance to the real servers on the secondary VLAN. Workaround: After performing the configuration, reboot the ACE module for traffic to work properly.

CSCtj20310—When you enable XML, the XML output is not seen for the show serverfarm name retcode details command. Workaround: None.

CSCtj21592—When you configure two or more probes to a server farm, the probe instance is not created after removing one of the probes on the server farm and entering the inservice command on the real server. Workaround: None.

CSCtj24740—When you enter the ucdump -d command to force the DP core on IXP 1 and IXP 2 in the active ACE, traffic flow stops. Workaround: None.

CSCtj35994—When you configure a user context on the ACE for KAL-AP, the ACE unexpectedly reboots and generates a gslb_proto_log.943.tar.gz core file. The last boot reason is Service "gslb_proto". Workaround: None.

CSCtj54534—Under normal operating conditions with logging configured, you cannot disable a specific syslog identification string. Workaround: Disable logging.

CSCtj61334—When you configure SNMP traps on the ACE, any reason (for example, an entire server farm goes down) generates a large number of traps. When the SNMP trap queue is full, the ACE displays error messages similar to the following:

snmpd[1027]: (ctx:9)send_notification: new: enqueueing notification........ 
snmpd[1027]: (ctx:9)ERROR: notif_enqueue_tail : Size of the notif queue is more than 
the MAX size 250 
 
   

You can also display error messages by enabling the debug snmp errors and debug snmp notif_trace commands. Workaround: Use syslogs instead of traps. If possible, decrease the number of SNMP traps and the rate that they are sent.

CSCtj67085—When the ACE has Layer 7 traffic greater than 40K CPS and you enter a show command that displays more than 3 to 5 lines of output, the Telnet management connections to the CP may become unresponsive. Workaround: None.

CSCtj67114—When you configure a large number of directly connected real servers on the ACE and they are in the DOWN state, ARP resolution may fail intermittently for the directly connected hosts. Workaround: Transition the directly connected hosts to the UP state or decrease the number of directly connected hosts.

CSCtj70903—When you configure an engine ID on the ACE and then configure users, if you change the engine ID to another value, the ACE deletes the users. However, if you reconfigure the original engine ID value, the ACE restores the deleted users. The ACE should permanently delete the users after you change the engine ID. Workaround: None.

CSCtj71785—An invalid ACK handshake may move a Layer 7 TCP connection to the ESTABLISHED state. This behavior may be a result of the TCP three-way handshake not being RFC 793 compliant.

For example, a standard three-way handshake functions as follows:

1. SYN-SENT --> <SEQ=100><CTL=SYN> --> SYN-RECEIVED

2. ESTABLISHED <-- <SEQ=300><ACK=101><CTL=SYN,ACK> <-- SYN-RECEIVED

3. ESTABLISHED --> <SEQ=101><ACK=301><CTL=ACK> --> ESTABLISHED

However, with crafted packets, the following three-way handshake may occur, which results in the connection moving to the ESTABLISHED state:

1. SYN-SENT --> <SEQ=100><CTL=SYN> --> SYN-RECEIVED

2. ESTABLISHED <-- <SEQ=300><ACK=101><CTL=SYN,ACK> <-- SYN-RECEIVED

3. ESTABLISHED --> <SEQ=101><ACK=300><CTL=ACK> --> ESTABLISHED

In this case, the final ACK handshake is not equal to SEQ+1.

Workaround: None.

CSCtj72215—When you configure SNMP local and peer engine IDs on the active ACE and you enter the show startup config|include snmp command on the standby ACE, the standby ACE does not display the exchanged value for the engine IDs (local ID as the peer, peer ID as the local). Both active and standby ACEs display the same output. Workaround: None.

CSCtj75477—When you simultaneously run SNMPv3 walk on the Admin context and a script to add or delete an SNMP user on a user context, and the ACE receives HTTP load-balancing traffic, the snmpd service terminates. Workaround: Do not simultaneously run SNMP walk on a context and a script to add or delete an SNMP user on another context.

CSCtj75506—When you run SNMPv3 walk and add or delete user scripts in parallel on different contexts, the snmpd service terminates on receiving a signal 6 and the ACE reboots. Workaround: None.

CSCtk04002—When both active and standby ACEs are running software version A2(3.1) with routed connections and you upgrade the standby ACE to software version A2(3.2) or later, if you use the ft switchover all command in the Admin context of the active ACE, it advertises an RST to the client and server. The connections close on the client and server. However, the connections still exist on the ACE. This issue is seen only with software version A2(3.1). Workaround: Use the ft switchover all command in the Admin context of the standby ACE running software version A2(3.2) or later.

Command Changes in Software Version A2(3.3)

Table 9 lists the command changes in software version A2(3.3).

Table 9 CLI Command Changes in Version A2(3.3)  

Mode
Command and Syntax
Description

Exec

show np [1|2] me-stats -scde

Per CSCtf44818, three counters in the output for this show command have been replaced, as follows:

The IMPH length errors counter replaces the Runts errors counter. This counter increments when the IMPH header lists the length as greater than 10000 bytes.

The L2 offset errors counter replaces the Invalid header errors counter. This counter increments when the pkt_ctxt.offset.l2 (an internal data value) is greater than 100.

The L2 offset > IMPH length errors counter replaces the Bad CDE length errors counter. This counter increments when the L2 offset (pkt_ctxt.offset.l2) is greater than the IMPH length (in the IMPH header).

Exec

show np [1|2] me-stats -shttp

Per CSCtj05814, the output for this show command now includes the Cookie parse errors ignored counter that increments when the cookie-error-ignore feature, enabled by the parameter-map HTTP cookie-error-ignore command, is in effect for a request.

Exec

show np [1|2] me-stats -stcp

Per CSCtj18711, the output for this show command now includes the Reassembly timeouts counter that increments when reassembly timeouts occur.

Exec

show parameter-map name

Per CSCtj05814, the output of this show command now displays the status for the cookie-error-ignore command, disabled or enabled.

Exec

show snmp engineID

Per CSCth59753, the output of this show command now includes the Local SNMP engineID and Peer SNMP engineID fields.

Configuration

snmp-server engineid value_1 [peer engineid value_2]

Per CSCth59753, the new peer engineid value_2 option allows you to configure a different engine ID for the standby ACE in a redundant configuration. If you do not enter this option, the active and standby ACEs have the same SNMP engine ID.

For more information on this command, see the "Configuring SNMP Peer Engine ID for the Standby ACE" section.

Configuration

[no] crypto rehandshake enabled

Per CSCth85502, this command allows you to enable SSL rehandshake for all contexts on the ACE. Enter this command in the Admin context.

Use the no form of this command to reset the default behavior. By default, the ACE rejects SSL rehandshake and you can enabled it at the SSL-proxy level by configuring the rehandshake enable command in the SSL parameter map.

Parameter map HTTP

[no] cookie-error-ignore

Per CSCtj05814, this command configures the ACE to ignore malformed cookies in a request and continue parsing the remaining cookies.

Use the no form of this command to reset the default behavior. By default, when the ACE finds a malformed cookie in a flow, it stops parsing the remaining packets.

Policy map inspection HTTP class

Policy map inspection HTTP match

[no] passthrough [log]

Per CSCti13494, this command configures the ACE to bypass HTTP parsing after it processes the CONNECT request. It only works with a matching port misuse configuration and a CONNECT request.

By default, the ACE performs HTTP parsing after it processes the CONNECT request. Use the no form of this command to reset the default behavior.

For more information on this command, see the "Bypassing HTTP Parsing After Processing a Connection Request" section.


System Log Messages

Software version A2(3.3) includes following system log (syslog) message changes.

901001

Error Message    %ACE-2-901001 kernel:  Arpmgr busy, Possible ARP flood, packets arp 
pkts were dropped over last seconds secs

Explanation    When the arpmgr is busy, this message appears on the console and this syslog.

Recommended Action    If a large number of real servers are in the ARP failed state, bring them up. Verify that a broadcast storm is not occurring in the network. Verify that an ARP flood is not occurring in the network. For example, in the case of many servers with Active-Active NIC teaming in the network, change them to Active-Standby NIC teaming or similar.

Software Version A2(3.2a) Resolved Caveats and Open Caveats


Note Software version A2(3.2a) has replaced software version A2(3.2).


This release note includes the resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.2a):

Software Version A2(3.2a) Resolved Caveats

Software Version A2(3.2a) Open Caveats

Software Version A2(3.2a) Resolved Caveats

The following resolved caveats apply to software version A2(3.2a):

CSCti29242—When you configure access control lists (ACLs) in the ACE, using the access-list name resequence command to renumber the line numbers may cause an ACL merge error and the access-list configuration fails to download to an interface. Workaround: Do not use the access-list name resequence command when you are configuring ACLs.

CSCti88248—When the ACE is waiting to reassemble client packets, it may reset TCP-based client connections if all the following conditions exist:

ACE is configured with a Layer 7 load-balancing policy where the ACE proxies the client-side TCP connection before making a load-balancing decision

Client-side connection experiences packet loss

The TCP TX racing messages (data) counter in the output of the show np n me-stats -stcp is incrementing

This problem can also occur with secure (SSL) terminated connections. Workaround: Configure an empty connection parameter map and add it to a multi-match policy map under the class map that is configured for the VIP experiencing the problem. For example:

parameter-map type connection TCPReassembly
policy-map multi-match MultiMatch_PolicyMap
   class HTTP_VIP_80
      loadbalance vip inservice
      loadbalance policy L7_HTTP_PolicyMap
      loadbalance vip icmp-reply active
      connection advanced-options TCPReassembly
 
   

CSCti89812—When the ACE is reloading or becomes unresponsive, you may observe flow control on the supervisor engine. A syslog message similar to the following is generated on the supervisor engine:

%FABRIC-SP-6-TIMEOUT_ERR: Fabric in slot slot_number detected excessive flow-control 
on channel channel_number
 
   

Workaround: None.

Software Version A2(3.2a) Open Caveats

The open caveats in software version A2(3.2a) are the same as those in software version A2(3.2). For details, see the "Software Version A2(3.2) Open Caveats" section.

Software Version A2(3.2) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

This release note includes resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.2):

Software Version A2(3.2) Resolved Caveats

Software Version A2(3.2) Open Caveats

Command Changes in Software Version A2(3.2)

System Log Messages

Software Version A2(3.2) Resolved Caveats

The following resolved caveats apply to software version A2(3.2):

CSCsm39305—After performing tests in which the FT port channel flaps 1,000 consecutive times with a 5-second delay between flaps, the active and standby ACEs are left in the active/active state. On the active ACE, the show ft peer detail command displays the FSM_PEER_STATE_SRG_CHECK state. On the standby ACE, this command displays the FSM_PEER_STATE_TL_SETUP state. Workaround: None.

CSCsm53617—When you configure the ACE with an inspect action and then remove it from the running configuration, the ACE displays the following error.

Error: Cannot delete this object as this is referenced by inspect action.
 
   

Workaround: Reboot the ACE.

CSCsy05318—When you add a class map to a configuration with a large number of class maps and the ACE fails to add it to the running configuration, the ACE displays an error message that does not describe the actual issue. Workaround: None.

CSCsy74228—When a connection gets stuck in the CLSRST state, it does not disappear after the idle timeout. The clear conn all has no impact on it.

CSCsy94458—The output of the show resource usage command may show that the bandwidth has been denied in the Admin context of the ACE. The counters indicate that bytes have been dropped prior to a configuration having completed, but the count does not increment thereafter. There is no adverse effect of these drops; it is a cosmetic issue only. This behavior occurs in the display for the Admin context only. Workaround: None.

CSCsz14033—If you delete disk0 without the filename and you assign a filename on ACE, it deletes the entire disk0 directory rather than the file. If the directory is empty and you enter a dummy filename, it deletes the disk0 directory and disk0 cannot be used after that. The disk0 directory is lost and is not created until the next reboot of the ACE. Workaround: Reboot the ACE.

CSCsz15921—The ACE does not correctly convert the logging host commands in the configuration to XML form which causes the resulting XML file to load improperly. Workaround: None.

CSCsz54546—When a probe is successful, the output of the show probe detail command may display 0 in the Last status code field instead of the actual code. If the probe is failing, the Last status code field value is correct. Workaround: None.

CSCsz87249—The following log messages may appear sporadically in the ACE log:

"can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg"

"can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg"

These messages do not impact the operation of the ACE. The messages may be caused by more than one device that is accessing the ACE context through XML. Workaround: None.

CSCta21363—When you create a username and assign the role of server maintenance, the Config > Operations > Real Servers menu options for activate and suspend are not there. There are no buttons to allow you to activate or suspend a real server. Workaround: Create a custom role.

CSCta40378—When backend SSL connections fail handshake or rehandshake, the ACE closes the connection without acknowledging the FIN from the real server. The real server connections are left in a FIN_WAIT1 state, and must rely on the server operating system to time out. Workaround: None.

CSCtb02056—When you configure the ACE with SSL certificates and keys in multiple contexts, the output of the show crypto certificate all command may become corrupted. Workaround: Use the show crypto certificate cert_name command instead of the show crypto certificate all command.

CSCtb44729—When you configure the ACE for Layer 7 load balancing and a connection is closed before it is processed by the load balancer, the show conn command displays no connections but the show serverfarm command displays the current connection for the real server even after all traffic has stopped. Workaround: Remove the real server and readd it.

CSCtb56199—When the ACE applies a configuration to the network processor engines, it may become unresponsive and the following message appears on the console:

ERROR : DRV : PCI send failed! PCI RIngs in Use
 
   

Workaround: None.

CSCtb86697—When you modify a NAT pool under an interface configuration, the ACE may log the following error that you can display by using the show logging command:

Sep 4 2009 12:34:03 ace/ace: %ACE-1-106028: WARNING: Unknown error while processing 
service-policy. Incomplete rule is currently applied on interface vlan953. Manual roll 
back to a previous access rule configuration on this interface is needed.
 
   

You may also see Service download failures in the show interface command output. Workaround: Remove and reapply the NAT pool configuration.

CSCtb95036—When you enter the checkpoint create command, it generates errors and the output displays attempted execution of shell commands. Workaround: When you create checkpoints, avoid using opening braces, closing braces, white spaces, or any of the following symbols: `$&*()\|;'"<>/?

CSCtb95153—After you apply configuration changes to a NAT pool, the ACE may become unresponsive because a network processor (NP) microengine (ME) becomes unresponsive on X_TO_ME. Workaround: None.

CSCtb95136—When a server sends a request to a client in an RTSP configuration, the ACE resets the RTSP connections. RTSP servers are supported only in an asymmetric client-server mode (required and recommended methods). Workaround: None.

CSCtc07741—When you configure Message Of The Day (MOTD) for the ACE banner with the $(hostname) keyword, the show banner motd command displays the banner but generates an error and a VSH core file to the core: directory. This issue only occurs when the you use the $(hostname) keyword for the banner but the ACE has not been rebooted. Workaround: Reboot the ACE.

CSCtc12692—If you manually place a VIP out of service, for example, through the no loadbalance vip inservice command, the VIP continues to respond to ARP requests after it goes out of service. Workaround: Delete the interface where the service policy is applied. Then, reconfigure the interface and attach the service policy.

CSCtc20009—When you configure the ACE to send SNMP traps, specifically server farm traps, and apply a server farm to a service policy, the server farm changes state and the ACE generates duplicate traps. Also, if you apply the server farm more than one class map under a multi-match policy map, the ACE generates duplicate trap for each. If you apply two class maps, the ACE generates four traps. If you apply three class maps, the ACE generates six traps. Workaround: None other than disabling the traps.

CSCtc54698—When you enable logging on the ACE, the ACE does not generate syslog message when a server farm goes down and the VIP goes to the OUTOFSERVICE state. Workaround: None.

CSCtc55162—When the ACE TCP protocol stack is processing a large amount of data, the two ACE modules in a redundant configuration may become unresponsive, generate a core dump file, and reboot. Workaround: Configure the TCP options in a connection parameter map to clear (not allow) window scaling.

CSCtc77380—When you use the XML management protocol to query the ACE for context configuration, the ACE generates invalid XML output for the show context command when you enter this command in a user context. Workaround: Enter the show context command in the Admin context.

CSCtc87588—When TACACS+ is configured, the ACE does not account for configuration mode commands that contain sensitive information (for example, keys and passwords). These commands do not appear in the local ACE accounting log nor in the TACACS server accounting log. In the ACE accounting log, there are descriptive entries, (for example, "deleted user"). In the supervisor engine accounting log, the commands are logged but the sensitive information is masked. Workaround: None.

CSCtc91087—When you change the value of the limit-resource all minimum command, the ACE may start rate-limiting traffic at a different throughput level from the level that the show resource usage command displays. Workaround: None.

CSCtc94802—When the ACE performs SSL URL rewrite for a hostname that matches string.cisco.string and you configure a .*\.cisco\..* regex, the ACE rewrites the URL to HTTPS and adds a forward slash (/) at the end of the URL. SSL URL rewrite with the configured regex needs to match string.cisco.string. Workaround: Use the .*[.]cisco[.].* regex.

CSCtc94844—When you configure cookie insert and failaction purge and the probe status goes up and down repeatedly, the show serverfarm detail command may display a current connections counter that is not accurate (it should be null). Workaround: None.

CSCtd04486—When you are using an SNMP probe for the least-loaded server farm predictor and the probe returns the OID value of 0 from the real server (the server is least loaded), the real server may not receive any connections and the ACE distributes all the connections to the other servers in the server farm. Workaround: Change the predictor autoadjust value from the default of max to average. The ACE autoadjusts the load to the average load of the server farm and the real server receives connections based on its having the average load of the server farm.

CSCtd22008—When the ACE perform end-to-end SSL and a client sends either an RST-ACK after the connection has already been established or an RST-ACK to a real server FIN, the counter failures for the show serverfarm name or show rserver name command increments. Workaround: None.

CSCtd40797—When you use KAL-AP with GSS and active or standby redundant ACE modules, the GSS reports an invalid answer state if the ACE VIP fails on the active ACE but not on the standby ACE and there is no failover between the redundant ACE modules. ACE and GSS communicating via KAL-AP, by VIP / by TAG. Active ACE VIP reports an OUTOFSERVICE state and standby ACE VIP reports an INSERVICE state. VIP state discrepancy can occur due to probe failure or some other manual intervention. Also no failover occurs between the redundant ACE modules.

The GSS answer initially transitions to an OFFLINE state when the active ACE VIP fails and then the GSS answer transitions back to an ONLINE state as it is now receives KAL-AP load information from the standby ACE. Any new DNS query sent to the GSS receives an A-record VIP response because the answer is ONLINE but connectivity to the ACE VIP fails due to the fact that the Active ACE VIP is still considered down. Workaround: Use the ACE alias IP address rather than both the active and standby ACE interface VLAN IP address so that only the active ACE will provide VIP state.

CSCtd52722—When a large number of processes are active on the CP, the CP console displays the following message text:

Couldn't save crashinfo.
Error</noCmdBold> just prior to a reload.
 
   

The crash info data is actually saved and can be submitted to the TAC. Workaround: None needed, the message is incorrect. The crashinfo file is saved; however, it is truncated. The truncation may keep some detailed information from being saved to the crashinfo file.

CSCtd53161, CSCsy98701—Occasionally, the connection sticks to the wrong server. The show sticky database command displays multiple entries for the same sticky hash. This occurs if there is an expired entry within the same bucket. Workaround: Clear the sticky database to remove the wrong entries.

CSCtd66906—When you upgrade beyond ACE A2(1.3) version, ACE user roles defined as network monitor can no longer enter the delete... command. Workaround: None.

CSCtd69388—When you configure two ACEs for redundancy and an ACE processes a load-balancing redundancy message from the peer, it may become unresponsive temporarily and reboots. Workaround: None.

CSCtd75203—The output of show sticky database detail command displays a hexadecimal equivalent for IP addresses. Workaround: None. Manually convert hexadecimal to decimal.

CSCtd83789—If customized scripted probes fail repeatedly, the core files produced as a result fills up the disk and prevents other operations from functioning properly. It impacts any activity that implies writing on the disk. Specifically, configurations are truncated because of the missing space on the disk. Workaround: None.

CSCte05073—When the ACE is running software release A2(3.0) under normal operating conditions, it reboots and generates a core dump. The ACE displays either of the following reasons:

last boot reason: NP 1 Failed : NP ME Hung

dir core:
qnx_1_mecore_log.999.tar.gz
ixp1_crash.txt 
 
   

last boot reason: NP 2 Failed : NP ME Hung

dir core:
qnx_2_mecore_log.999.tar.gz
ixp2_crash.txt
 
   

Workaround: None.

CSCtf00210—When you configure the mac-address autogenerate command with the ip dhcp relay command on an interface, the ACE fails to relay the DHCP request to the configured server and the counters displayed by the dhcp relay statistics command do not increment. Workaround: Remove the mac-address autogenerate command from the interfaces and reboot the ACE.

CSCtf01034—The standby ACE may have a higher number of connections than the active ACE. Workaround: Configure a shorter connection inactivity timeout.

CSCtf14370—When you submit the following syntax to the XML agent, it fails with the complaint about backup attribute.

<policy-map_lb type="loadbalance" match-type="first-match" pmap-name="testuk-1">

<class_pmap_lb match-cmap="www99-www-url-1">

<serverfarm_pmap sfarm-name="www99" backup="WWW-NOT-AVAILABLE"></serverfarm_pmap>

</class_pmap_lb>

</policy-map_lb>

Workaround: The correct syntax is shown in the following syntax:

<policy-map_lb type="loadbalance" match-type="first-match" pmap-name="testuk-1">

<class_pmap_lb match-cmap="www99-www-url-1">

<serverfarm_pmap sfarm-name="www99" config="backup" backup-name="WWW-NOT-AVAILABLE"/>

</class_pmap_lb>

</policy-map_lb>

CSCte16068—When you attach a probe to two different real servers and delete one of the servers, the probe instance for the other server remains in the INVALID state. Workaround: Delete probes that remain in the INVALID state for all real servers and readd them.

CSCte25964—When you enter the show snmp group command from a non-Admin context, the command does not display any output. Workaround: None.

CSCte28915—The show snmp group command displays two outputs for the same SNMP group making it unclear which one is the real output for default SNMP group. Workaround: None.

CSCte44232—The output of the show logging message all command displays numeric syslogd identifiers for unsupported messages. Workaround: None.

CSCte45777—If you configure a connection with a timeout value of 2,147,483,648 or greater, the connection times out immediately after the connection is set up. Workaround: Do not set a timeout value greater than 2,147,483,648.

CSCte46550— When you configure a catch-all VIP that HSRP traffic can hit and the ACE reboots, it forwards the HSRP multicast traffic for a few seconds as it is going down. Workaround: Configure an input ACL to deny HSRP traffic.

CSCte53218—International step-up certificates fail with older browsers and ACE SSL termination. SSL connections fail with reset. The show np 1 me-stats "-s nitrox" command displays a FINISH_MAC_MISCOMPARE error. The show stats crypto command displays counts in the BAD_RECORD_MAC sent field. Workaround: None.

CSCte56065—Under normal operating conditions, Linux b-shell executables occasionally core on the ACE module. These cores are either incorrectly packaged as Virtual Shell (VSH) cores or not packaged and compressed and left in the core: directory as core. Workaround: None.

CSCte61479—The show buffer usage command displays incorrect values that are very large for the usage of the type of internal ACE buffers. Workaround: Reboot the ACE to clear the values.

CSCte63173—A buffer leak occurs due to Inter-Process Control Plan (IPCP) messages between the Control Plane (CP) and Data Plane (DP). Workaround: When the problem becomes severe, reboot the ACE.

CSCte68716, CSCtf72863—When you configure window scaling for a VIP on the ACE, if a client does not support window scaling access, the ACE uses the configured window scale to scale the receive window and still advertises a scaled receive window after the TCP 3-way handshake is completed. Workaround: Configure the tcp-options window-scale clear command in the connection parameter map.

CSCte61409—When you enter the show cde health command and the ACE module is under a high load, it incorrectly displays the BRCM pull status as [Not pulling]. Workaround: None.

CSCte66195—On a ACE running software version A2(2.3), when you do not configure SIP UDP probes with the rport enable command, the ACE sends two semicolons in the Via header, similar to the following:

Via: SIP/2.0/UDP 10.10.10.10:32789;;branch=z9hG4bK25708969
 
   

Workaround: Configure the rport enable command if you are able to use the rport extension in the set up.

CSCte66071—TCP or UDP configured port ranges are being inherited for non-TCP non-UPD protocols when configured inside an object group right after a TCP or UDP range. Workaround: Configure the ACL directly without using the object group.

CSCte66814—The ACE sends RHI messages to remove static routes and then immediately sends a new message to readd the same route. Workaround: None.

CSCte83538—When you enter show buffer usage, an additional field "Hi watermark" is displayed, which allows more visibility into the buffer usage to monitor the high watermarks.

CSCte83727—When you enter the show ipcp event-history command, it indicates that the internal IPCP queue is full causing possible failures. Workaround: None.

CSCte83745—The ACE sends traps when a real server in a server farm changes state (for example, a probe fails and then the server becomes operational again). When the probe failure is detected, the ACE sends the cesRealServerStateChangeRev1 trap. When the real server becomes operational again, the probe succeeds, but the ACE incorrectly sends the cesRealServerStateUpRev1 trap. The cesRealServerStateUpRev1 trap should only be seen after user intervention (for example, after you enter the inservice command). The ACE should send the cesRealServerStateChangeRev1 trap when a server becomes operational after a probe failure.

CSCte92842—When you try to remove the limit-resource all command, all the ACE contexts associated with that resource class are left out of resources that are not separately defined. Workaround: After the fix with version A2(3.2), the following warning message is displayed when you try to remove the limit-resource all from a resource class:

ACE-tb3/Admin(config)# resource-class a
ACE-tb3/Admin(config-resource)# no limit-resource all minimum 0.00 maximum unlimited
Warning: The context(s) associated with this resource-class
will be denied of all the resources that are not explicitly
configured with minimum limit in this resource-class
 
   

CSCte96172—When you configure a service policy on an interface, if you configure overlapping the subnets within a class map, a syslog error is generated. This should not be generated as there is no real problem in configuring overlapping subnets. Workaround: None required.

CSCte99505—MTS leak is observed if the login fails due to the securityd process being busy. Workaround: None.

CSCtf10882—When you configure an HTTP class map with more than one URL matching statement, ACE XML interface returns 105.

CSCtf18582—The show running-config command does not show the username "user." The copy running-config startup-config and the write memory commands do not save the "username user password password" line in the startup-configuration file. Workaround: Avoid using the username "user."

CSCtf19792—When you configure the shared-VLAN host ID for the local and peer ACE modules, the ACE allows you to configure the same value for the shared-vlan-hostid and peer shared-vlan-hostid commands. However, the ACE should not support this configuration. Workaround: Configure different values for the shared-vlan-hostid and peer shared-vlan-hostid commands.

CSCtf31125—When any hash predictor is enabled on a server farm and the hashing result is to a real server that is in the STANDBY state, the ACE load balances the incoming request to that server instead of recalculating the hash and finding the next available server. Workaround: None.

CSCtf31485—When the ACE memory consumption reaches 99%, you should have a configurable option to proactively reboot the ACE. Workaround: None.

CSCtf31573—When you enter the ft switchover command to transfer mastership, the ACE resets some connections. Workaround: None.

CSCtf33319—When you enter the header rewrite or ssl url rewrite command and a static parse error occurs because of an incorrect field on the server side, the ACE does not forward the page back to the client and sits idle. The client does not receive data back from the GET request. Workaround: Either fix the server-side HTTP headers or do not use rewrite.

CSCtf36813—When you use the context command to create a user context through the ACE CLI, the command fails and the ACE displays a message that the disk is out of space, which is not the case. Workaround: Complete an RMA for the Compact Flash (CF) in the ACE appliance to determine if it clears the condition.

CSCtf55374—The ACE displays the rate limit by default without it being configured or enabled. Workaround: Ignore the rate-limit values in this scenario.

CSCtf55391—When issuing an SNMP GET of the sysObjectID OID, the ACE10 and ACE20 currently have the same sysObjectID OID value. Workaround: Verify the ACE model by using ACE CLI or the show mod command from the MSFC.

CSCtf57455—When a standby ACE running software version A2(1.5a) unexpectedly reboots, it generates the following core dump:

Last boot reason:  Service "itasca_route_mgr"
 
   

Workaround: None.

CSCtf60389—If you configure TCP probes with small intervals and set the termination mode as forced, the TCP probe stops firing if the server sends an RST after the TCP handshake. Workaround: Remove and readd the faulty probe from the real server.

CSCtf65713—When you configure the ACE with multiple interfaces that share the same multi-match policy, you cannot ping a VIP configured on the ACE. Workaround: Removing and reapplying a class map under the multi-match policy may correct this issue.

CSCtf65934—Client authentication fails if the authentication group contains the sub-CA certificate instead of the root CA certificate. Workaround: Configure the authentication group with the root CA.

CSCtf70322—When a remote authentication with TACACS is configured and if the username has a "!" (exclamation character), the login fails for usernames with "!" character. Workaround: Do not configure usernames with "!" (exclamation character) in the TACACS server.

CSCtf75106—When SIP inspected requests are bigger than 2KB, the ACE drops them and the counter 'SIP: Memory Allocation Failure:' increases. This is observed with many large SIP PUBLISH requests sent over connections inspected by the ACE. Workaround: You can monitor appInspect memory through the show np 1|2 memory | inc appInspect command. The nominal memory usage is 11M. When this value begins to approach 29M, a proactive reboot would be required.

CSCtf75936— In both FT or non-FT configurations, normally when you add a new entry to the object group, it expands. If you add a new entry after removing and adding the first access list where the object group is associated, it does not expand. Workaround: Remove all of the access list and readd it.

CSCtf76222—When you configure the ACE module for Role Based Access Control (RBAC) using custom domains and roles, and you log in to the ACE as a user with a user-configured domain and role, some commands do not work. Workaround: Use the specific versions of the show reserver name and show serverfarm name commands.

CSCtf83851—When you set user-defined resource allocation rates on an ACE module, the percentage for connection rate is based on the data sheet performance 325K CPS. Once the limits are applied you will see the user context maximum is now 500K, which is also not right. It should also work where the maximum value of user context is the ACE module maximum of 325K minus any configured minimums.

CSCtf88100—When you upgrade the ACE from software version A2(1.x) to A2(2.x), the ACE did not accept any SSL filename previously configured with 40 characters in length. This issue caused a major network outage. Workaround: Perform either of the following:

Downgrade the ACE to software version A2(1.x). Transfer any keys and certificates with names exactly 40 characters from the ACE and reimport them as 39-character or less filenames.

Regenerate the keys and certificates from the ACE running software version A2(2.x) and import them as 39-character or less filenames.

CSCtf88901—When you use a management tool that uses a script for XML formatting, the ACE adds four extra lines. You can see the extra lines when you use the show service-policy detail command. The failure is specific to the context where you have the formatting. When you remove a class map, the next one down has the same issue. Workaround: Split the respective policy map where the VIP is configured.

CSCtf94399—When you configure a redundant ACE pair with an FT peer, FT interface and query VLAN interface, and then you delete the FT interface within the FT peer, the ACE also removes the query VLAN interface from the FT peer configuration. However, when you enter the show ft peer ... detail command, the query VLAN interface remains. Workaround: First remove the query VLAN interface. Then, remove the FT interface.

CSCtg01079—When you configure a forward action on a VIP instead of a server farm on the standby ACE and enter the show stats loadbalance command, the Total IDMap Lookup Failures counter increments. Workaround: Replace the forward action with a transparent server farm and a real server corresponding to the next-hop route.

CSCtg10476—When you place a real server in and out of service, the ACE reboots and generates a loadBalance core file. Workaround: Wait until the real server inservice or no inservice configuration change completes before reversing the action.

CSCtg17765—When you enter the show service-policy policy_name detail command, the console becomes unresponsive until you press Ctrl-C. Workaround: Press Ctrl-C.

CSCtg18549—When you configure the ACE in routed mode with FTP inspection and dynamic NAT, if a server behind the ACE acting as an FTP client makes a connection to a server outside the ACE, the active data channel fails. While the control channel messages are properly corrected with FTP inspection, when the server opens the data channel with a SYN to the NATed port, the ACE sends this SYN with the translated IP address to the client but does not translate the port. Workaround: Use the inspect ftp strict command.

CSCtg20636—When you configure the 128th global service policy in the running configuration, the ACE displays an error that is not informative. Workaround: None.

CSCtg23302—When you configure the IP relay agent on the ACE VLAN interfaces, due to the fix for CSCta47529, the ACE DHCP relay agent forwards DHCP Unicast packets which is incorrect. Workaround: None.

CSCtg27655—The Fault Tolerant (FT) TL connection is the TCP connection between primary and secondary ACE is used for FT communications. When the ACE cannot establish the FT TL connection after a period of several minutes, it stops attempting to reestablish this connection. This results in the ACEs failing to achieve the proper fully redundant state of ACTIVE/STANDBY_HOT.

Workaround: You can use the show conn command in the Admin context to determine whether the TCP connection between the addresses on the FT VLAN exists. If the TCP connection does not exist, enter the shutdown and no shutdown command on the FT VLAN to cause the ACE to attempt to reestablish the TL TCP connection.

If this does not work, investigate why the TCP connection could not be established. After the underlying issue, such as external network interruption which caused the TCP connection to fail, has been resolved, you can retry the workaround.

During the period when the TL connection cannot be established, you can expect delays in the response from some of the FT show commands due to the ACE primarily spending resources in attempts to bring up the TL connection.

CSCtg30362—When you apply or modify ACLs or object groups to an ACE that has operated for a long time and undergone many ACL configuration changes, issues in the ACL object group expansion during the configuration download may cause an unexpected traffic drop. The show interface command displays a non-zero download failure counter, similar to the following:

Access-group download failures : 8
 
   

Workaround: Remove and readd the object group.

CSCtg31251—When you change the cookie name of an HTTP-cookie sticky group, the change does not take place until you remove cookie group and then reapply it to the policy map. Workaround: Remove the cookie group and reapply it to the policy map.

CSCtg40070—When you configure DHCP-related changes on the VLAN and BVI interfaces, the ACE becomes unresponsive due to a cfgmgr termination. Workaround: None.

CSCtg41165—When you disable normalization for remote real servers, the ACE may resend a TCP RST for the probe. For a probe TCP SYN, the remote real server responds with a TCP RST. The ACE may respond with a TCP RST in the case where the real servers are reachable through a gateway and you have disabled normalization on the source interface for probe traffic. This issue may occur only if the remote server is another ACE and the VIP is out of service. Workaround: Perform either of the following:

Enable normalization on the source interface for probe traffic.

Configure the following ACL command on the ACE for every remote ACE VIP and port for probing from the ACE:

access-list test1_acl line 1 extended deny tcp host RSERVER_VIP_IP eq 
RSERVER_VIP_PORT host ACE_INTERFACE_IP 
 
   

CSCtg47919—When you configure long and complex regular expressions in new or existing commands, the ACE does not allow you to make any additional changes and may become unresponsive for a long duration of time. Workaround: Shorten the regular expressions in the commands.

CSCtg52570—When the ACE core directory contains the core.number, core_server_log.number.gz and core_client_log.number.gz files, the ACE module repeatedly reports the following message:

03:54:55 : %ACE-2-443001: System experienced fatal failure.Service name:System Manager
(core-server)(8119) has terminated on receiving signal 11,system will not be reloaded
 
   

Workaround: Reboot the ACE.

CSCtg52956—When you enter the show probe detail command from a user context, this command may display invalid output and may generate a vsh core file in the core: directory. Workaround: None.

CSCtg54739—When the ACE receives multiple show commands with XML queries, the ACE httpd service becomes unresponsive. Workaround: Restart the httpd service manually by using the debug plugin.

CSCtg66873—When the snmpd service becomes unresponsive on an ACE running software version A2(2.3), the ACE generates a core dump and reboots. Workaround: None.

CSCtg70654—When the ACE has different inbound flows that have a common outbound flow, it sends the RADIUS accounting response from the wrong VIP address. Workaround: Force source NAT upstream to avoid this situation in which inbound flows share a common outbound flow.

CSCtg73822—If you press Ctrl-C after entering the show np 1 | 2 me-stats command, you may observe in the output of the show buffer usage command that the IPCP buffer count starts to increment. Within a few minutes, the ACE module may reboot without a core file (silent reboot). Workaround: None.

CSCtg73931—When the SSL part of the ACE module is faulty and is not detected, the module may boot and run. Workaround: Complete the RMA process for the ACE module.

CSCtg85460—The ACE divides the sticky table and cookies between its two IXP network processors (NPs). If a connection on one NP uses a cookie with a hash that resolves to the other NP, the NPs must perform additional inter-IXP messaging to process the cookie. In a default TCP connection configuration, if the server sends 32K or more of data in less than 10 milliseconds (msec), a zero window may result on the backend. Some server TCP stacks may inadvertently introduce a 5-second delay in this situation. The ACE should advertise a non-zero window to the sending server when the buffers are released. Workaround: You can configure the set tcp wan-optimization rtt 0 command to apply TCP optimizations to packets for the life of a connection. However, this command results in increased resource consumption.

CSCtg90164—When you delete a configuration associated with the inspect icmp error command in following order, Service policy > Policy map >Class map >Access list, the ACE displays the following error message:

Error: Cannot delete this object as this is referenced by inspect action.
 
   

Workaround: Delete the configuration associated to the inspect icmp error command in the following order, Service policy > the inspect icmp error command > Policy map > Class map > Access list.

CSCtg94254—When you assign VLANs to the ACE module in a Cisco Catalyst SUP-2T VSS configuration, error messages flood the supervisor console. Workaround: None.

CSCth03644—When you use the send-data command in a TCP probe and the server sends a FIN ACK after it receives the data from the ACE, the ACE incorrectly assumes it receives a non-FIN packet containing no data and displays the following error message:

Last disconnect err : Unrecognized or invalid response
 
   

Workaround: Do not use the send-data command in a TCP probe.

CSCth07695—When you include any address with the show np [1|2] me-stats "-N 0x....." command, it calls an internal function with an invalid address that may cause the ACE to reboot. CSCtg73822 fixed the rebooting of the ACE. Workaround: None.

CSCth21796—When you place interfaces up and down several times or configure several interfaces or static routes, some interfaces or static routes may not work properly and connectivity to peers can be lost. Workaround: None.

CSCth23239—When you configure the first VIP as 0.0.0.0/0, all subsequent VIPs displayed by the show cfgmgr internal table vip command match 0.0.0.0/0. Also, during the ACE runtime, the ACE may enter this state for reasons currently unknown. Workaround: Perform either of the following:

Reconfigure the multi-match policy to push the catch-all VIP down to the bottom. You may need to reboot the ACE for this to take effect and reorder the VIP table in the show cfgmgr internal table vip command.

Change the KALAP configuration to KALAP by tag which is unaffected by this defect.

CSCth24080—When you enter the show logging message all command on the ACE module, it displays a log message for the ACE appliance. Workaround: None.

CSCth32236—When you register an organization name containing an & with a certificate authority (CA) and attempt to create a certificate signing request (CSR) on the ACE with this organization name, the ACE displays the following error message:

Error: Organization name cannot be composed of 
these special characters <>~!@#$%^*\&
 
   

Workaround: Use an external tool (for example, OpenSSL) to generate a CSR or request the CA to generate the key pair and certificate for the ACE.

CSCth34301—When you run tests on the ACE for redundancy and the SSL throughput is above the maximum for 6 to 7 hours, the ACE module unexpectedly reboots. Workaround: None.

CSCth41169—When the ACE is running software version A2(2.4) and load balancing RADIUS traffic, multiple client-side connections simultaneously associated with a single server-side connection cause a leak in which the connections on the NP are not reclaimed. This event causes the consumption and hitting of 2 million internal maximum connections per real server. After a period of time, traffic is dropped. Workaround: Reboot the ACE when the connections per real server is 1.5 million per NP.

CSCth63508—When you run a SIP test that initiates 50 calls per second until there are 10,000 calls, wait a few minutes for the release of all ACE resources after the test completes, and repeat the test seven or eight times, call timeout and drop failures occur. After you reboot the ACE, the SIP test complete without failure. Workaround: Reboot the ACE.

CSCth80330—When the ACE modules in a redundant configuration are running different software versions and you edit the probe configuration on the active ACE, the standby ACE reboots with a Service "hm" last boot reason. Workaround: None.

Software Version A2(3.2) Open Caveats

The following open caveats apply to software version A2(3.2):

CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, and then try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in an RSA format instead of an RSA1 format. For example, enter the following command:

host1/Admin# ssh key rsa 1024 force
 
   

CSCso76154—When performing configuration rollback, existing classes in a policy are not re-ordered according to the new configuration. The running configuration has a policy that contains several classes. The checkpoint contains that policy with some or all of the classes in a different order. After performing the rollback, the order of the classes stays as it was in the running config. Workaround: Two possible workarounds exist: 1. Erase the policy that is being changed during the rollback and then perform the rollback. 2. If there are many such policies, perform a rollback to an empty configuration and then rollback to the wanted configuration.

CSCsr76812—When you configure the ACE with Layer 7 load balancing, TCP connection may be disrupted. Packets arrive at the client in reverse order or packets are forced to be resent. Workaround: None.

CSCsu22856— When you configure VIPs with sticky and the ACE resets new connection requests to these VIPs, the ACE displays the following show command output:

The show stats sticky command displays over 400,000 active sticky entries.

The show conn count command displays approximately 10,000 active connections.

The show sticky database detail command displays a large number of sticky entries, the active-conn-count field with 0, and the time-to-expire (secs) field with 0.

Workaround: Clear the sticky database in the affected context.

CSCsv80430—When you configure RBAC on an ACE with a custom role and domain, any permit rule allows all show commands to be entered regardless of the configured permissions. Workaround: None.

CSCsv82779—The ACE treats the deny function inside a management policy or class map as a SKIP. The ACE does not deny the traffic. Instead, it skips the class map and try to match another one. Workaround: None.

CSCsx13061—When you perform a checkpoint rollback in a specific order or execute a match and no match statement under a class map, ACL memory is leaked and some entries configured in the ACL are not removed from the interface. Workaround: Remove the interface and readd it or do not perform a rollback in the specific order mentioned in the steps to reproduce of the bug description.

CSCsx28587—When the maximum aclmerge instance limit of 8191 is reached and then freed, ACL merge will not occur. Also, after reaching the maximum limit of instances, if you remove the outbound ACL from the interface, the policy action nodes are not released. Workaround: None.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx55228—When you remove an entry with an object group from an ACL which is associated as global access group and then readd it, merge errors occur and nonallowed traffic goes through the ACE. Workaround: Unconfigure and then reconfigure the access group.

CSCsx62330—When SSL is configured in one or more contexts and a large number of certificates and keys (approximately 2000 or more) are configured on the ACE, HTTPS probes may fail if you reload the module. The ACE appears to send the HTTPS probes, but they are not successful. You will not see this problem if you do not reload the module after the configuration. Workaround: If possible, reduce the number of certificates and keys to below 2000, and then reload the ACE.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: None.

CSCsy23268—The ACE may send probe traffic with the source IP address of the alias IP address instead of the local interface IP address. This issue occurs on the active ACE only. Workaround: None.

CSCsy31553—When traffic traverses the ACE module with the same source and destination port and dynamic NAT for that traffic is enabled, the ACE performs an implicit PAT. This behavior interrupts some sessions. This problem does not occur when NAT is not involved. Workaround: If possible, disable dynamic NAT.

CSCsy91540—When the supervisor engine detects that the ACE is not responding to keepalives, the ACE may silently reboot and not generate core dump files. Workaround: None.

CSCsy98701—When you configure two ACEs as FT pairs that are replicating sticky entries and enter show commands on the active ACE, the standby ACE generates a load-balancing core file. Workaround: None.

CSCsz19782—When you convert the configuration from a non-full proxy to a full proxy configuration for full proxied new connections and you add new VIPs for load balancing, traffic to these VIPs do not go through the ACE. Workaround: Reboot the ACE.

CSCsz22742—When you copy a large configuration to the running-configuration file, an API timeout error may occur. Workaround: None.

CSCsz37412—When the software and license on the ACE are compatible, ANM does not display their compatibility status. The XML show ft peer 1 detail command on the ACE is not correct. Workaround: None.

CSCsz62556—When you apply connection limits by entering the conn-limit command at the real-server level and connection limits are already applied at the server-farm level, some real servers may become stuck in the stopped list forever and not perform load balancing. Workaround: Reload the ACE.

CSCsz67761—When a network error, such as a network interface going down, occurs during the bulk importing of crypto files, the temporary storage space for imported crypto files is not gracefully released. Some of the temporary files remain in the temporary storage area until the system is reloaded. Bulk import procedures currently do not perceive network failures or inactivity if the transfer of the files has begun. Workaround: None.

CSCsz85367—When you configure and unconfigure access lists in a loop, the ACE experiences a memory leak. Workaround: Do not configure and unconfigure access lists in a loop.

CSCsz92540—If the configuration contains inline match statements under a policy map, the check point rollback fails. For example:

policy-map type inspect http all-match http-match

match test strict-http

reset

Workaround: Remove all the inline match statements before doing the checkpoint rollback.

CSCta13446—When you remove and then reapply the inspect ftp command, the ACE drops connections. Workaround: None.

CSCta39372—When you perform repetitive checkpoint rollbacks, the ACE becomes unresponsive after 5 to 6 hours. Workaround: None.

CSCta49917—When Telnet connections, SSH connections, or a debug session are active for a long time on the ACE, they do not close properly as indicated by one of the following:

The MTS buffers increases after each changeto command as displayed by the show system internal mts buffers command.

The following error message occurs:

IPC queue full. Clear idle telnet/ssh connections or debug plugin sessions to 
recover err
 
   

Workaround: You can either Telnet to each context to make configuration changes or reboot the ACE.

CSCta73571—When you configure ft track for an interface that is constantly down and then attempt a checkpoint rollback from a large configuration to an empty configuration, the rollback ends prematurely, resulting in a partial rollback. The ACE, however, indicates that the rollback is complete. Workaround: Attempt the rollback once again. If it fails again, configure ft track with a greater difference between the active and standby priority settings.

CSCta92891—If you change the load-balance predictor from least conns to hash URL with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCta99792—When you are making configuration changes to an ACE that has 30 contexts with traffic running, the control plane configuration manager process may become unresponsive while it is processing a configuration download or configuration changes. Workaround: None.

CSCtb00726—If the VIP address conflicts with the shared interface address across contexts, the standby ACE goes into the cold state with the show ft config-error command displaying the following error message:

interface vlan number 
Error: Global Policy applied, conflicts with VIP, NAT or Interface IP in shared 
interface!
 
   

Workaround: Do not configure a VIP address with the same address as the shared interface IP address on which the service policy is configured.

CSCtb03138—If you configure SNMP traps on a VLAN that has either the IP address or the peer IP address missing and redundancy is enabled, then the active ACE does not synchronize the SNMP traps to the standby ACE. The show ft group detail command displays the following error:

Error "Incremental Sync Failure: snmp config sync to sby." 
 
   

Workaround: Configure both an IP address and a peer IP address on the interface VLAN that you are using as the trap source.

CSCtb21313—When you configure persistence rebalance in a configuration with two server farms containing the same real server with different port numbers and attached to two different Layer 7 policy maps, connections are dropped intermittently after a rebalance occurs to a different Layer 7 policy. Workaround: None.

CSCtb28077—When you add the nat dynamic pool id vlan vlan-id command to a Layer-3 rule (combination of Layer-3 policy map and Layer-3 class map), which already has one dynamic NAT pool configured. For example:

policy-map multi-match pm1
class vip1
nat dynamic 1 vlan 731
 
   

This configuration already contains one dynamic NAT statement. If you add another statement for NAT dynamic, that configuration will not be downloaded. Dynamic NAT configuration is not downloaded to Data Plane and dynamic NAT does not work. Workaround: Remove and add the service policy under the client interface.

CSCtb30178—If you configure a RADIUS client Layer 7 policy map and continuously send accounting On/Off packets for 12 hours, the system fails. Workaround: None.

CSCtb32537—The ip name-server command is seen in the standby mode even after removing it in active mode. This issue happens in redundant configuration. Workaround: None.

CSCtb55526—With HTTP and SMTP traffic flowing and approximately 140,000 concurrent connections, the ACE module may exhibit CP slowness and eventually reboot with no core dump files. Workaround: None.

CSCtb55845—When a Virtual Switching System is configured on two Catalyst 6500 series switches, active-active redundancy is configured on the two ACEs in separate chassis, and you run stateless UDP traffic through the ACEs, some connections may fail. A trace shows that the successful flows use the ACE virtual MAC as the destination and the unsuccessful flows use the physical interface MAC of the standby ACE. A display of the default route and the svclc RHI routes shows two entries for the VIP in question. If you enter the show ip route command, the preferred route is the standby interface instead of the alias IP address. Workaround: None.

CSCtb66309—When you add a set of hosts and later delete the same under the network type object group, you will observe a policy action nodes leak for an object group. Workaround: None.

CSCtb72635—When you run a script for the show tech detail command on an ACE that has 4000 BVI and 4000 VLAN interfaces configured, the ACE may become unresponsive. Workaround: None.

CSCtb77652—When you configure the failaction reassign or failaction across-interface command and you enter related show commands or attempt to ping from the ACE, the ICMP and ARP manager generates an rpc call failure message. Workaround: Do not configure the failaction reassign or failaction across-interface command.

CSCtb82146—When you configure a global service policy and add a new interface, the ACE drops packets to the existing interface for a short duration. Workaround: Add a service policy where you add the new interface if the configuration is dynamic.

CSCtc80207—If the ACL merge resources are almost exhausted and you add a configuration statement that places the resources over the limit, the ACE may drop traffic on the VLAN interface in which the configuration statement applies. Workaround: To restore service, remove the last configuration change that you made. To determine the current ACL merge resource status, enter the show np 1 access-list resource command in the Admin context and the show acl-merge merged-list vlan number in non-redundant command in the context or VLAN where you will apply the configuration change.

CSCtd27259—When the ACE is running cron with the logrotate to rotate the Apache and wtmp logs, the ACE reboots with /proc/meminfo corruption in the SNMP, SYSMGR, HM and SCRIPTED HM core files. Workaround: Disable cron, and the Apache and wtmp logs.

CSCtd94085—You may observe an MTS memory leak for an invalid or a nonexistent process or PID. For a Vshell process, the MTS message queue limit is limited to a maximum of 4096 messages. Beyond that limit, any new message (for example, a changeto command is being executed), will get dropped and the following warning message is displayed on the console:

Warning:- MTS queue is full for opcode "<opcode value>" sap "<sad_id>" pid "<pid>" 
clear idle debug plugin sessions or telnet/ssh connections to recover. 
 
   

Sometimes, the PID that is displayed here may be invalid (no real process associated with it). Workaround: None.

CSCte03073—ACE HTTPS probes fail when you configure them for an IIS server that is configured with the Accept client certificates option. Workaround: None.

CSCte26173—During periodic XML queries on ACE for show commands, such as show ft group status, the ACE places the bash core files in the core: directory. Some files are unpackaged and other files are mispackaged as VSH core files. Workaround: None.

CSCte81257—When you perform dynamic configurations of usernames in multiple contexts and enter the no username name command in a user context, the ACE module unexpectedly reboots and generates an SNMP core file. Workaround: None.

CSCtf33100—If two or more probes associated with the server farm are in the failed state, at least one probe is in the passed state and the fail-on-all configuration is removed, the real server remains in the OPERATIONAL state and is not moved to the PROBE-FAILED state. Workaround: None.

CSCtf36703—When the device undergoes stress or excess load, the performance of the generic protocol parsing, HTTP L7 loadbalancing enabled with SYN-COOKIE or HEADER-INSERT decreases by 7 to10 percent. Workaround: None.

CSCtf38995—After you reboot the ACE, you cannot remotely log in to the ACE using RADIUS authentication. Workaround: Perform a ping between the server and ACE before using authentication.

CSCtf39655—If you configure send-data option inside a finger probe with length greater than four characters, the probe fails. Workaround: Configure send-data length with less than four characters.

CSCtf43237—The show xlate command displays thousands of entries. However, the show resource usage command displays zero peak and zero current. Workaround: Reboot the ACE.

CSCtf44818—The ACE module occasionally displays the incorrect value in the Unicast bytes input counter for the interface. This issue can cause problems for SNMP tracking the traffic, which in turn displays ~50Gbps flowing through the ACE. Workaround: Configure the SNMP application to ignore the counter increases above a certain value.

CSCtg18442—When the ACE is running a software version earlier than software release A2(3.3) and configured for SSL termination, if an incoming SSL client sends a encrypted stream of traffic during the SSL handshake phase and the ACE decrypts it, the ACE may reboot. The reboot occurs in the ACE SSL Nitrox chip that performs the decryption. Workaround: None.

CSCtg22592—After you make a change to a large ACE configuration and enter show commands, the CLI becomes unresponsive for a period of time. In this case, the show processes cpu | include cfgmgr command displays one of the configuration manager (cfgmgr) processes consuming CPU resources. After you apply the configuration change, the cfgmgr CPU usage drops to zero, and the CLI becomes unresponsive. Workaround: Wait until the cfgmgr process completes its previous operation before entering the show command.

CSCtg23875—During a supervisor switchover in a VSS system that contains an ACE module in each Catalyst 6500 chassis, the ACE may reboot due to the AAA daemon. Workaround: None.

CSCtg35291—The ACE may unexpectedly reboot and display a crashinfo file only in the dir core: output. This is a kernel crash of the CP. Workaround: None.

CSCtg46241—During a high rate of SIP calls per second and during the initial processing of packets, if the SIP inspection engine encounters resource allocation failures such as memory allocation, object allocation, inspect config version mismatch failures, the ACE may reboot. Workaround: Disable the SIP inspection feature, if possible.

CSCtg70913—When users whose accounts have expired attempt to log in to the ACE through SSH or Telnet, they succeed. Workaround: None.

CSCtg76859—After inserting a module or enabling power to a module in the VSS while it is passing traffic, the ACE module fails to respond to an EARL Recovery and the ACE module becomes unresponsive. Also, this issue forces all ASICs to resynchronize and other modules in the chassis may become unresponsive. Workaround: None.

CSCtg83716—When you enable DNS inspection and a DNS response hits a PAT policy map, the ACE drops the response. Workaround: None.

CSCtg93332—When you configure the mac-address autogenerate command on the client VIP interface in bridge mode, traffic to VIP starts failing. Workaround: Delete the client side interface and readd it.

CSCtg94333—When you create 10 contexts and allocate 10 percent of the resources to each context, ACL merge-list creation fails and management traffic to the VLANs fail. When you enable the debug access-list merge errors command and add or delete VLANs, the ACE displays merge-list errors. Also, attempts to ping and Telnet for the management access fail. Workaround: Remove the resource allocation from the contexts.

CSCth02932—When you enter the show np 1 me-stats | memory | status or show tech-support commands in the user context, the ACE displays an error message. Workaround: Enter these commands in the Admin context.

CSCth15050—When you place a VIP in a Layer-3 policy map out of service, the ACE does not remove the VSERVER-related ARP entries from the ARP cache. Workaround: Clear ARP to clear all ARP entries.

CSCth21361—When the ACE receives HTTP traffic containing special characters in the cookie value, it does not properly parse the cookie. The ACE accepts a space inside the cookie value. However, a quoted string containing the comma (,) character inside the string may cause a parsing error. Based on RFC2068, special characters are not legal in the cookie value and are not allowed inside a quoted string. Refer to the following information from RFC2068:

token   = 1*<any CHAR except CTLs or tspecials>     
tspecials 				= "(" | ")" | "<" | ">" | "@"     
				| "," | ";" | ":" | "\" | <">    
				| "/" | "[" | "]" | "?" | "="    
				| "{" | "}" | SP | HT 
 
   

Workaround: Do not use special characters inside the cookie value.

CSCth30569—When you apply a large multi-context configuration, the arp_mgr service in ACE becomes unresponsive. Workaround: None.

CSCth34050—Under normal operating conditions with logging enabled on the ACE module, the ACE unexpectedly reboots and generates a syslogd crash file. Workaround: None.

CSCth36358—Under normal operating conditions, the ACE unexpectedly reboots and generates a Linux kernel crashinfo file. Workaround: None.

CSCth41583—When the ACE receives a cookie string that contains many cookies and encounters a space character in the cookie value, it stops processing the cookies. Spaces are not permitted in the cookie name or cookie value. Persistence or stickiness fail. Workaround: None.

CSCth52830—The supervisor reboots the ACE module due to a diagnostic failure. he last boot reason on the ACE is unknown and the ACE does not generate core files. The sup logs indicate the following:

date_time UTC: %LINK-5-CHANGED: Interface TenGigabitEthernet2/1, changed state  to 
administratively down  
date_time UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface  TenGigabitEthernet2/1, 
changed state to down 
date_time UTC: %OIR-SP-3-PWRCYCLE: Card in module 2, is being power-cycled off  
(Diagnostic Failure) 
date_time UTC: %LINK-SP-5-CHANGED: Interface TenGigabitEthernet2/1, changed  state to 
administratively down 
date_time UTC: %C6KPWR-SP-4-DISABLED: power to module in slot 2 set off  (Diagnostic 
Failure)  
date_time UTC: %SNMP-5-MODULETRAP: Module 2 [Down] Trap  
date_time UTC: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface  
TenGigabitEthernet2/1, changed state to down  
date_time UTC: %DIAG-SP-3-INVALID_TEST: Invalid test: TestRwEngineOverSubscription 
date_time UTC: SP:  TestRwEngineOverSubscription is not valid for Module 2 
 
   

Workaround: None.

CSCth63772—When the ACE is using SSL and is oversubscribed beyond capacity, it reboots unexpectedly and generates a core file. Workaround: None.

CSCth66757—When you configure many servers with active/active NIC teaming, the ACE arp_mgr service may consume 100% of the CPU due to the ARP flood caused by teaming mode. Workaround: Reduce ARP traffic. Always use active/standby NIC teaming.

CSCth69747—When you manage the ACE with SNMPv3 in CiscoWorks LAN Management Solution (LMS)/CiscoWorks Resource Manager Essentials (RME), the ACE intermittently reports false usmStatsUnknownUserNames (1.3.6.1.6.3.15.1.1.3.0) during LMS/RME inventory collection, and the RME inventory collection may fail occasionally. Workaround: Manage the ACE with SNMPv2 in LMS/RME

CSCth69782—When you configure a VIP on the ACE, the ARP entry is inconsistent but the connections are working. Workaround: None.

CSCth73392—When you configure the ACE with SSL termination and the CP crash detection process detects that the Nitrox-II chip is unresponsive, the ACE reboots and generates a nitrox_core.tar.gz core file in the core: directory. This occurrence was a one-time event. Workaround: None.

CSCth74249—When the ACE is using SSL client authentication and is oversubscribed beyond capacity, HTTPS probes fail even after traffic has failed over to the standby ACE. The connections become stuck. Workaround: Do not allow the ACE to be oversubscribed. Clear all of the connections and allow the connections to continue.

CSCth75242—When you configure the ACE module with SSL probes and these probes intermittently fail, the VIPs go down and a network outage occurs. Workaround: Reboot the ACE.

CSCth75674—When the SCP HW watchdog on the ACE fails to detect the timer expiry, the Catalyst 6500 supervisor power cycles the ACE with an SCP keepalive failure message. The watchdog may fail to detect the timer expiry when the internal counters overflow. The SCP HW watchdog mechanism detects when the ACE becomes unresponsive and collects the core files in error case scenarios which prevents the power cycling by the Catalyst 6500 supervisor. On very rare occasions, the watchdog may fail to detect the timer expiry. Workaround: None.

CSCth75707—If the client or server retransmits a packet and the remote end exceeds the acceptable window size, the ACE incorrectly drops the retransmission packet and increments the [Drops] fp TCP window left edge counter. Workaround: Disable normalization or correct the client or server to honor the window sizes.

CSCth77963—When you upgrade ACE to software version A2(2.4), the ACE logs the following message after the reboot message:

%ACE-4-901001 kernel: Cannot find mapfile.
 
   

Workaround: None.

CSCth80972—On a rare occasion, the ACE reboots and generates a crashinfo file. Workaround: None.

CSCth85288—When a Layer-2 connected real server is in the ARP_FAILED state and you shutdown or delete the corresponding Layer-3 interface, the real server state does not transition from the ARP_FAILED state to OPERATIONAL or the probe failed state. Workaround: Remove the real server and reconfigure it.

CSCth94715—When you configure multiple contexts in an FT configuration and configure probes for each context but you configure one context with an FT track probe, if you remove these contexts from the FT configuration and delete them, health monitoring may become unresponsive. Workaround: None.

CSCth99982—When you configure an ECHO TCP probe with send-data and regular expression (regex) values, the probe always passes even if the server sends a regex that does not match the sent-data value. Workaround: You can use a TCP probe with send-data and regex values as required instead of an ECHO TCP probe.

CSCti03626—When you apply the same NAT pool to multiple VLAN interfaces, the show ip route command displays the pool of NAT addresses for only one VLAN interface. Workaround: None.

CSCti08045—Intermittently, a race condition can occur when the ACE is using the same VIP to listen on two different ports with persistence rebalance that is also load balancing to the same real server with port redirection on the backend using the same port. The ACE resets the connection.

For example, the ACE has the following configuration:

vip 192.168.100.20: 443
vip 192.168.100.20: 80
rserver 10.10.10.20: 81
 
   

The first connection enters on port 443. The ACE creates a second connection on port 80 while the first connections is still open. When the ACE attempts to set up the outbound flow for port 80, the race condition can occur. The ACE sees that the second flow has a redundant connection causing it to drop the flow. You can see the Drop [redundant connection]: counter increment in -socm. Workaround: Make sure that the VIP real servers are listening on different ports, for example:

VIP 192.168.100.20:443, real server10.10.10.20:81

VIP 192.168.100.20:80, real server 10.10.20:82

Do not have more than one VIP redirecting to the same port number on the same server on the backend. You could also use different real servers for each VIP port pair.

CSCti13494—When the ACE load balances clients towards the HTTP proxies, the ACE resets proxied SSL connection; an RST on the Client Hello. This issue may be associated with HTTP/1.1 in the CONNECT request or response. Workaround: You can configure HTTP/1.0 on the client and server. Do not inspect the HTTP connections

CSCti13660—When the ACE has high CPU usage in the SME processor with an PCI TX Q full error and attempts to generate QNX core dump, it may become unresponsive. Workaround: None:

CSCti15939—When you configure PAT and heavy traffic flows on the ACE, the show xlate command displays the Got no reply message. Workaround: None.

CSCti18687—When you perform various levels of end-to-end or back-end SSL traffic performance tests on the ACE, SSL-initiation and HTTPS probe connections remain in the CLSRST state until the inactivity timeout clears them. Workaround: Decrease the inactivity timeout or manually close the connections through the CLI commands.

CSCti27209—The sample keys and certificate feature is now available in the Admin context only. If you upgrade from software version A2(3.0) or A2(3.1) to A2(3.2), the sample certificate and key for the existing context remain as they are. However, the sample certificate and key are not be seen in new contexts. If you upgrade from software version A2(2.X) or earlier, sample certificates and keys are present in the Admin context only. You can generate a key named cisco-sample-key but cannot delete it. Workaround: None.

Command Changes in Software Version A2(3.2)

Table 10 lists the command changes in software version A2(3.2).

Table 10 CLI Command Changes in Version A2(3.2)  

Mode
Command and Syntax
Description

Exec

checkpoint create

Per CSCtb95036, when you create checkpoints, avoid using opening braces, closing braces, whitespaces, or any of the following symbols: `$&*()\|;'"<>/?

Exec

show acl-merge statistics

The new statistics option displays the compiler statistics of acl-merge

Exec

show buffer usage

The command now displays the Hi watermark field which allows more visibility for buffer usage when monitoring high watermarks.

Exec

show ft group status

The output of this command now displays the following fields for all FT groups that are configured in the ACE:

Running cfg sync status

Startup cfg sync status

Exec

show ft group brief

The output of this command now displays the Running Cfg Sync Status field for all FT groups that are configured in the ACE.

Exec

show ft group detail

Per CSCsl83506, the output of this command now displays the status of connection replication as enabled or disabled.

Exec

show parameter-map

Per CSCta24844, the output of this command now displays the reassembly timeout (seconds) field. This field displays the timeout configured through the set tcp reassembly-timeout command.

Exec

show probe name detail

Per CSCse36558, the FTP method and FTP filename options are removed from the configuration of an FTP probe.

Exec

show resource usage context all resource rate connections

Per CSCtf83851, the percentage for the maximum connection rate is now based on 325000. Previously, the rate was based on 1000000.

Exec

show sticky database

Per CSCtd75203, this command now displays the SRC and destination IP address instead of hash values for IP sticky.

Exec

show system internal mts sap_all

The new sap_all option displays dynamic SAPS which are used at any given point in time.

Exec

show system internal dmesg

The new dmegs option examines or controls the kernel ring.

Exec

show tech-support

Per CSCte78865, this command now displays the following:

The show system internal dmesg command output

Contents of kernel_log_message file

Exec

system [no] watchdog [lcp | memory | scp]

Per CSCtf31485, the system [no] watchdog command now enables or disables all watchdogs. Previously, this command enabled or disabled the SCP watchdog.

The options are as follows:

The lcp option enables or disables the LCP process. The current SCP watchdog watches this process. However, if the LCP process is not scheduled on time, this watchdog reboots the ACE.

The memory option enables or disables the low memory watchdog when ACE memory reaches 99 percent.

The scp option enables or disables the watchdog that monitors the SCP keepalive messages from the hardware timer interrupt level.

Note This command is intended for use by trained Cisco personnel for troubleshooting purposes only.

Configuration, Interface

arp ip_address mac_address

Per CSCsr19346, this command now allows the configuration of a multicast MAC address.

Configuration

ft connection-sync disable
no ft connection-sync disable

Per CSCsl83506, this new command disables or reenables connection replication in the current context. For more information, see the "Disabling Connection Replication" section.

Configuration

no limit-resource all

When you remove the limit-resource all command from a resource class, all the ACE contexts associated with the resource class will be left out of resources that are not separately defined.

Per CSCte92842, when you remove the limit-resource all command from a resource class and if you use a temporary license, the ACE displays the following warning message:

Warning: The context(s) associated with this 
resource-class will be denied of all the resources 
that are not explicitly configured with minimum 
limit in this resource-class

Configuration

regex compilation-timeout minutes

Per CSCtg47919, this new command allows you to configure the timeout for regular expression (regex) compilation. When you configure a regex and its compilation is longer than the configured timeout, the ACE stops the regex compilation. The minutes argument is the time period in minutes. Enter an integer from 1 to 500. The default timeout has been set to 60 minutes. This command is available only in the Admin context for an admin role and is applicable across all contexts.

For example, to configure a compilation timeout of 80 minutes, enter the following command:

host/Admin(config)# regex compilation-timeout 80

Configuration

switch-mode timeout seconds

Per CSCtf91257, the new timeout option allows you to configure the inactivity timeout for TCP or UPD connections in Switch mode.

The seconds argument is the time period in seconds for idle connections after which the ACE disconnects the connection. Enter an integer from 1 to 65535. By default, the timeout is 8100 seconds.

For example, to configure a timeout of 10 seconds, enter the following command:

host/Admin(config)# switch-mode timeout 10
 
        

For more information, see the "Accounting Logs Containing Sensitive Information" section.

Object group service

udp eq sip

Per CSCsm53617, this new command associates the SIP port number to the service object. Whenever you configure the udp eq 5060 command for the SIP port, the ACE internally changes it to "udp eq sip."

Parameter map connection

set tcp reassembly-timeout seconds

Per CSCta24844, the new reassembly-timeout option allows you to set the reassembly timeout value in seconds for a TCP connection.

For the seconds argument, enter a number from 1 to 255. The default value is 60 seconds.

Real server redirect

Server farm redirect

Server farm redirect real server

probe name
ip address address routed

Per CSCtg31161, the ACE now allows you to configure a probe command to a redirect server.

You can configure only probes with an IP address in routed mode under a redirect server, real and server farm. You cannot associate a scripted probe to a redirect server.

For more information, see the "Probing a Redirect Server" section.

Role

rule number {permit | deny} create feature exec-commands

Per CSCte52340, the feature exec-commands option now permits or denies access to the telnet command by the user on the ACE.

Previously, when you configured a rule to deny the exec-command feature from a user, the user still had access to the telnet command.

Server farm host predictor

autoadjust maxload

Per CSCtd04486, the default autoadjust setting for the least-loaded predictor now is average load. Previously, the default setting was maximum load.

The new autoadjust maxload command allows you to set the the least-loaded predictor to maximum load.

SSL proxy

key filename

Per CSCtf88100, you can now enter a key filename with a maximum length of 40 characters. Since software version A2(2.0), the maximum filename length is 39 characters.


System Log Messages

Software version A2(3.2) introduces the following new system log (syslog) messages.

251014

Error Message    %ACE-3-251014: Could not probe server IP_address on port port_number 
for number consecutive tries - Internal error.

Explanation    The health probe could not be sent because of an internal error. The probe is skipped.

Recommended Action    Remove and then readd the probe to the real server or server farm.

441003

Error Message    %ACE-5-441003: Serverfarm (serverfarm-name) failed in policy_map 
(policy-map name) --> class_map (class-map name) without backup. Number of 
failovers = count1, number of times back in service = count2

Explanation    This syslog is generated when server farm goes out of service and when there is no backup server farm.

Recommended Action    None required.

442007

Error Message    %ACE-4-442007: VIP in class: 'class-map name' changed state from 
Initial state to new-state

Explanation    This syslog is generated when the vserver state changes.

Recommended Action    None required.

751001

Error Message    %ACE-4-751001: Delay in message processing observed for <process_name> 
with pid <ppp>, message_id <mmm>, opcode <ooo>, src_sap <sss> ,dest_sap <ddd> 
<process_name> is the name of the process. Ex: syslogd, vacd. 
<ppp> is the process id. 
<ooo> is the MTS opcode that is at the top of the process's MTS queue 
<mmm> is the message Id corresponding to the MTS message, at the top of the 
process's MTS queue
<sss> is the SAP ID from where the message is originated. 
<ddd> is the SAP ID  of the process that is hung.

Explanation    A new log message is added. The new group is 751 (MTSMON_GROUP).

Recommended Action    Contact Cisco TAC.

901001

Error Message    %ACE-<severity depending on the printk serverity>-901001: kernel 
message.

Explanation    A new log message is added. The new group is 901.

Recommended Action    For severity 1 and 2 syslogs, contact the TAC.

Software Version A2(3.1) Resolved Caveats, Open Caveats, Command Changes, and Syslog Messages

This release note includes resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.0):

Software Version A2(3.1) Resolved Caveats

Software Version A2(3.1) Open Caveats

Command Changes in Software Version A2(3.1)

System Log Messages

Software Version A2(3.1) Resolved Caveats

The following resolved caveats apply to software version A2(3.1):

CSCse71077—When you configure multiple static routes for the same destination but only one route is reachable, the route table output for the show ip route and show ip fib commands displays that the ECMP flag is set for the unique route entries. This flag should be set only if more than one route for the prefix is in the routing table. Workaround: None.

CSCsi16267—When you include regex strings in a load-balancing or inspection configuration, the output of the show service-policy command does not provide a way to tell if the last regex compilation and download was successful. Workaround: Monitor the regex download status by enabling system logging (syslog) messages.

CSCsi61783—If you initially configure a real server as a Layer 2 real server, and then the interface goes down or is deleted from the configuration, the real server may transition to an ARP_FAILED state and remain in this state after it becomes a Layer 3 real server. Workaround: Reconfigure the real server.

CSCsk82966—Occasionally, when the allocation of the regex resource is out of memory, the regex deny counter displayed by the show resource usage command does not increment. Workaround: None.

CSCsm04626—If you create a user context with a name that is a substring (for example, CONTEXTA) of an existing user context name (for example, CONTEXTABC) and you enter the changeto ? command at the CLI, the substring context name does not appear in the list of user contexts. This issue is a CLI hinting problem and is cosmetic only. You can still enter the changeto CONTEXTA command successfully. Workaround: Do not create a user context whose name is a substring of an existing user context name.

CSCsm92045—When you configure server-farm NAT on the ACE and remove a policy map, the ACE does not remove the association between the interface and NAT. Workaround: To remove the association between the interface and NAT, first remove the Layer 3 rules and then remove the policy map.

CSCsr01570, CSCsy90965—The Set-Cookie: length is null. Changing the default class-map from a sticky server farm to none does not eliminate a cookie insertion. Workaround: Remove and then reenter the class class-default command.

CSCsu54652—When inspect dns is enabled, the ACE removes the checksum. However, DNS queries are still resolved and it does not effect the functionality. When inspect dns is disabled, you do not see this issue. Workaround: None.

CSCsu76777—When you configured context names that use special characters that are interpreted by the command shell (for example, semicolon, pipe, and so on) and you entered write memory all command, the command generated errors and the output showed the attempted execution of shell commands. Workaround: When you define a context name, avoid using white space or any of the following special characters: `$&*()\|;'"<>/?.

CSCsu88684, CSCsq27062—When you configure the ACE with a large number of contexts and enable redundancy, as traffic flows on the ACE, the ACE becomes unresponsive and displays the following messages on the console:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784

The ACE then reboots. Workaround: None.

CSCsu94371—When you remove a VIP from a policy map, the show cfgmgr internal table icmp-vip command continues to display the removed VIP. Workaround: Reboot the ACE.

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv54222—When an HTTP client sends pipelined requests, if the next request comes in the middle of the server response, the HTTP connection becomes unresponsive and data is missing on the web page. Workaround: Configure a connection parameter map with the set tcp wan-optimization rtt 0 command.

CSCsv92321, CSCsx25981—The ACE module reboots unexpectedly and writes a core file to disk. Workaround: None.

CSCsw22826—When you configure sticky on the ACE and the traffic generates dynamic sticky entries, if you change the configuration from a sticky to a nonsticky configuration through a rollback or manually, the old sticky entries remain. Workaround: Clear the sticky entries before changing a configuration to a nonsticky configuration.

CSCsw43177—If a real server becomes unresponsive, you may observe that the show rserver command indicates the real server status as ARP_FAILED and the show arp command displays the MAC address for the real server, but the MAC address status is displayed as LEARNED instead of RSERVER. Under these conditions, you can ping the real server from the ACE, but the real server is down for load balancing because of its ARP_FAILED state. This issue is seen only on the standby ACE and only when the ARP entry for a host has already been learned by the active ACE and has been synchronized to the standby ACE ARP cache and later the same host is configured as a real server. Workaround: Delete the real server and then reconfigure it.

CSCsw43425—When a backup server farm is in use the ACE needs to intimate the GSS to send new clients to another data center, while allowing the VIP to remain in a INSERVICE state. In this case the user has a redirect server farm configured as the backup server farm. When the primary server farm fails the backup redirects client requests to the other data center.

CSCsx05150—When using 2048-bit certificate and key pairs with block and export ciphers, a rehandshake lead to stuck connections. Workaround: Either use nonblock and nonexport ciphers or use certificate and key pairs that are less than 2048 bits.

CSCsx13853—When you specify TCP as the protocol in a global access list configured for DNS traffic, DNS inspection fails. Workaround: Specify only UDP as the protocol in the global access list configured for DNS traffic.

CSCsx19525—When you configure 1,000 SSL VIPs on the ACE and then you change the configuration on those VIPs, a buffer leak occurs as displayed by the show np 1 me-stats command "-scommon" output and traffic conditions. Workaround: Reboot the ACE and do not make configuration changes that affect those VIPs.

CSCsx34767—When you enter the changeto command or create or delete a context, you may observe an MTS memory leak. After a long time or after you enter many such CLI commands, the MTS buffer queue may become full, which may result in the failure of show or configuration commands, or, in some cases, a reload of the ACE module. Workaround: Clear any idle Telnet, SSH, or debug plugin sessions that are open in your ACE.

CSCsx81743—HSRP or other multicast control packets might be either lost for up to 10 seconds toward the CPU or flooded in case of a link flap, as observed in the following conditions:

The Catalyst 6500 series switch is running Cisco IOS release 12.2(33)SXH3a or 12.2(33)SXI.

The port channel spans multiple modules. This condition has been seen in a combination of WS-X6708-10GE and supervisor engine EtherChannel or WS-X6708-10GE and WS-X6708-10GE EtherChannel.

The Supervisor Engine 720 and Supervisor Engine 4 is in the Catalyst 6500 series switch chassis.

The port that is flapping is not a port on the supervisor.

The ACE is load balancing traffic in the chassis.

Workaround: None.

CSCsx83292—When MTU is configured on the client, the ACE drops Layer 4 class-default packets. Workaround: Remove the MTU configuration.

CSCsy01247—When the SSL proxy server comprises of chaingroup and authgroup with a large size CA or intermediate CA certificates, the SSL handshake fails and hence the connection resets. Workaround: Either use smaller size CA or intermediate CA certificates or do not use both chaingroup and authgroup in the SSL proxy.

CSCsy29181—If either of the DP processors is at MAXCONN, the ACE should show MAXCONN in the show commands. However, the ACE waits until both DP processors are at MAXCONN. This issue occurs when the cde-same-port-hash is configured. Workaround: None.

CSCsy34814—The syslog message 305010 includes the duration of the Xlate translation. However this duration is always equal to the Xlate idle timeout. Workaround: Use the timestamps in the creation and tear down of the Xlate connections to calculate the Xlate duration.

CSCsy54551—The show service-policy command displays the connection counts from the service policy but it does not display the Layer 3 rule in the service policy. Workaround: None.

CSCsy58843—When the ACE has a high rate of management traffic, it may become unresponsive due to an ARP failure. Workaround: None.

CSCsy65650—When the ACE reports the termination of TCP flows, it may have displayed incorrect values for the duration and amount of data transferred. This issue occurred with HTTP and connections that are terminated with TCP RST. Workaround: None. If accounting is needed and relies on this log, use another method.

CSCsy68974—When you configure the SYN cookie and FTP inspection features on the ACE, and the number of embryonic connections reach the threshold, the first FTP inspection connection may encounter a problem if the same connection issues more than one FTP GET request, causing the second FTP GET request to fail. This problem only applies to the first FTP inspection requests that trigger the SYN cookie feature. Subsequent FTP connections succeed as long as the SYN cookie feature is not triggered. Workaround: Disable the SYN cookie feature.

CSCsy88379—The TAC diagnostic script showtech generated large output due to the show xlate command. Workaround: None.

CSCsz09362—When pinging the ACE with small packets, the ACE inserts Ethernet padding into the ICMP data field of a request less than 18 bytes. Workaround: Use larger ICMP packets to stop the ACE from inserting the padding.

CSCsz09364—When you create a context with a name that includes a space and allocate an interface VLAN to it, if you either remove the configured context or issue the write memory command, the SSL process becomes unresponsive and the ACE reboots and displays the following message:

Service name:itasca_ssl(922) has terminated on receiving signal 11 
 
   

Workaround: Do not configure a context with space.

CSCsz10107—When you configured preempt and the Catalyst 6500 with an active ACE module reloaded, the ACE may not correctly replicate connections when it rebooted and became active again. Some connections may get dropped. Workaround: None. This issue does not occur when reloading only the ACE or if preempt is not configured.

CSCsz14634—The ACE had issues when you copied large configurations from TFTP to the running-configuration and used the snmp-server community command to add the public group Network-Monitor to a context when the command was not in the original configuration. Workaround: None.

CSCsz18739—The ACE reloaded while running software version A2(1.4) and RADIUS AAA was configured. Workaround: None.

CSCsz20325—If you attempt to remove a nonexisting inspection policy map and then attempt to remove a configured inspection policy map, the ACE displays an error and does not remove the policy map. Workaround: Reboot the ACE.

CSCsz21527—When you configure an SNMP V3 user with authentication and privacy options on the ACE and attempt to perform an snmpwalk with the authNoPriv option for the same user, the snmpwalk succeeds. Workaround: None.

CSCsz25000—When the ACE is running front-end SSL traffic, a memory leak occurs on both IXPs. This leak happens if the tcp-env information is very lossy and many drop packets in the network occur with duplicate packets and fragmentation. Workaround: None.

CSCsz27257—When you configure the ACE for SSL termination and a client sends multiple single-byte SSL records, the ACE advertises a zero TCP window when terminating the front-end SSL connection and subsequently does not open the window after the underlying data is processed. In some packet scenarios, the ACE does not open the TCP window after the server acknowledges the payload. Part of the scenario also involves the server advertising a zero window to the ACE in conjunction with the ACE advertising a zero window to the client. Workaround: None.

CSCsz28035—Accessing the qnx shell from the physical console port of either NP on an ACE puts you in a shell. If you type exit, the NP console hangs and becomes inaccessible. Workaround: None.

CSCsz29641—With back-end SSL traffic (SSL initiation), some connections may not be closed properly and may remain in CLSRST state for approximately one hour or until the TCP timeout interval expires. Front-end SSL (SSL termination) appears to work normally. Workaround: Enter the clear conn command in the context to clear the connections or wait for the TCP timeout to occur.

CSCsz31739—When the VIP is out of service and loadbalance icmp-reply is not configured, the virtual server entry still exists in the ARP cache. The ACE will respond to ARP requests sent for this VIP. Workaround: None.

CSCsz34011—After a series of reboots, both ACE modules lose their context configurations. If the active ACE halts and reloads, after it reboots it will read the first half of the startup-config, establish FT with the standby ACE (the new active), and synchronized the configuration to obtain the rest of the configurations from the other ACE. If the other ACE stops functioning, the active ACE will not have obtained the rest of the configurations, including context configurations. Context configurations may be lost, although they still exist in the startup-config. Workaround: None.

CSCsz34933—The ACE may send a reset with the sequence number zero for a probe configured with the connection term forced command. Workaround: Use the graceful termination no connection term command.

CSCsz37259—If the context name is 64 characters long, the cert and key are lost after a ACE reload. Workaround: Import the cert and the key again and reapply them to SSL proxy.

CSCsz40699—When you use the SLB-Admin, Server-Appln-Maintenance, or a custom role with a create feature server farm rule, you cannot bring real servers in or out of service under the server farm. Workaround: None. There are currently no workarounds using these specific roles. However, you can complete these tasks using the Admin role.

CSCsz49088—When you monitor the ACE CPU, you can only monitor it using an Admin role. The show processes cpu command is available only in the Admin role. The Network-Monitor role, which should have access to all show commands is unable to access the show processes cpu command. Configuring a new role on the ACE does not allow you to monitor the system feature. Therefore, only Admin users are able to run this command. Workaround: Run the show processes cpu command in an Admin role.

CSCsz50090—When you quickly remove a NAT pool and add a new one with more IP addresses and those addresses are not present in the ARP cache, the ACE does not respond to an ARP request sent for IP addresses in its NAT pool. Workaround: None.

CSCsz58417—When you configure any inline match statement in a policy map, the ACE becomes unresponsive for a few minutes and does not apply the configuration. Workaround: None.

CSCsz63457—When you add inspect RTSP under a Layer 4 policy map that is already configured with inspect RTSP, the ACE triggers a download configuration to the data plane. Workaround: None.

CSCsz68435—When the ACE has many concurrent SSL connections and high peak rates, the ACE becomes unresponsive under the SSL traffic load. Workaround: None.

CSCsz69433—When the FT transitions from non-redundant or active to standby, withdraw injected RHI routes from the SUP.

CSCsz82740—When you attempt to disable DHCP relay, the ACE fails to delete the ACL and displays the following error:

Failed to delete acl
 
   

Workaround: None.

CSCsz83033—When traffic on the ACE matches a Layer 7 rule, the DSCP/TOS bits set in the packets received from the server are not preserved. Workaround: None.

CSCsz84462—When you configure redundancy on the ACE and then add or delete interface VLANs in a loop or frequently, the active ACE becomes unresponsive and generates an IFMGR core file. Workaround: Do not add or delete VLAN or BVI interfaces in a loop or frequently.

CSCsz86630—DNS inspection may not work after you upgrade from software version A2(1.1) to a higher release. The problem occurs only for a percentage of responses and it builds over the time. The following errors appear in the output of the show np me-stats -sfixup command in the higher release:

+[Hash miss errors]

+[NAT app fixup response error]

Workaround: Disable DNS inspection and configure more aggressive timeouts (for example, 4 seconds) for UDP and port 53.

CSCsz92671—When you configure the ACE in bridged mode with a Layer 3 VIP, the ACE bridges relayed DHCP packets in bridged mode instead of load balancing these packets if they match a configured VIP. Workaround: None.

CSCta01789—When the ACE has a large configuration with multiple contexts, and each context has a unique route for the same destination with a different next hop, clearing and copying this configuration can cause the SE flag to be set incorrectly in the routing table. Workaround: None.

CSCta03202, CSCsz92427—When you remove and readd the inspect protocol command under a VIP class from a multi-match policy map, the following error occurs:

Error: This class doesn't have tcp protocol and a specific port
 
   

You cannot unconfigure inspection other than HTTP inspection from a policy map. Workaround: Remove the VIP class from the multi-match policy map and reconfigure it.

CSCta03825—When the UDP booster is configured, the ACE does not forward every first packet from a new client's DNS request to a real server on each network processor (NP). Two packets (one for each NP) are dropped for each session. Workaround: Disable the UDP booster.

CSCta06378—If a control plane process (for example, snmpd, sysmgr, hm, and scripted hm) encounters memory corruption of the /proc/meminfo data, the ACE may reboot and produce a core dump file. Memory corruption may occur with other processes or threads, too. Workaround: None.

CSCta08715—When you configure CSR fields with certain special characters on the ACE, the following error message occurs:

Error: Organization-unit name cannot be composed of these special characters.
 
   

Workaround: Use an external tool to generate a CSR (for example, OpenSSL) or ask the CA to generate a key pair and certificate for the ACE.

CSCta09574—When you configure TACACS on the ACE and a TACACS key with a comma (,) character and you reboot the ACE, you must enter the key again for TACACS to work properly. Workaround: Configure the TACACS key on the ACE and TACACS server without a comma character.

CSCta20756, CSCsx15558—When the ACE has over 120,000 concurrent SSL connections, it displays SSL connection rate denies, FastQ transmit back pressure, and SSL RX back pressure. Eventually, the ACE becomes unresponsive. Workaround: None.

CSCta23362—When the SSL traffic stresses the system, the backend SSL or many small SSL records in a tcp packet, an excessive number of particle buffers are allocated even when there are no connections in the system. This causes buffer threshold drops of new connections. Workaround: None.

CSCta25613—When using RADIUS load balancing, the ACE may become unresponsive and generate a loadBalance_g_ns core file. Workaround: None.

CSCta28624—When you configure the MTU in an interface to a value other than the default of 1,500, reuse and reproxy fail. When you configure the MTU in the client interface, SYN cookie fails. Workaround: Remove the MTU configured for the interface.

CSCta29049—When the UDP booster is enabled, the ACE drops the UDP packets that originate from the server. Workaround: Disable the UDP booster.

CSCta30959—When you configure redundancy on the ACE, configuration mode is enabled on the active ACE when the standby ACE is in the standby-configuration state. During standby-configuration synchronization, configuration mode is enabled for a short time and any command that you enter during that time is lost. Workaround: Do not enter or change any command during a bulk configuration synchronization.

CSCta41421—The ACE module may become unresponsive due to an internal error, but it does not reboot and it does not generate complete core files. Workaround: None.

CSCta43466—When you do not configure a real server in the server farm, the ACE does not generate the closing XML tag for the server farm detail output. Workaround: Configure a dummy real server on the server farm.

CSCta45580—When the ACE is unable to download a CRL because the CRL server is down, it does not always attempt another download when the CRL server returns to an online state. This condition occurs when more than 50 VIPs use the same SSL proxy with CRL applied. Workaround: Remove the CRL configuration and then configure it again.

CSCta47529—When you configure the ACE for DHCP relay on an interface, the ACE may route DHCP traffic that uses a nonbroadcast destination address without using the DHCP relay feature. Workaround: None.

CSCta53085—When you configure scripted probes on the ACE, if the disk is full and the ACE retrieves the exit_msg command from the script, occasionally the ACE reboots. Workaround: None.

CSCta56143—If the ACE reboots, the service-policy input command may be missing in some user context configurations. If you enable cfgmgr debugging, it is possible to see that this condition is due to:

(ctx:2)cm_is_dup_ipaddr_in_shrdvlan_priv : vip address x.x.x.x is already in use by 
shared interface vlan x
 
   

This problem occurs if a VIP address is duplicated in multiple contexts that have shared VLANs. Normally, when it applies a service policy, the ACE checks to see if the configured VIP (IP and ports) is already configured in other contexts and, if so, it does not allow you to apply the service policy:

ACE/context1(config-if)# service-policy input SP  Error: Cannot overlap vip or NAT 
address configured in a shared interface!
 
   

However, if a service policy is already applied and you add a class-map with a VIP to the policy map, this check is not performed anymore. In this case, you could have multiple contexts with duplicated VIPs. Workaround: Do not configure an incremental VIP in a class map, add it to a policy map, and apply it to an interface as a service policy.

CSCta57280—When you use the capture command to take packet captures on the ACE, some frames may be truncated. Workaround: None.

CSCta71906—When expired CRLs are in use and the expired-crl reject command is configured in an SSL parameter map, the SSL process on the ACE control plane may become unresponsive. Workaround: Do not reconfigure VIPs while traffic is flowing.

CSCta76782—If a client or a server certificate contains a multitiered chain, an SSL handshake may fail when the order of the certificates within the chain is altered. Workaround: Do not use chained certificates.

CSCta78220—When the ACE is under heavy load through XML connections to the local interface, the ACE can reboot without a core file, generate a kernel crash, or lock out management functions. This condition is due to over consumption of resources by XML of memory and CPU. Workaround: Disable XML access to the ACE or stop XML polling of the ACE from customer management stations.

CSCta83978—If you downloaded an unusually large number of best-effort CRLs from a server, the SSL process on the control plane may become unresponsive. Workaround: Do not use best-effort CRLs.

CSCta89560—When you configure a match statement for a called party with an invalid regex that has double quotation marks under a SIP inspection policy, the ACE may become unresponsive and generate a core dump file. Workaround: None.

CSCta92673—When SSL traffic is flowing and you reconfigured SSL proxies that contain authgroups, the ACE may leak memory in the control plane. The memory leak is directly proportional to the number of reconfigurations that you perform. Workaround: Avoiding reconfiguring an SSL proxy when an authgroup is applied to the proxy.

CSCta93957—If you upgrade a redundant ACE pair to software version A2(2.1), downgrade the standby to software version A2(1.4), and allow the pair to synchronize configurations, and then upgrade the standby again to A2(2.1), the standby ACE does not lock configuration mode, allowing you to make configuration mode changes. Workaround: Enable a bulk synchronization by entering the no ft auto-sync command followed by the ft auto-sync command on the active ACE.

CSCtb03844, CSCtb47541—When you configure the failaction reassign command in a server farm and all the real servers in the server farm are down, the ACE becomes unresponsive to most CLI commands and its CPU spikes up to 100 percent by the cfgmgr process. Workaround: Use the no failaction command to disable failaction reassign in the server farm.

CSCtb07772—When the ACE is reproxying, it drops server packets larger than the server advertised maximum segment size (MSS) which leads to the stalling and eventual timeout of the connection. Workaround: Configure a parameter map with the exceed-mss allow command.

CSCtb08318—When you configure the snmp-server unmask-community command in a non-Admin context on the active ACE, incremental synchronization does not synchronize this command on the standby ACE. Workaround: Perform bulk synchronization to the standby ACE. You can execute the no ft auto-sync running-config and ft auto-sync running-config commands on the active ACE whenever you are configuring or unconfiguring the snmp-server unmask-community command in a non-Admin context.

CSCtb08836—If the ACE is configured with cookie stickiness and persistence rebalance and a client switches cookies and then switches back mid-TCP stream, persistence rebalance works, but the sticky table is never updated when the connection closes. In this case, connections build up in the sticky database. Workaround: Perform the following steps:

a. Enter the clear sticky database command to clear the sticky database manually.

b. Add the timeout-activeconns command to the cookie sticky configuration.

CSCtb12976—When UDP fast age is configured and the ACE is running close to capacity, the ACE may become unresponsive. Workaround: Disable UDP fast age and/or use UDP booster, and set the UDP timeout to approximately 10 seconds.

CSCtb13426—After the ACE has run for a long time without a reboot or there is a lot of communication between the supervisor engine and the ACE, when you enter the show scp stats command, the TX bytes field displays a negative byte count in its output. Workaround: None.

CSCtb13438—When you enter the supervisor no power enable module slot_number command for the slot number of the standby ACE, the standby ACE asserts itself to be the active ACE before the shutdown and both ACEs become active. Workaround: None.

CSCtb15183—When you configure the ACE with an access list and then perform multiple dynamic configurations and the use of the resequence option on it, duplicate access-list line numbers may occur on the ACE, further resequence commands fail, and you can not add an object. Workaround: Reboot the ACE to clear this condition.

CSCtb16605—When you add the cookie secondary command to a sticky group after you assigned the group to a policy and an interface, this command has no effect. Workaround: Remove the policy and reconfigure it.

CSCtb23312—The ACE becomes unresponsive when its uptime reaches approximately 485 days. Workaround: Gracefully reboot the ACE before its uptime reaches 480 days.

CSCtb23798—If you configure a BVI interface and a VLAN interface in two different contexts with the same ID and apply a global policy in the context with the BVI, the configuration may fail with either of the following errors:

Error: Global Policy applied, conflicts with VIP, NAT or Interface IP in shared 
interface!
 
   
Error: Cannot overlap vip or NAT or interface address configured in a shared 
interface!
 
   

Workaround: None.

CSCtb25491—After modifying an access list and then resequencing it in quick succession, the following error message appears in the syslog file:

WARNING: Unknown error while processing access-group. Incomplete rule is currently 
applied on interface vlanXXXX. 
 
   

Workaround: Manually roll back to a previous access rule configuration on the interface. Do not issue resequence commands in quick succession. After you execute a command, reenter it with a different line number.

CSCtb27018—When you configure the ACE for SIP UDP, the ACE does not accept the SIP UDP probes requests because the source port of the 200 OK message from the server is different from the destination port of the OPTIONS method. Workaround: None.

CSCtb28897—If you repeatedly enter commands related to SNMP traps for the server farm or the username command on the ACE CLI, an MTS buffer can leak. Overtime, a shortage of MTS buffers can cause the ACE to be unresponsive to management commands. Workaround: Do not repeatedly enter commands related to SNMP traps for the server farm or username command from the CLI. Monitor the MTS buffers through the show system internal mts buffer details command. If you detect a leak, schedule a reboot of the ACE.

CSCtb29571—After you repeatedly configure and unconfigure DHCP in Admin and user contexts, the DHCP relay service may restart. Workaround: None.

CSCtb35900—When all of the ports for the first IP address in the NAT pool are used up, NAT pool exhaustion occurs and ACE-wide problems occur. Workaround: Configure a single NAT pool range, for example, nat-pool 5 10.147.2.11 10.147.2.14 netmask 255.255.255.255 pat.

CSCtb38297—When you configure the weighted leastconn configuration on the ACE, the ACE sends a majority of the traffic to a few of the real servers in a server farm and very little traffic to the other real servers. When the real servers are in a failed state (PROBE_FAILED) and configured with custom weights, a configuration download occurs.

Workaround: Perform one of the following:

Change any configuration on the affected server farm when all the real servers are operational. For example, enter the no inservice and inservice commands of any real server in the server farm.

Remove the weight configuration.

Remove the probe configuration and then make a configuration change when all real servers are operational. Readd the probe configuration after 30 seconds.

CSCtb38910—If you force the core of the syslogd process twice by entering the system internal snapshot service syslogd command two times, the control plane becomes unreachable (similar to CSCsz78275). Workaround: None.

CSCtb39287—During the bootup of an ACE that has multiple contexts with large configurations, some probe commands may time out due to an mts_recv error. The context may be in the STANDBY_COLD state after the reboot. This behavior occurs because the probe commands time out while the configuration manager is busy downloading a large configuration. Workaround: Manually reconfigure the probe commands that failed because of the above error.

CSCtb40872—With a large configuration that generates many ACL entries, ACL memory usage can increase and never return to the previous usage level even after you remove the configuration. Workaround: None.

CSCtb48429—When repeatedly logging into and out of the ACE, a memory leak occurs. Workaround: None.

CSCtb49907—If the ACE fails and the standby ACE becomes active, a gratuitous ARP on the standby ACE in bridge mode does not update the ARP table causing a probe failure. After the ARP entry times out, the standby ACE recovers. Workaround: None.

CSCtb60118—After you reboot the ACE, the SSH key for management connections is different from the SSH key prior to the reboot. When the SSH key is generated on an active ACE and synchronized to the standby ACE, the standby ACE does not properly store the new SSH key in NVRAM. Workaround: If you remove the SSH key, use the write memory command. After a key is generated, use the write memory command on the active and standby ACE prior to the reboot.

CSCtb65921—In a redundant configuration, the show conn count command or the show resource usage all | inc conc- command may show a disproportionately higher number of current connections on the standby ACE as compared with the active ACE. The show conn | inc CLS command on the standby may show many connections in the CLSRST state. This problem appears to be a race condition when short-lived connections end in RST. In this case, the connection remove directive from the active to the standby may arrive before the connection create directive. Workaround: None. However, you can reduce the number of connections waiting to time out by lowering the idle timeout parameter from the default of 60 minutes. A higher discrepancy rate in the connection count between the active and the standby may require that you configure a more aggressive idle timeout.

CSCtb68393—When you configure the ACE for LDAP authentication but incorrectly define an LDAP server, the ACE CLI becomes unresponsive if there are not enough MTS buffers for intrabox communication. Workaround: Remove the LDAP authentication configuration. Then, properly configure the LDAP server.

CSCtb69990—If a probe is a associated with a tracking host, the clear probe command has no effect. If a probe is associated with a server farm or a real server, the clear probe command works properly. Workaround: None.

CSCtb70103—When you apply an action list to a policy, you may receive the following configuration manager error:

Error: Error in creating link between SLB Policy and action-list.
 
   

Workaround: Delete and then recreate the context.

CSCtb70382—In a client/server configuration that uses window scaling (WS) and with the ACE performing FTP inspection, the ACE may not use window scaling on FTP connections, which causes packets sizes to be smaller than expected. Workaround: Do not allow WS options, which is the default, or specify the clear option.

CSCtb72972—If you enter a command with more than 2048 spaces at the CLI, one of the following three problems may occur:

The ACE may be become unresponsive

You may lose your Telnet session

The VSH process may become unresponsive

Workaround: Do not include more than 2000 characters of white space in the command line.

CSCtb87775—When timing out an incomplete TCP three-way handshake (SYN, SYN-ACK seen), the ACE sends a RST, ACK to the client, but only RST to the server. Workaround: Disabling normalization using the no normalization command may help in some cases.

CSCtb96594—The TAC diagnostic show tech details command output contains multiple instances of the same command when you enter it at the CLI. Workaround: None required.

CSCtb99452—The ACE may become unresponsive as a result of a kernel issue in the find process. Workaround: None.

CSCtc01581—When multiple VIPs share the same IP address on different ports and the loadbalance vip icmp-reply active command is configured, the VIPs stop replying to ICMP pings whenever any server farm changes state for any load-balancing policy map. A VIP will reply or not reply to an ICMP ping based on the latest (chronological) change of state of a server farm defined under any of the VIPs sharing the IP address. Workaround: Configure the loadbalance vip icmp-reply command without the active option.

CSCtc03638—If an ACE Module is configured for the same TACACS server in the Admin context and in a user context and you delete the TACACS server with the TACACS key in the Admin context, the server is incorrectly removed from the TACACS group in the user context, which causes TACACS authentication to fail. Workaround: Do not delete a TACACS server in the Admin context while the server is valid in the user context.

CSCtc11723—A user with the Network Monitor role cannot run some show commands. For example, show ft is not available. Workaround: Define a new role based on the feature and rights you want to assign.

CSCtc12917—New connections on an active ACE that was formerly a standby ACE may ignore their matching sticky database entries. The sticky entry is learned when the ACE is acting as a standby, then the context fails over to the active. The sticky entry must time out before it is refreshed with a new connection that matches the sticky entry. When this happens, the sticky entry is ignored instead of being consulted for the load-balancing decision. Configuring a long sticky timeout will increase the probability that a new connection will refresh the sticky entry prior to its timing out. For UDP connections in particular, short connection inactivity timeouts will also increase this probability. Workaround: Clear the offending connections and force the client to reinitiate its session.

CSCtc22808—If you enter the show crypto chaingroup name command in a user context at the command line interface (CLI), the ACE may become unresponsive and generate a core dump file. Workaround: Avoid using the show crypto chaingroup name command at the CLI.

CSCtc25043—When FTP inspection is enabled in bridged mode with a catch-all VIP (0.0.0.0), the ACE does not source NAT (SNAT) a passive FTP data connection. Workaround: Disable inspection or change to routed mode.

CSCtc25527—When redundancy is configured, the ACE may reboot and generate a core file for the ha_mgr. Workaround: None.

CSCtc36837—When a client sends traffic to a secondary IP on a BVI interface in the standby ACE (peer secondary IP under the BVI), the ACE may not process the traffic correctly if either of the following conditions exist:

The client knows the standby ACE MAC address, but the ACE has not learned the client MAC address.

You clear the MAC address table in the Catalyst 6500 series switch or the Cisco 7600 series router and enter the clear arp-cache interface vlan vlan_id command.

Workaround: Enter the clear mac-address-table dynamic vlan vlan_id on the supervisor engine and the clear arp no-refresh command on the standby ACE. Then, ping the client PC from the standby ACE.

CSCtc39615—If you configure a parameter map with the TCP window-scaling (WS) option, the ACE may use the wrong TCP WS option in the server-side TCP SYN when the client WS is greater than the configured WS on the ACE. Workaround: None.

CSCtc43641—While the ACE is processing an SRAM parity error in the buffer freelist, an me_dump process issue occurs, the ACE reboots, and the following files are seen using the dir core: command:

314320 Oct 4 00:09:33 2009 qnx_2_mecore_log.999.tar.gz

467552 Oct 4 00:09:19 2009 qnx_2_me_dump_g_ns_core_log.<pid>.tar.gz

38662 Oct 4 00:09:36 2009 ixp2_crash.txt

An SRAM parity error must occur to cause this me_dump process problem. Workaround: None. The ACE reboots and recovers on its own.

CSCtc46913—For all proxied connections, the ACE may send packets to a client with a maximum segment size (MSS) of 536 bytes regardless of the maximum transmit unit (MTU) that is configured on the client interface of the ACE. Such proxied connections including the following:

Layer 7 SSL

Layer 7 HTTP traffic with a chunked response

All Layer 7 connections using a connection parameter map with the set tcp wan-optimization rtt command set to 0


Note For a Layer 7 connection, the behavior remains as long as the connection is in the proxied state. When the ACE unproxies the connection, the behavior is not seen.


This behavior does not apply to the following traffic:

Layer 4 connections (for example, regular Layer 4 load balancing, IP stickiness, and so on)

L7 connections where proxy-unproxy occurs. When the ACE unproxies the connection, the behavior is not observed. However, the behavior is seen during the proxied state.

Workaround: Downgrade to software version A2(1.5a). No software workaround is available.

CSCtc52085—After a client sends a ClientHello message, the SSL hand shake may fail with a fatal alert internal error sent by the ACE. This behavior is intermittent and may occur under the following conditions:

1. An SSL service is configured with the session-cache timeout command (session reuse).

2. SSL connections are aborted by the client after the client sends a ClientHello message to the service in condition 1 and before an internal resource state is changed. This behavior puts the internal resource in an improper state. This error is very timing sensitive.

3. The next connection that uses the internal resource in the improper state fails with a fatal alert internal error. That connection does not have to go to the service in condition 1 to experience this error because the internal resource is shared by all the SSL services.

Workaround: None.

CSCtc55134—When persistence rebalance is configured on the ACE and an MTU that is lower than the default MTU is configured on the client interface, reproxied Layer 7 connections may not learn the MTU that is configured on the client interface. This behavior causes the ACE to send unfragmented packets to the fast path where the packets are dropped and the Drop: No fragmentation of L3 Encap field of the show np 1 me-stats "-s fp" command is incremented. This behavior occurs only for Layer 7 reproxied connections that hit the persistence rebalance configuration. For all other Layer 7 connections, including proxied-reproxied, fully proxied, and SSL, and all Layer 4 connections, this behavior is not seen. Workaround: Disable persistence rebalance or remove the client MTU configuration.

CSCtc58925—With SSL configured, the ACE module may become unresponsive with the following error message: NP 1 Failed : Nitrox Crash Detected. Workaround: None.

CSCtc60445—A rare environmental condition may cause the ACE network processor to become unresponsive due to reason "SRAM Parity Error". The memory address that is the source of the parity error is in a specific region of memory. This condition is present in releases 3.0(0)A2(1.6) and A2(2.2). Workaround: Reboot the ACE to clear the state. This reboot is accomplished automatically when the core dump file is created.

CSCtc76933—When you configure a policy-map of type generic and this policy is linked to an SSL proxy server, generic parsing over SSL fails in the middle of the connection. Workaround: Configure a connection parameter-map and assign it to the policy as follows:

parameter-map type connection StayProxy
  set tcp wan-optimization rtt 0 
 
   

CSCtc77029—When you configure a scripted probe that sends an XML request to the interface of the ACE (from another ACE) and executes the show service-policy command, the output of the show proc cpu command shows that the CPU of the control plane (CP) is almost always at approximately 90% usage and that the XML CP processes is consuming those cycles.

Workaround: Instead of sending an XML request, send a RAW request and turn XML output on before executing the show service-policy command as follows:

xml_cmd=<request_raw>xml-show on%0ashow service-policy</request_raw>
 
   

The resulting XML output will have an extra exec_command node in the response for the xml-show on command, but the show service-policy response will be the same as with the XML request.

CSCtc81556—When you configure SSL sessionID stickiness with generic protocol parsing, SSL connections may hang after the server sends the HELLO packet. Workaround: None.

CSCtc82817—When you configure the ACE in a Virtual Switching System (VSS) deployment, multicast OSPF is not bridged. Workaround: Install the active ACE in the same chassis as the active supervisor engine.

CSCtc95224—When you configure the sticky-serverfarm command under a radius policy map, the policy map entries are not cleared on radius response and the radius requests are not evenly load balanced due to false retransmission issues. Workaround: Configure a sticky server farm under the radius L7 policy map.

CSCtc96770—If RADIUS traffic is being sent or you enter the show conn rserver rserver_name command, the outstanding messages in the load-balancing queue build up over time, which causes the ACE to become unresponsive eventually. This issue is not seen with the show conn command. Workaround: Do not use the sh conn rserver command.

CSCtd00816—An industry-wide vulnerability exists in the Transport Layer Security (TLS) protocol that could impact any Cisco product that uses any version of TLS and SSL. The vulnerability exists in how the protocol handles session renegotiation and exposes users to a potential man-in-the-middle attack. This advisory is posted at: http://www.cisco.com/warp/public/707/cisco-sa-20091109-tls.shtml.

CSCtd03994—When the status of a real server probe goes up and down repeatedly due to lack of response from the server, a static cookie entry may be removed and never reinstated. In this case, the ACE uses roundrobin load balancing for the first HTTP GET request in connection with the cookie set instead of sending the request to the real server associated with the cookie. Workaround: Enter the no inservice command followed by the inservice command for the real server to reinstate the static cookie.

CSCtd19970—When you configure an ACE in a user context in a very large configuration with 10 contexts and multiple SSL certificates, the ACE may reboot and generate a CFGMGR core dump file. Workaround: None.

CSCtd27448—When SSL is configured in version A2(1.6a), RSA_WITH_AES_128_CBC_SHA and RSA_WITH_AES_256_CBC_SHA are configured and a rehandshake is performed, the ACE may reboot and generate SSL (Nitrox) core dump files. Workaround: Downgrade to the previous release.

CSCtd83467—When an ACE contains a key that is protected by a pass phrase or the crypto item contains both a cert and a key, backup or restore command fails. Workaround: None

CSCte09563—Resetting the admin password fails if the username is of the following format: For example, admin@01 or admin-. Workaround: Do not use the formats of this type.

CSCte91633—The number of management connections that the ACE used, as shown in the output of the show resources command slowly increases until there are none available. The configured probes fail at that point. Workaround: Remove all configured probes so that the management connections are not needed.

CSCtf45647—When you import the SSL files which are larger than 2048 bits, the ACE encounters the following error:

"Error: Key Size in the file exceeds system limit (2048 bits in length), not imported"
 
   

Workaround: Use SSL files that are less than 2048 bits.

Software Version A2(3.1) Open Caveats

The following open caveats apply to software version A2(3.1):

CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, and then try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in an RSA format instead of an RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.

CSCso76154—When performing configuration rollback, existing classes in a policy are not re-ordered according to the new configuration. The running configuration has a policy that contains several classes. The checkpoint contains that policy with some or all of the classes in a different order. After performing the rollback, the order of the classes stays as it was in the running config. Workaround: Two possible workarounds exist: 1. Erase the policy that is being changed during the rollback and then perform the rollback. 2. If there are many such policies, perform a rollback to an empty configuration and then rollback to the wanted configuration.

CSCsr76812—When you configure the ACE with Layer 7 load balancing, TCP connection may be disrupted. Packets arrive at the client in reverse order or packets are forced to be resent. Workaround: None.

CSCsu22856—If ACE resets new connections to VIP with sticky configured, you see the following output:

show stats sticky command shows over 400,000 active sticky entries

show conn count command shows about 10,000 active conns

show sticky database detail command shows a huge number of sticky entries with active-conn-count = 0, time-to-expire (secs) = 0.

Workaround: Clear the sticky database in the affected context.

CSCsv80430—When you configure RBAC on an ACE with a custom role and domain, any permit rule allows all show commands to be entered regardless of the configured permissions. Workaround: None

CSCsx13061—When you perform a checkpoint rollback in a specific order or execute a match and no match statement under a class map, ACL memory is leaked and some entries configured in the ACL are not removed from the interface. Workaround: Remove the interface and readd it or do not perform a rollback in the specific order mentioned in the steps to reproduce of the bug description.

CSCsx28587—When the maximum aclmerge instance limit of 8191 is reached and then freed, ACL merge will not occur. Also, after reaching the maximum limit of instances, if you remove the outbound ACL from the interface, the policy action nodes are not released. Workaround: None.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx55228—When you remove an entry with an object group from an ACL which is associated as global access group and then readd it, merge errors occur and nonallowed traffic goes through the ACE. Workaround: Unconfigure and then reconfigure the access group.

CSCsx62330—When SSL is configured in one or more contexts and a large number of certificates and keys (approximately 2000 or more) are configured on the ACE, HTTPS probes may fail if you reload the module. The ACE appears to send the HTTPS probes, but they are not successful. You will not see this problem if you do not reload the module after the configuration. Workaround: If possible, reduce the number of certificates and keys to below 2000, and then reload the ACE.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: None.

CSCsy23268—The ACE may send probe traffic with the source IP address of the alias IP address instead of the local interface IP address. This issue occurs on the active ACE only. Workaround: None.

CSCsy31553—When traffic traverses the ACE module with the same source and destination port and dynamic NAT for that traffic is enabled, the ACE performs an implicit PAT. This behavior will interrupt some sessions. This problem does not happens if NAT is not involved. Workaround: If possible, disable dynamic NAT.

CSCsy74228—When a connection gets stuck in CLSRST state, it will not disappear after the idle timeout and the clear conn all has no impact on it.

CSCsy91540—When the supervisor engine detects that the ACE is not responding to keepalives, the ACE may silently reboot and not generate core dump files. Workaround: None.

CSCsy94458—The output of the show resource usage command may show that bandwidth has been denied in the Admin context of the ACE. The counters indicate that bytes have been dropped prior to a configuration having completed, but the count does not increment thereafter. There is no adverse effect of these drops; it is a cosmetic issue only. This behavior occurs in the display for the Admin context only. Workaround: None.

CSCsy98701—The standby ACE generates a load-balancing core file when you configure two ACEs as FT pairs that are replicating sticky entries and you enter certain show commands on the active/master ACE. Workaround: None.

CSCsz14033—If you delete disk0 without the filename and you assign a filename on ACE, it deletes the whole disk0 directory rather than the file. If the directory is empty now and you enter a dummy filename, it deletes the disk0 directory; hence disk0 can not be used after that. The disk0 directory is lost and is not created until the next reload of the box. Workaround: Reload.

CSCsz19782—When you convert the configuration from a non-full proxy to a full proxy configuration for full proxied new connections and you add new VIPs for load balancing, traffic to these VIPs do not go through the ACE. Workaround: Reboot the ACE.

CSCsz22742—When you copy a large configuration to the running-configuration file, an API timeout error may occur. Workaround: None.

CSCsz54546—When a probe is successful, the output of the show probe detail command may display 0 in the Last status code field instead of the actual code. If the probe is failing, the Last status code field value will be correct. Workaround: None.

CSCsz62556—When you apply connection limits by entering the conn-limit command at the real-server level and connection limits are already applied at the server-farm level, some real servers may become stuck in the stopped list forever and not perform loadbalancing. Workaround: Reload the ACE.

CSCsz67761—When a network error, such as a network interface going down, occurs during the bulk importing of crypto files, the temporary storage space for imported crypto files is not gracefully released. Some of the temporary files remain in the temporary storage area until the system is reloaded. Bulk import procedures currently do not perceive network failures or inactivity if the transfer of the files has begun. Workaround: None.

CSCsz78275—The ACE control plane becomes unreachable using either Telnet or SSH and eventually the VIPs become unresponsive. Workaround: Reload the ACE.

CSCsz85367—When you configure and unconfigure access lists in a loop, the ACE experiences a memory leak. Workaround: Do not configure and unconfigure access lists in a loop.

CSCsz87249—The following log messages may appear sporadically in the ACE log:

"can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg"

"can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg"

These messages do not impact the operation of the ACE. The messages may be caused by more than one device that is accessing the ACE context through XML. Workaround: None.

CSCsz92540—If the configuration contains inline match statements under a policy map, the check point rollback fails. For example:

policy-map type inspect http all-match http-match

match test strict-http

reset

Workaround: Remove all the inline match statements before doing the checkpoint rollback.

CSCta13446—When you remove and then reapply the inspect ftp command, the ACE drops connections. Workaround: None.

CSCta39372—When you perform repetitive checkpoint rollbacks, the ACE becomes unresponsive after 5 to 6 hours. Workaround: None.

CSCta40378—When backend SSL connections fail handshake, or rehandshake, ACE closes the connection without acknowledging the FIN from the rserver. The rserver connections are left in a FINWAIT1 state, and must rely on the server OS to time out. Workaround: None.

CSCta49917—When Telnet connections, SSH connections, or a debug session are active for a long time on the ACE, they do not close properly as indicated by one of the following:

The MTS buffers increases after each changeto command as displayed by the show system internal mts buffers command.

The following error message occurs:

IPC queue full. Clear idle telnet/ssh connections or debug plugin sessions to recover err

Workaround: You can either Telnet to each context to make configuration changes or reboot the ACE.

CSCta73571—When you configure ft track for an interface that is constantly down and then attempt a checkpoint rollback from a large configuration to an empty configuration, the rollback ends prematurely, resulting in a partial rollback. The ACE, however, indicates that the rollback is complete. Workaround: Attempt the rollback once again. If it fails again, configure ft track with a greater difference between the active and standby priority settings.

CSCta92891—If you change the load-balance predictor from least conns to hash URL with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCta99792—When you are making configuration changes to an ACE that has 30 contexts with traffic running, the control plane configuration manager process may become unresponsive while it is processing a configuration download or configuration changes. Workaround: None.

CSCtb00726—If the VIP address conflicts with the shared interface address across contexts, the standby ACE goes into the cold state with the show ft config-error command displaying the following error message:

interface vlan number

Error: Global Policy applied, conflicts with VIP, NAT or Interface IP in shared

interface!

Workaround: Do not configure a VIP address with the same address as the shared interface IP address on which the service policy is configured.

CSCtb02056—When you configure the ACE with SSL certificates and keys in multiple contexts, the output of the show crypto certificate all command may become corrupted. Workaround: Use the show crypto certificate cert_name command instead of the show crypto certificate all command.

CSCtb03138—If you configure SNMP traps on a VLAN that has either the IP address or the peer IP address missing and redundancy is enabled, then the active ACE does not synchronize the SNMP traps to the standby ACE. The show ft group detail command displays the following error:

Error "Incremental Sync Failure: snmp config sync to sby." 
 
   

Workaround: Configure both an IP address and a peer IP address on the interface VLAN that you are using as the trap source.

CSCtb21313—When you configure persistence rebalance in a configuration with two server farms containing the same real server with different port numbers and attached to two different Layer 7 policy maps, connections are dropped intermittently after a rebalance occurs to a different Layer 7 policy. Workaround: None.

CSCtb28077—When you add nat dynamic <pool id> vlan <vlan-id> to an l3-rule (combination of l3 policy-map and l3 class-map), which already has one dynamic nat-pool configured. For example:

policy-map multi-match pm1
class vip1
nat dynamic 1 vlan 731
 
   

This configuration already contains one dynamic NAT statement. If you add another statement for NAT dynamic, that configuration will not be downloaded. Dynamic NAT configuration is not downloaded to Data Plane and dynamic NAT does not work. Workaround: Remove and add the service-policy under the client interface.

CSCtb30178—If you configure a RADIUS client Layer 7 policy map and continuously send accounting On/Off packets for 12 hours, the system fails. Workaround: None.

CSCtb32537—The ip name-server command is seen in the standby mode even after removing it in active mode. This issue happens in redundant configuration. Workaround: None.

CSCtb44729—When you configure the ACE for Layer 7 load balancing and a connection is closed before it is processed by the load balancer, the show conn command displays no connections but the show server farm command displays the current connection for the real server even after all traffic has stopped. Workaround: Remove the real server and readd it.

CSCtb55526—With HTTP and SMTP traffic flowing and approximately 140,000 concurrent connections, the ACE module may exhibit CP slowness and eventually reboot with no core dump files. Workaround: None.

CSCtb56199—The ACE may become unresponsive while it is applying a configuration to the network processor engines. The following message appears on the console: ERROR : DRV : PCI send failed! PCI RIngs in Use. Workaround: None.

CSCtb55845—When a Virtual Switching System is configured on two Catalyst 6500 series switches, active-active redundancy is configured on the two ACEs in separate chassis, and you run stateless UDP traffic through the ACEs, some connections may fail. A trace shows that the successful flows use the ACE virtual MAC as the destination and the unsuccessful flows use the physical interface MAC of the standby ACE. A display of the default route and the svclc RHI routes shows two entries for the VIP in question. If you enter the show ip route command, the preferred route is the standby interface instead of the alias IP address. Workaround: None.

CSCtb66309—When you add a set of hosts and later delete the same under the network type object group, you will observe a policy action nodes leak for an object group. Workaround: None.

CSCtb72635—When you run a script for the show tech detail command on an ACE that has 4000 BVI and 4000 VLAN interfaces configured, the ACE may become unresponsive. Workaround: None.

CSCtb82146—When the service policy is global and a new interface is added, ACE drops packets to the existing interface for a short duration. Workaround: User service policy at interface level, if the configuration is dynamic and where a new interface needs to be added.

CSCtb86697—When you modify a NAT pool under an interface configuration, the following error may be logged and can be displayed using the show logging command: "Sep 4 2009 12:34:03 ace/ace: %ACE-1-106028: WARNING: Unknown error while processing service-policy. Incomplete rule is currently applied on interface vlan953. Manual roll back to a previous access rule configuration on this interface is needed." You may also see Service download failures in the show interface command output. Workaround: Remove and then reapply the NAT pool configuration.

CSCtb95036—When you enter the checkpoint create command, it generates errors and the output shows attempted execution of shell commands. Workaround: When you create checkpoints avoid using opening braces, closing braces, white spaces, or any of the following symbols: `$&*()\|;'"<>/?

CSCtb95136—When a server sends a request to a client in an RTSP configuration, the ACE resets the RTSP connections. RTSP servers are supported only in an asymmetric client-server mode (required and recommended methods). Workaround: None.

CSCtb95153—After you apply configuration changes to a NAT pool, the ACE may become unresponsive because a network processor (NP) microengine (ME) became unresponsive on X_TO_ME. Workaround: None.

CSCtc12692—If VIP is manually placed out of service, for example, via no loadbalance vip inservice. VIP continues to respond to ARP requests after it goes out of service. Workaround: Delete the interface where service-policy is applied and then reconfigure the interface and attach the service-policy.

CSCtc20009—When you configure ACE to send SNMP traps, specifically server farm traps, the server farm is applied to a service policy and the server farm changes state, the duplicates are seen. For example, if the server farm is applied to more than one class under a policy multi-match we will see duplicates for each. If it is applied to two class maps we see four traps, if applied to three class maps we see six traps and so on. Workaround: None other than turning off the traps.

CSCtc54698—When you login into the ACE, the following is observed:

a) Syslog is not generated if server farm goes down.

b) If VIP goes to OUTOFSERVICE there is no syslog generated for this event also. Workaround: None.

CSCtc77380—When you use the management protocol XML to query the ACE for context configuration, the ACE generates a invalid XML output for the show context command when issued in a user configured context. Workaround: The XML output for show context appears correct from the Admin context.

CSCtc80207—If ACL merge resources are close to exhaustion and you add a configuration statement that pushes the ACE over the limit, the ACE may drop traffic on the VLAN interface to which the configuration statement applies. Workaround: To restore service, reverse the last configuration change that you made. To determine your current ACL merge resource status, enter the show np 1 access-list resource command in the Admin context and the show acl-merge merged-list vlan number in non-redundant command in the context or VLAN to which your configuration change applies.

CSCtc87588—When TACACS+ is configured, the ACE does not account for configuration mode commands that contain sensitive information (for example, keys and passwords). Such commands do not appear in the local ACE accounting log nor in the TACACS server accounting log. In the ACE accounting log, there are descriptive entries, (for example, "deleted user"). In the supervisor engine accounting log, the commands are accounted for, but the sensitive information is masked. Workaround: None.

CSCtc89245—When you copy a file with the exact same filename as the previously deleted file name on ACE with different lower or upper case characters, the ACE picks the initial file name.

Workaround: 1. In this case you can rename the image to a new name and make sure we have correct bootvar on both. For example:

ACE1:

copy image:c6ace-t1K9-mz.A2_1_0a.bin image:c6ace.A2_1_0a.bin

ACE2:

copy image:c6ace-t1k9-mz.A2_1_0a.bin image:c6ace.A2_1_0a.bin

CSCtc94802When it is performing SSL URL rewrite for a hostname that matches XXXXX.cisco.XXXXX (X = anything), if we use a ".*\.cisco\..*" regex for this, the ACE is rewrites the URL to HTTPS, but it also adds "/" (forward slash) at the end of the URL. SSL URL rewrite with that needs to mach XXXXX.cisco.XXXXX. Workaround: Use the alternative regex ".*[.]cisco[.].*"

CSCtc91087—A configuration change in the limit-resource all minimum command value may cause the ACE to start rate-limiting traffic at a different throughput level than that indicated by the show resource usage command. Workaround: None.

CSCtc94844—When cookie insert and failaction purge are configured and the probe status is going up and down repeatedly, the show serverfarm detail command may display a current connections counter that is not accurate (not null when it should be). Workaround: None.

CSCtd04486—When you are using an SNMP probe for the least-loaded server farm predictor and the OID value returned by the probe from the real server is 0 (the server is least loaded), that real server may not receive any connections and the ACE distributes all the connections to the other servers in the server farm. Workaround: Change the predictor autoadjust value from the default of max to average. The ACE will autoadjust the load to be the average load of the server farm and the real server will get connections based on its having the average load of the server farm.

CSCtd22008—When you perform a end-to-end SSL, for example, show serverfarm <name> or show rserver <name>, the counter increments when a client sends a RST-ACK after the connection has already been established or when a client sends a RST-ACK to a Rserver FIN. Workaround: None.

CSCtd25891—The ACE may be slow to respond to CLI commands. This behavior has been observed with an MTS buffer leak that can be seen with the show system internal mts buffer command for opcode 4001. Workaround: None.

CSCtd40797—When you use KAL-AP with GSS and active or standby redundant ACE modules, the GSS reports an invalid answer state if the ACE VIP fails on the active ACE but not on the standby ACE and there is no failover between the redundant ACE modules. ACE and GSS communicating via KAL-AP, by VIP / by TAG. Active ACE VIP reports an OUTOFSERVICE state and standby ACE VIP reports an INSERVICE state. VIP state discrepancy can occur due to probe failure or some other manual intervention. Also no failover occurs between the redundant ACE modules.

The GSS answer initially transitions to an OFFLINE state when the active ACE VIP fails and then the GSS answer transitions back to an ONLINE state as it is now receives KAL-AP load information from the standby ACE. Any new DNS query sent to the GSS receives an A-record VIP response because the answer is ONLINE but connectivity to the ACE VIP fails due to the fact that the Active ACE VIP is still considered down. Workaround: Use the ACE alias IP address rather than both the active and standby ACE interface VLAN IP address so that only the active ACE will provide VIP state.

CSCtd52722—When a large number of processes are active on the CP, the CP console displays the following message text:

Couldn't save crashinfo.
Error</noCmdBold> just prior to a reload.
 
   

The crash info data is actually saved, and can be submitted to the TAC. Workaround: None needed, the message is incorrect. The crashinfo file is saved, however, it is truncated. The truncation may keep some detailed information from being saved to the crashinfo file.

CSCtd53161—If there is an expired entry within the same bucket, the connection sticks to the wrong server. For example:

show sticky database

show multiple entries for the same sticky hash.

Workaround: Clear sticky database to remove the wrong entries.

CSCtd66906—When you upgrade beyond ACE A2(1.3) version, ACE user roles (RBAC) defined as network monitor can no longer issue "delete..." command. Workaround: None.

CSCtd69388—When two ACEs are configured for redundancy, an ACE may become unresponsive temporarily while processing a load-balancing redundancy message from the peer and then the ACE reboots. Workaround: None.

CSCtd69941—ACE reboots and creates a load-balancing core file and the ixp1_crash.txt file displays many lines with the following message:

No particle in TCP Msg
 
   

Workaround: None.

CSCtd75203—The output of show sticky database detail command displays hex equivalent for IP addresses. Workaround: There is a no workaround for this now. The hex value should be manually converted to decimal.

CSCtd83789—If customized scripted probes fail repeatedly, the core files produced as a result fills up the disk and thus prevents other operations from functioning properly. It impacts any activity that implies writing on the disk. Specifically, configurations are seen truncated because of the missing space on the disk. Workaround: None.

CSCtd94085—You may observe an MTS memory leak for an invalid or a nonexistent process or PID. For a Vshell process, the MTS message queue limit is limited to a maximum of 4096 messages. Beyond that limit, any new message (for example, a changeto command is being executed), will get dropped and the following warning message is displayed on the console:

Warning:- MTS queue is full for opcode "<opcode value>" sap "<sad_id>" pid "<pid>" 
clear idle debug plugin sessions or telnet/ssh connections to recover. 

Sometimes, the PID that is displayed here may be invalid (no real process associated with it). Workaround: None.

CSCte03073—ACE HTTPS probes fail when you configure them for an IIS server that is configured with the Accept client certificates option. Workaround: None.

CSCte12130—ANM reports operational status as "not applicable (N/A)" for many virtual servers. This issue is generally seen if the ANM has been polling the ACE for a long time. ANM sometime does not read all the SNMP responses back from the ACE. Issue is seen irrespective of ACE release. Issue was also seen in ANM 2.0 and 2.2. Workaround: Rebooting the ACE fixes the issue.

CSCte16068—When you attach a probe to two different rservers and delete one of the rserver, the instance for the other rserver will be stuck to INVALID state. Workaround: Delete and re-add probe for all such rserver where probe is stuck into INVALID state.

CSCte25964—When you execute from any context different from the Admin context, the ACE show snmp group command does not give any output. This happens on all ACE versions. Workaround: None.

CSCte26173—When you perform a XML query on ACE for show commands, for example, show ft group status, the ACE contains the bash core files in the core: directory, in which some are unpackaged and some are mispackaged as VSH core files. Workaround: None.

CSCte28915—The output of the show snmp group command gives two outputs for the same SNMP group making it unclear which one is the real output for default SNMP group. Workaround: None required, this is a display issue only.

CSCte44232—The output of the show logging message all command displays numeric syslogd identifiers for unsupported messages. Workaround: None.

CSCte45777—If a timeout value is set to 2^31 or greater, the connection times out prematurely, immediately after the connection is setup. Workaround: Do not set a timeout value greater than 2^31.

CSCte46550—If ACE reboots and the HSRP multicast traffic is forwarded by ACE for a few seconds as the ACE is going down, a catchall VIP is configured that the HSRP traffic can hit. Workaround: Configure an input ACL to deny HSRP traffic.

CSCte53218—International step up certificates fail with older browsers and ACE SSL termination, SSL connections fail with reset, for example:

-snitrox reporting FINISH_MAC_MISCOMPARE error
stats crypto is reporting BAD_RECORD_MAC sent
 
   

CSCte56065—Linux b-shell executables occasionally core on the ACE module. These cores are either incorrectly packaged as Virtual Shell (VSH) cores or not packaged and compressed and left in the core: directory as core. <PID>. Workaround: None.

CSCte56420—When you configure ACE in the FT mode, it reports the following syslog error message on standby unit:

20100118-15:30:02; vss-dc-ace-01b; Jan 18 2010 15:29:35 Admin: %ACE-2-443001: System 
experienced fatal failure.Service name:System Manager (core-server)(19277) has 
terminated on receiving signal 11,system will not be reloaded

Workaround: None

CSCte61409—When you enter the show cde health command and if the ACE module is under high load, it displays the BRCM pull status as "[Not pulling]" incorrectly. Workaround: None.

CSCte61479—The show buffer usage shows incorrect values, very large values for the usage of certain type of internal ACE buffers. Workaround: The ACE module must be rebooted to clear.

CSCte63173—A buffer leak appears due to Inter-Process Control Plan (IPCP) messages between the Control Plane (CP) and Data Plane (DP). Workaround: When the problem becomes severe the ACE module needs to be reloaded.

CSCte66071—TCP or UDP configured port ranges are being inherited for non-tcp non-udp protocols when configured inside an object-group right after a TCP or UDP range. Workaround: Configure the ACL directly without using object-group.

CSCte66195—On a ACE A2(2.3) version, SIP UDP probes configured without rport enable sent by the ACE contains two semicolons in the Via header, for example:

Via: SIP/2.0/UDP 10.10.10.10:32789;;branch=z9hG4bK25708969
 
   

Workaround: configure rport enable if it is fine to use the rport extension in the setup.

CSCte66814—ACE sends RHI messages to remove static routes and then immediately sends a new message to add the same route back. Workaround: None.

CSCte77866—On an ACE running A2(2.1) version and without normalization on the VLAN interface the NAT table becomes corrupted. Workaround: A reload clears the condition on a temporary basis.

CSCte78972—If configuration changes are made prior to the core Layer 7 rules were added to the network processor core on A2.2.2, FASTPATH and CM_CLOSE tasks are stuck due to deadlock caused by lbrx and lbrxhi queues being full. Workaround: None.

CSCte81257—When you log in to the ACE and perform dynamic configuration of usernames in multiple contexts, in a user context and issue no username <name>, the ACE module fails unexpectedly and writes out a SNMP core file. Workaround: None.

CSCte83538—When you enter show buffer usage, an additional field "Hi watermark" is displayed, which allows more visibility into the buffer usage to monitor the high watermarks.

CSCte83727—When you enter the show ipcp event-history command, it indicates that the internal IPCP queue is full causing possible failures. Workaround: None.

CSCte83745—The ACE sends traps when a real server in a server farm changes state (for example, a probe fails and then the server becomes operational again). When the probe failure is detected, the ACE sends the cesRealServerStateChangeRev1 trap. When the real server becomes operational again, the probe succeeds, but the ACE incorrectly sends the cesRealServerStateUpRev1 trap. The cesRealServerStateUpRev1 trap should only be seen after user intervention (for example, after you enter the inservice command). The ACE should send the cesRealServerStateChangeRev1 trap when a server becomes operational after a probe failure.

CSCte91198—If a ACE module is configured for FTP inspection and NAT in routed mode and when a server behind the ACE acts as a FTP client and makes a connection to a server outside the ACE, the active data channel fails. You will observe the control channel messages are properly fixed up with FTP inspection, when the server opens the data channel with a SYN cookie to the NATed port, the ACE sends the SYN along with the client it translates the IP but does not translate the port. Workaround: Use inspect ftp strict command.

CSCte92842—When a user tries to remove the limit-resource all command, all the ACE contexts associated with that resource class will be left out of resources that are not separately defined. Workaround: After the fix with version A2(2.4), the following warning message is displayed when the user tries to remove the limit-resource all from a resource class:

ACE-tb3/Admin(config)# resource-class a
ACE-tb3/Admin(config-resource)# no limit-resource all minimum 0.00 maximum unlimited
Warning: The context(s) associated with this resource-class
will be denied of all the resources that are not explicitly
configured with minimum limit in this resource-class
 
   

CSCte96172—When you configure a service policy on an interface, if you configure overlapping the subnets within a class map, a syslog error is generated. This should not be generated as there is no real problem in configuring overlapping subnets. Workaround: None required.

CSCte99505—MTS leak is observed if the login fails due to the securityd process being busy. Workaround: None.

CSCtf06376—When you enter ft switchover command, some bridged connections get RST by ACE. The ft switchover command is issued to transfer mastership. Workaround: None.

CSCtf10882—When you configure an HTTP class map with more than one URL matching statement, ACE XML interface returns 105.

CSCtf14370—When you submit the following syntax to the XML agent, it fails with the complaint about backup attribute.

<policy-map_lb type="loadbalance" match-type="first-match" pmap-name="testuk-1">

<class_pmap_lb match-cmap="www99-www-url-1">

<serverfarm_pmap sfarm-name="www99" backup="WWW-NOT-AVAILABLE"></serverfarm_pmap>

</class_pmap_lb>

</policy-map_lb>

Workaround: The correct syntax is shown in the following syntax:

<policy-map_lb type="loadbalance" match-type="first-match" pmap-name="testuk-1">

<class_pmap_lb match-cmap="www99-www-url-1">

<serverfarm_pmap sfarm-name="www99" config="backup" backup-name="WWW-NOT-AVAILABLE"/>

</class_pmap_lb>

</policy-map_lb>

CSCtf18582—The show running-config command does not show the username "user." The copy running-config startup-config and the write memory commands do not save the "username user password xxx" line in the startup-configuration file. Workaround: Avoid using the username "user."

CSCtf31573—When you issue the ft switchover command to transfer mastership, some connections get RST by ACE. Workaround: None.

CSCtf33100—If two or more probes associated with the serverfarm are in the failed state, at least one probe is in the passed state and the fail-on-all configuration is removed, the rserver remains in the OPERATIONAL state and is not moved to PROBE-FAILED state. Workaround: None

CSCtf33319—When you use header-or ssl rewrite and a static parse error occurs because of an incorrect field on the server side, the ACE does not forward the page back to the client and sits idle. Client does not receive data back from GET request. Workaround: Fix server side HTTP headers or do not use rewrite.

CSCtf36703—When the device undergoes stress or excess load, the performance of the generic protocol parsing, HTTP L7 loadbalancing enabled with SYN-COOKIE or HEADER-INSERT decreases by 7 to10 percent.

CSCtf38995—You cannot login remotely using radius authentication, after a reload.

CSCtf39655—If you configure send-data option inside a finger probe with length greater than four characters, the probe fails. Workaround: Configure send-data length with less than four characters.

CSCtf43237show xlate displays thousands of entries, however show resource usage displays zero peak and zero current. Workaround: Reload the ACE.

CSCtf44818—ACE module sometimes looses its count on interface unicast bytes input counter. This can cause problems for SNMP tracking the traffic, which in turn shows ~50Gbps flowing through the ACE. Workaround: SNMP application can be configured so it ignores the counter increases above some value.

CSCtf47473—When you enter the show conn command there are no connections but sticky shows active connection count. Workaround: Clear the sticky database.

CSCtf55374—ACE displays rate limit by default, without it being configured or enabled. Workaround: Ignore the rate-limit values in this scenario.

CSCtf55391—When issuing SNMP GET of the sysObjectID OID, the ACE10 and ACE20 currently have the same sysObjectID OID value. Workaround: Verify ACE Model using ACE CLI or a show mod from the MSFC.

CSCtf57455—When a standby ACE A2(1.5a) unexpectedly reloads, it creates the following core dump:

Last boot reason:  Service "itasca_route_mgr"
 
   

Workaround: None.

CSCtf60389—If you configure TCP probes with small intervals and set the termination mode as forced, TCP probe stops firing if the server sends RST after TCP handshake. Workaround: Problem can be solved by removing and re-adding the faulty probe from rserver.

CSCtf70322—When a remote authentication with TACACS is configured and if the username has a "!" (exclamation character), the login fails for usernames with "!" character. Workaround: Do not configure usernames with "!" (exclamation character) in the TACACS server.

CSCtf72863, CSCte68716—If the client does not use window scaling option, ACE still uses the configured window scale to scale the receive window. With window scaling configured for a VIP on the ACE, if a client does not support window scaling accesses, ACE still advertises a scaled receive window after the TCP 3-way handshake is completed. Workaround: Configure tcp-options window-scale clear in connection parameter-map.

CSCtf75106—SIP inspected requests bigger than 2KB are dropped by the ACE with the counter 'SIP: Memory Allocation Failure:' increased. This is observed with many large SIP PUBLISH requests sent over connections inspected by the ACE. Workaround: User can monitor appInspect memory via show np 1|2 memory | inc appInspect command. The nominal memory usage is "11M", when this value begins to approach "29M" a proactive reboot would be required.

CSCtf75936—On a FT ACE pair, version A2(2.3) network type object, when you add a new entry to the same Object group that is associated with two different ACLs, it will appear in the expansion of one ACL, but not the other. This entry should appear in both the ACLs. Workaround: None.

CSCtf83851—When you set user-defined resource allocation rates on an ACE module, the percentage for connection rate is based on the data sheet performance 325K CPS. Once the limits are applied you will see the user context maximum is now 500K, which is also not right. It should also work where the maximum value of user context is the ACE module maximum of 325K minus any configured minimums.

Command Changes in Software Version A2(3.1)

Table 11 lists the new commands in software version A2(3.1).

Table 11 New CLI Commands in Version A2(3.1)  

Mode
Command and Syntax
Description

Exec

show interface internal seciptable

Displays the interface manager's (ifmgr) view of a logical interface and displays all the configured secondary IP addresses under an interface

Policy map class
configuration

kal-ap primary-oos

Enables or disables the new KALAP feature that informs the GSS when a primary server farm fails and a backup server farm is being used. See the "Downgrading Effects on the kal-ap primary oos Command (Downgrade from version A2(3.1) to A2(3.0))" section.

SSL parameter map
configuration

authentication-failure redirect reason serverfarm name

Before this release, the CLI would silently fail if you run out of sfarm ids while adding SSL redirects. After the fix, the CLI displays the following message:

"Number of sfarms in the config have reached the 
maximum limit!"

Table 12 lists the commands and options that have been changed in software version A2(3.1).

Table 12 CLI Commands Changed in Version A2(3.1) 

Mode
Command and Syntax
Description

Exec

backup [all][pass-phrase text_string][exclude component]

restore {all | disk0:archive_filename} [exclude ssl-files | pass-phrase text_string]

This behavioral change is a part of the support for PKCS12 and encrypted PEMs. If certificates or keys are imported into a context using a crypto passphrase, backup will fail with the following error message, unless a backup-passphrase is provided. This is because the backup requires a backup passphrase to encrypt the crypto passphrase for the crypto item.

"Error,no backup pass-phrase provided for encrypting the crypto passphrases"


Commands Inherited from Software Version A2(2.3)

Table 13 lists the new commands in software version A2(2.3).

Table 13 New CLI Commands in Version A2(2.3)  

Mode
Command and Syntax
Description

Action list modify configuration

description

Allows you to enter text that describes the action list. Enter an unquoted text string with a maximum of 240 alphanumeric characters. If the text string includes spaces, enclose the string in quotes.

Debug

debug scp ping-failures

By default, displays SCP and hardware-related statistics and warning messages on the console in case the ACE does not receive SCP messages from the supervisor engine. To disable these messages, use the no debug scp ping-failures command.

Note This command is intended for use by trained Cisco personnel for troubleshooting purposes only.

Exec

show eobc registers

Displays the contents of the Ethernet out-of-band channel (EOBC) FIFO registers.

Note This command is intended for use by trained Cisco personnel for troubleshooting purposes only.

show eobc status

Displays the status of the EOBC.

Note This command is intended for use by trained Cisco personnel for troubleshooting purposes only.

Probe SIP UDP configuration

rport enable

When the ACE is configured for SIP UDP, this command forces the SIP server to send the 200 OK message from the same port as the destination port of the probe request OPTIONS method per RFC 3581. When this SIP UDP probe option is not configured, if the SIP server sends the 200 OK message from a port that is different from the destination port of the probe request, the ACE will discard the response packet from the server.

SSL parameter map configuration

rehandshake enabled

Starting with software version A2(2.3) and higher, SSL rehandshake is disabled by default. Use this command to enable SSL rehandshake. Enter the show parameter-map command to display the status of the rehandshake enable command. For further details, see resolved caveat CSCtd00816 in the "Software Version A2(3.1) Resolved Caveats" section.


Table 14 lists the commands and options that have been changed in software version A2(2.3).

Table 14 CLI Commands Changed in Version A2(2.3)  

Mode
Command and Syntax
Description

Exec

show parameter-map

A new rehandshake field reports the status of the new rehandshake enable command. Possible values are: enabled or disabled (the default).

Exec

show service-policy [policy_name] [detail]

The Regex dnld status field has been added to the output of the show service-policy [policy_name] [detail] command to display the status of a regular expression (regex) download. The possible field values are: QUEUED, SUCCESSFUL, or FAILED.


Commands Inherited from Software Version A2(1.6)

Table 15 lists the commands that changed in software version A2(1.6).

Table 15 CLI Commands Changed in Version A2(1.6)  

Mode
Command and Syntax
Description

Exec

clear stats resource-usage

The new resource-usage keyword clears the Peak and Denied fields displayed by the show resource usage command.

Exec

copy checkpoint:name {disk0:[path/]filename | image:[image_name] | startup-config | ftp://server/path[/filename] | sftp://[username@]server/path[/filename] | tftp://server[:port]/path[/filename]}

The new checkpoint keyword allows you to copy the checkpoint file to disk0, the image directory, the startup configuration file, or a remote server.

Exec

copy {disk0:[path/]filename | image:[image_name] | running-config | startup-config | ftp://server/path[/filename] | sftp://[username@]server/path[/filename] | tftp://server[:port]/path[/filename]} checkpoint:name

The new checkpoint keyword allows you to copy the checkpoint file from disk0, the image directory, the running configuration file, the startup configuration file, or a remote server.

Exec

show accounting log all

The new all option in the Admin context displays the accounting log for all contexts.

Exec

show interface

This command now displays the following:

The reason for the interface to transition to the Up state

Time stamp when the last change occurred

Number of transitions the interface experienced since it was created

Last three previous states including the timestamp and the reason for the Up or Down transitions

Exec

show np np_number nat policies

This command no longer displays bitmap information.

Exec

show service-policy [policy_name] summary

This command now displays a summary of current, hit and drop connections for all VIP addresses in a Layer 3 rule. Previously, this command displayed connection counts for each VIP address even if the address was not hit. However, the ACE calculates connection counts per Layer 3 rule, not per VIP address.

Exec

show stats loadbalance

This command now displays the following two counters:

Total proxy misses—Total number of dropped connections when the related proxy is closed, the connection is dead, or the proxy sequence number does not match.

Total misc errors—Total number of dropped connections for miscellaneous errors, for example, remote sticky lookup timeout, pmap errors, or POST message to an HTTP failure.

Total L4 Close Before Process—For future use. Currently, this counter does not increment.

Total L7 Closs Before Parse—For future use. Currently, this counter does not increment.

Total Close Msg for Valid Real—Total number of close connection messages with a valid real server ID.

Total Close Msg for Invalid Real—Total number of Total number of close connection messages with a valid real server ID. This counter increases only in the Admin context.

Exec

show system resources

This command is now available in all user contexts. Previously, this command was only available in the Admin context.

It also now displays the Average ME Utilization statistics.

Exec

show tech support

The CLIs that the show tech support command executes are no longer logged.

Also, the show tech support command includes the show accounting log all command in the Admin context.

Configuration

context name

Per CSCsu76777, this command now prohibits you from configuring a context name containing opening braces ({), closing braces (}), white spaces, or any of the following symbols: ` $ & * ( ) \ | ; ' " < > / ?

Configuration

logging reject-newconn

This command has been removed from the ACE CLI.

If you upgrade the ACE to software release A2(1.6) but had previously configured the logging reject-newconn command in the earlier release, the ACE will display the following execution error message:

'logging reject-newconn keyword' 
*** Context number: cmd parse error *** 
 
        

To avoid this error message, delete the logging reject-newconn command from the startup-config file before you upgrade the ACE.

Configuration

snmp-server enable traps slb serverfarm

The new serverfarm option sends a trap when all real servers are down in the server farm or the server farm changes state.

The CISCO-SLB-EXT-MIB MIB now includes the cslbxServerFarmStateChange trap. This notification is supported with the following varbinds:

cslbxServerFarmName

cslbxServerFarmState

cslbxServerFarmStateChangeDescr

cslbxServerFarmNumOfTimeFailOvers

cslbxServerFarmNumOfTimeBkInServs

The server farm can change from the inactive to active state or active to inactive state. The reasons for changing from the active to inactive state are as follows:

All the real servers are down.

One or more real server is in the maximum connection or maximum load state.

The server farm reaches its partial limits.

Parameter map

description string

no description

This new command allows you to provide a description for the parameter map. The string argument is a maximum of 240 characters. Use the no form of the command to remove the description.

The show parameter-map command displays the description string.

Policy map

description string

no description

This new command allows you to provide a description for the policy map. The string argument is a maximum of 240 characters. Use the no form of the command to remove the description.

Server farm

use-same-np

This new command enables the full maximum connection calculation in a single NP. Use the no form of the command to disable the full maximum connection calculation in a single NP.

Before configuring the use-same-np command, configure the hw-module cde-same-port-hash command in configuration mode.


System Log Messages

Software version A2(3.1) introduces the following new or revised system log (syslog) messages.

New Syslog Messages

504003

Error Message    %ACE-4-504003: Admin context is not guaranteed of one or more 
resources. Admin context might get starved of these resources, leading to denial 
of some of the services.

Explanation    This syslog will be generated when you do the following:

Associate a resource class with the Admin context for the first time and this association results in at least one of the resources not being guaranteed to the Admin context

Allocate 100 percent of any resource to a user context

Make the following configuration changes to the Admin context that has a resource class with all resources allocated:

Replace the resource class in the Admin context with the default resource class

Change the minimum guaranteed percentage for any of the resources associated with admin context to zero

Recommended Action    Use the show resource usage command to check which of the resources are allocated as zero percentage under the min column for Admin context and allocate these resources using resource-class command to avoid starvation.

Software Version A2(3.0) Resolved Caveats and Open Caveats

This release note includes resolved and open caveats that have a severity level of Sev1, Sev2, and customer-use Sev 3. The following sections contain the resolved and open caveats in software version A2(3.0):

Software Version A2(3.0) Resolved Caveats

Software Version A2(3.0) Open Caveats

Software Version A2(3.0) Resolved Caveats

The following resolved caveats apply to software version A2(3.0):

CSCsu29301—When the ACE module is in a Catalyst 6500 series chassis where SPAN is configured for RX or both, it duplicates ingress SPAN packets and does not duplicate TX packets. Workaround: None.

CSCsu88684, CSCsq27062—When you configure the ACE with a large number of contexts and enable redundancy, as traffic flows on the ACE, the ACE becomes unresponsive and displays the following messages on the console:

mts_acquire_q_space() failing - no space in sap 516 
sap=516 rq=102048 lq=0 pq=0 nq=0 sq=0 buf_in_transit=937, bytes_in_transit=82456
sap=1118 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=936, bytes_in_transit=145904
sap=514 rq=937 lq=0 pq=0 nq=0 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1084 rq=935 lq=0 pq=0 nq=1 sq=0 buf_in_transit=0, bytes_in_transit=0
sap=1025 rq=0 lq=0 pq=0 nq=0 sq=0 buf_in_transit=102052, bytes_in_transit=9388784
 
   

The ACE reboots after the messages are displayed. Workaround: None.

CSCsv52942—When a server farm with no backup goes to an inactive state because all the real servers go to a MAXCONNS state, the ACE DP processor does not accept connections to the real servers even when they are no longer in a MAXCONNS state until the DP processor gets a "Back-in-service message" from the CP processor. Workaround: Configure a backup to the server farm.

CSCsv74527—When DNS traffic is consistently running at more than 10000 cps in an ACE HA environment, after approximately 2 hours, proxy entries are leaked on the standby ACE. The proxy entries are leaked and not cleared on the standby ACE due to connection validation errors. Workaround: None.

CSCsv98101—When using A2(1.2), the ACE console and remote access failed but network traffic continued to pass. Workaround: Reboot the ACE.

CSCsw40764—When the ACE executes the no access-list command to delete an ACL configured with 64,000 entries, an API timeout occurs. Workaround: Do not delete all of these entries from an ACL at one time. Delete the entries from an ACL one at a time or in small chunks.

CSCsx19525—When you configure 1,000 SSL VIPs on the ACE and then you change the configuration on those VIPs, a buffer leak occurs as displayed by the show np 1 me-stats command "-scommon" output and traffic conditions. Workaround: Reboot the ACE and do not make configuration changes that affects those VIPs.

CSCsx39224—When you configure the sticky-serverfarm command in the policy map rather than the serverfarm command and the real servers are placed in an out-of-service state to make the server farm inactive, the backup server farm does not accept the connections. Workaround: Configure the serverfarm command in the policy map instead of the sticky-serverfarm command.

CSCsx68671—A Layer 7 UDP connection with generic protocol parsing, payload sticky, and UPD fast age traffic may cause a large memory leak in the internal buffer particles on the ACE dataplane. Workaround: None.

CSCsx93137 and CSCsx93995—When you enter either of the following commands in any context, but you do not enter the remote host password when prompted, the ACE waits for your input:

crypto import ftp | sftp | {bulk ftp}

crypto export ftp | sftp

Then, if you enter one of the following commands, the session may appear to be in an unresponsive state:

crypto delete

crypto export

crypto generate csr

crypto generate key

crypto import

crypto verify

show crypto authgroup

show crypto certificate

show crypto chaingroup

show crypto files

show crypto key

After a while, the command aborts with a "SSL PKI subsystem is busy. Please try again later" message. Reissuing the command results in the same behavior.

Workaround: Enter the remote host password as requested by the associated crypto import | export command. If the problem persists, clear the relevant sessions by executing one of the following commands:

clear users

clear telnet session_ID

clear ssh session_ID

You can execute those commands if you have the appropriate privileges (for example, Admin). For details about role-based access control (RBAC) and user roles, see the Cisco Application Control Engine Module Virtualization Configuration Guide.

CSCsx97484—When you reload the ACE while the primary server farm is out of service, the traffic does not switch to the backup server farm. This condition is specific to bootup only and does not occur if the primary server farm fails in runtime, such as when arp/probe fails for all real servers. Workaround: Configure one real server under Primary, which may trigger the failover.

CSCsy04371, CSCsy30440—When a server farm with no backup transitions to the Inactive state after all the real servers transition to the MAXCONNS state, if the real servers transition out of the MAXCONNS state, they may not accept connections. Workaround: Configure a backup to the server farm.

CSCsy13724—When transparent probes are configured, the ACE may incorrectly use the wrong real server's MAC address if a new probe is sent to another real server before the previous probe completes. For example, suppose that the ACE sends a TCP SYN (probe A) to the real server with the MAC address ending with 1a:0d. The real server will respond with a SYN-ACK. If the ACE sends another probe to a different real server (for example, one whose mac address ends in 15:2d) before probe A completes, the ACE may use the MAC address ending with 15:2d for the ACK instead of the MAC address ending with 1a:0d for probe A. The real server will send a TCP RST in response. Workaround: Use the real server's physical IP address as the probe destination address.

CSCsy34814—The syslog message 305010 includes the duration of the Xlate translation. However, this duration is always equal to the xlate idle timeout. Workaround: Use the timestamps in the creation and tear down of the xlate connections to calculate the xlate duration.

CSCsz77633—When there are more than 14 Layer 4 sticky connection requests and the ACE is processing both Layer 4 and Layer 7 traffic either in the same context or in different contexts, the ACE incorrectly resets and drops the connections after traffic is sent for some duration. Workaround: None.

CSCsz86630, CSCtb44983—When you upgrade the ACE from version A2(1.1) to A2(1.2) or greater, the DNS Inspect function may not work after the upgrade. When this condition exists, the following two errors occur under me-stats -sfixup statistics: +[Hash miss errors] + [NAT app fixup response error]. Workaround: Disable DNS Inspect and configure more aggressive timeouts (for example, 4 seconds) for UDP and port 53.

CSCta03825, CSCtb44976—When the UDP booster feature is enabled, every first packet is not forwarded to the real server on each NP, which results in two packets being dropped per session. Workaround: Disable the UDP booster feature.

CSCta29049, CSCtb44970—When the UDP booster feature is enabled, the ACE drops UDP packets that originate from the server. Workaround: Disable the UDP booster feature.

CSCta93957—If you upgrade a redundant ACE pair to software version A2(2.1), downgrade the standby to software version A2(1.4) and allow the pair to synchronize configurations, and then upgrade the standby again to A2(2.1), the standby ACE does not lock configuration mode, allowing you to make configuration mode changes. Workaround: Enable a bulk synchronization by entering the no ft auto-sync command followed by the ft auto-sync command on the active ACE.

CSCtb05686—When an interface is configured with multiple service policies and you delete one of the service policies, the Layer 7 connections in the other service policies may reset. Workaround: None.

CSCtb47541—When "failaction reassign" is enabled in a firewall load-balancing configuration under the server farm and all of the rservers in the server farm are down and their probes all fail at the same time, the ACE becomes unresponsive to most CLI commands. When this condition occurs, the CPU spikes up to 100 percent by the cfgmgr process. This condition does not occur if any of the rservers are online and are passing any probe. Workaround: Disable the failaction reassign command under the server farm.

Software Version A2(3.0) Open Caveats

The following open caveats apply to software version A2(3.0):

CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, and then try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in an RSA format instead of an RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.

CSCso33506—In a redundant configuration with the FT pair running mismatched code (A1(x) and A2(x)) and mismatched licenses, if the active ACE is rebooted, the Admin context comes up, but, in user contexts, the running-config file is blank. This behavior occurs only when there is both a license and a code mismatch. Workaround: Resolve one of the mismatches and reload the ACEs or enter the copy start run command in each user context.

CSCso76159—When you dynamically modify a service policy to use an HTTP parameter map with the header modify per request command, the ACE does not insert a header into every GET request for existing long-lived persistent flows. However, the ACE does insert a header into every GET request for new flows. Workaround: None.

CSCsr01570, CSCsy90965—The Set-Cookie: length is null. Changing the default class-map from a sticky server farm to none does not eliminate a cookie insertion. Workaround: Remove and then reenter the class class-default command.

CSCsr76812—When you configure the ACE with Layer 7 load balancing, TCP connection may be disrupted. Packets arrive at the client in reverse order or packets are forced to be resent. Workaround: None.

CSCsu76777—When you have configured context names that use special characters that are interpreted by the command shell (for example, semicolon, pipe, and so on) and you enter the write memory all command, the command generates errors and the output shows the attempted execution of shell commands. Workaround: When you define a context name, avoid using white space or any of the following special characters: `$&*()\|;'"<>/?.

CSCsv09963—When you repeatedly add and remove VLANs on a context, the ACE loses memory. Workaround: None.

CSCsv31046—When you configure the least-connections predictor on the ACE, the ACE may not sustain 160,000 CPS traffic. Workaround: None.

CSCsv54222—When an HTTP client sends pipelined requests, if the next request comes in the middle of the server response, the HTTP connection becomes unresponsive and data is missing on the web page. Workaround: Configure a connection parameter-map with the set tcp wan-optimization rtt 0 command.

CSCsx05150—When using 2048-bit certificate and key pairs with block and export ciphers, a rehandshake may lead to stuck connections. Workaround: Either use nonblock and nonexport ciphers or use certificate and key pairs that are less than 2048 bits.

CSCsx13147—When you import a number of SSL PKI key or certificate files into a context by using the crypto import command, if you remove the context without first removing the files through the crypto delete command, the ACE may not import additional SSL PKI key or certificate files. The failure is due to a lack of resources or during a subsequent file import process, some or all of the previously imported key or certificate files may be forcibly removed from some or all contexts. Workaround: Use the crypto delete command to explicitly delete the SSL PKI key or certificate files from the contexts before removing the context. Try rebooting the ACE if this problem has already happened.

CSCsx37047—When you configure and unconfigure an object group on an ACE, it allows invalid traffic and the acl-merge list becomes corrupted. Workaround: Remove and readd the access group to the interface or globally.

CSCsx52128—When you copy a large configuration with many ACLs to the running-config file and perform other configuration changes continuously, the aclmerged process does not get the CPU and also the configurations result in API errors. Workaround: When you copy a large configuration with many ACLs to the running-config file, wait approximately 2 minutes for it to complete. Do not perform any configuration changes at that time.

CSCsx80363—When the ACE uses a single IP source NAT with server connection reuse, PAT, and a high rate of traffic of approximately 30,000 connections per second in a one-arm topology, it reboots. Workaround: None.

CSCsy23268—The ACE may send probe traffic with the source IP address of the alias IP address instead of the local interface IP address. This issue occurs on the active ACE only. Workaround: None.

CSCsy29181—If either of the DP processors is at MAXCONN, the ACE should show MAXCONN in the show commands. However, the ACE waits until both DP processors are at MAXCONN. This issue occurs when the cde-same-port-hash is configured. Workaround: None.

CSCsy65650—When the ACE reports the termination of TCP flows, it may display incorrect values for the duration and amount of data transferred. This issue occurs with HTTP and connections that are terminated with TCP RST. Workaround: None. If accounting is needed and relies on this log, use another method.

CSCsy88379—The TAC diagnostic script showtech generates large output due to the show xlate command. Workaround: None.

CSCsy98701—The standby ACE generates a load-balancing core file when you configure two ACEs as FT pairs that are replicating sticky entries and you enter certain show commands on the active/master ACE. Workaround: None.

CSCsz10107—When you configure preempt and the Catalyst 6500 with an active ACE module is reloaded, the ACE may not correctly replicate connections when it reboots and becomes active again. Some connections may get dropped. Workaround: None. This issue does not occur when reloading only the ACE or if preempt is not configured.

CSCsz14634—The ACE has issues when you copy large configurations from TFTP to the running-configuration and use the snmp-server community command to add the public group Network-Monitor to a context when the command was not in the original configuration. Workaround: None.

CSCsz18739—The ACE reloads when running software version A2(1.4) and RADIUS AAA is configured. Workaround: None.

CSCsz19849—You cannot import an ACE VIP in WAF. Importing works in software version A2(1.2) and in A2(1.3). Workaround: None.

CSCsz28035—Accessing the qnx shell from the physical console port of either NP on an ACE puts you in a shell. If you type exit, the NP console hangs and becomes inaccessible. Workaround: None.

CSCsz31739—When the VIP is out of service and loadbalance icmp-reply is not configured, the virtual server entry still exists in the ARP cache. The ACE will respond to ARP requests sent for this VIP. Workaround: None.

CSCsz34011—After a series of reboots, both ACE modules lose their context configurations. If the active ACE halts and reloads, after it reboots it will read the first half of the startup-config, establish FT with the standby ACE (the new active), and synchronized the configuration to obtain the rest of the configurations from the other ACE. If the other ACE stops functioning, the active ACE will not have obtained the rest of the configurations, including context configurations. Context configurations may be lost, although they still exist in the startup-config. Workaround: None.

CSCsz34933—The ACE may send a reset with the sequence number zero for a probe configured with the connection term forced command. Workaround: Use the graceful termination no connection term command.

CSCsz40699—When you use the SLB-Admin, Server-Appln-Maintenance, or a custom role with a create feature server farm rule, you cannot bring real servers in or out of service under the server farm. Workaround: None. There are currently no workarounds using these specific roles. However, you can complete these tasks using the Admin role.

CSCsz49088—When you monitor the ACE CPU, you can only monitor it using an Admin role. The show processes cpu command is available only in the Admin role. The Network-Monitor role, which should have access to all show commands is unable to access the show processes cpu command. Configuring a new role on the ACE does not allow you to monitor the system feature. Therefore, only Admin users are able to run this command. Workaround: Run the show processes cpu command in an Admin role.

CSCsz67761—When a network error, such as a network interface going down, occurs during the bulk importing of crypto files, the temporary storage space for imported crypto files is not gracefully released. Some of the temporary files remain in the temporary storage area until the system is reloaded. Bulk import procedures currently do not perceive network failures or inactivity if the transfer of the files has begun. Workaround: None.

CSCsz85367—When you configure and unconfigure access lists in a loop, the ACE experiences a memory leak. Workaround: Do not configure and unconfigure access lists in a loop.

CSCta20756, CSCsx15558—When the ACE has over 120,000 concurrent SSL connections, it displays SSL connection rate denies, FastQ transmit back pressure, and SSL RX back pressure. Eventually, the ACE becomes unresponsive. Workaround: None.

CSCta39372—When you perform repetitive checkpoint rollbacks, the ACE becomes unresponsive after 5 to 6 hours. Workaround: None.

CSCta45580—When the ACE is unable to download a CRL because the CRL server is down, it does not always attempt another download when the CRL server returns to an online state. This condition occurs when more than 50 VIPs use the same SSL proxy with CRL applied. Workaround: Remove the CRL configuration and then configure it again.

CSCta73571—When you configure ft track for an interface that is constantly down and then attempt a checkpoint rollback from a large configuration to an empty configuration, the rollback ends prematurely, resulting in a partial rollback. The ACE, however, indicates that the rollback is complete. Workaround: Attempt the rollback once again. If it fails again, configure ft track with a greater difference between the active and standby priority settings.

CSCta83978—If you download an unusually large number of best-effort CRLs from a server, the SSL process on the control plane may become unresponsive. Workaround: Do not use best-effort CRLs.

CSCta92673—When SSL traffic is flowing and you reconfigure SSL proxies that contain authgroups, the ACE may leak memory in the control plane. The memory leak is directly proportional to the number of reconfigurations that you perform. Workaround: Avoiding reconfiguring an SSL proxy when an authgroup is applied to the proxy.

CSCta92891—If you change the load-balance predictor from least conns to hash url with a mixed traffic flow that consists of both TCP and UDP, the ACE may become unresponsive and generate a loadBalance_g_ns core dump file. Workaround: None.

CSCtb02056—When you configure the ACE with SSL certificates and keys in multiple contexts, the output of the show crypto certificate all command may become corrupted. Workaround: Use the show crypto certificate cert_name command instead of the show crypto certificate all command.

CSCtb03138—When the configuration for the VLAN that you use as an SNMP trap source is missing either the IP address or peer IP address, the SNMP configuration does not synchronize to the standby ACE. This condition creates the following error: Incremental Sync Failure: snmp config syn. Workaround: Configure the VLAN with an IP address and peer IP address.

CSCtc14102—When the total size of all headers exceeds the maximum size of 512 bytes, the CLI does not issue an error message to prevent the user from exceeding the limit. This condition results in the ACE deleting all configured header information. Workaround: None.

CSCtc36837—When a client sends traffic to a secondary IP on a BVI interface in the standby ACE (peer secondary IP under the BVI), the ACE may not process the traffic correctly if either of the following conditions exist:

The client knows the standby ACE MAC address, but the ACE has not learned the client MAC address.

You clear the MAC address table in the Catalyst 6500 series switch or the Cisco 7600 series router and enter the clear arp-cache interface vlan vlan_id command.

Workaround: Enter the clear mac-address-table dynamic vlan vlan_id on the supervisor engine and the clear arp no-refresh command on the standby ACE. Then, ping the client PC from the standby ACE.

Available ACE Licenses

By default, the ACE supports virtualization with one Admin context and five user contexts, 4 gigabits per second (Gbps) module bandwidth, and 1,000 SSL transactions per second (TPS). You can increase the number of default user contexts, module bandwidth, and SSL TPS by purchasing the following licenses:

ACE-VIRT-020—20 virtual contexts

ACE-VIRT-050—50 virtual contexts

ACE-VIRT-100—100 virtual contexts

ACE-VIRT-250—250 virtual contexts

ACE-08G-LIC—8 Gbps bandwidth

If you purchase an ACE with a bandwidth of 4 Gbps, you can upgrade the module bandwidth to 8 Gbps by using the ACE-UPG1-LIC license.

ACE-16G-LIC—16 Gbps bandwidth (ACE20-MOD-K9 module only)

If you purchase an ACE with a bandwidth of 8 Gbps, you can upgrade the module bandwidth to 16 Gbps by using the ACE-UPG2-LIC license (ACE20-MOD-K9 module only).

ACE-SSL-5K-K9—SSL with 5,000 TPS

ACE-SSL-10K-K9—SSL with 10,000 TPS

ACE-SSL-15K-K9—SSL with 15,000 TPS

You can upgrade virtualization in increments, provided that you do not exceed the limits of the ACE (a maximum of 250 contexts), by using the following licenses:

ACE-VIRT-UP1—Upgrades 20 to 50 contexts

ACE-VIRT-UP2—Upgrades 50 to 100 contexts

ACE-VIRT-UP3—Upgrades 100 to 250 contexts

You can upgrade SSL in 5,000 TPS increments up to a maximum of 15,000 TPS by using the following SSL upgrade licenses:

ACE-SSL-UP1-K9—Upgrades SSL from 5,000 TPS to 10,000 TPS (3.0(0)A1(3) or later)

ACE-SSL-UP2-K9—Upgrades SSL from 10,000 TPS to 15,000 TPS (3.0(0)A1(3) or later)

You can also obtain an ACE demo license for each type of virtualization, bandwidth, or SSL TPS license, including upgrade increments for contexts. You can get a demo license that is valid between 30 and 90 days. At the end of this period, you will need to update the demo license with a permanent license to continue to use the ACE software. To view the expiration of the demo license, use the show license usage command in Exec mode. If you need to replace the ACE module, you can copy and install the licenses onto the replacement module.


Note You can access the license and show license commands only in the Admin context. You must have the Admin role in the Admin context to perform the tasks of installing, removing, and updating the license.


Ordering an Upgrade License and Generating a License Key

This section describes the process to order an upgrade license and to generate a license key for your ACE. To order an upgrade license, perform the following steps:


Step 1 Order one of the licenses from the list in the "Available ACE Licenses" section using any of the available Cisco ordering tools on Cisco.com.

Step 2 When you receive the Software License Claim Certificate from Cisco, follow the instructions that direct you to the cisco.com website. As a registered user of cisco.com, go to this URL:

http://www.cisco.com/go/license

Step 3 Enter the Product Authorization Key (PAK) number found on the license certificate as your proof of purchase.

Step 4 Provide all the requested information to generate a license key.

Step 5 After the system generates the license key, you will receive a license key e-mail with an attached license file and installation instructions. Save the license key e-mail in a safe place in case you need it in the future (for example, to transfer the license to another ACE).


For information about installing and managing ACE licenses, refer to Chapter 3, Managing ACE Software Licenses, in the Cisco Application Control Engine Module Administration Guide.

Upgrading Your ACE Software

For complete instructions on how to upgrade your ACE software, see the Cisco Application Control Engine Module Administration Guide.


Note Per CSCtk04002, when both active and standby ACEs are running software version A2(3.1) with routed connections and you upgrade the standby ACE to software version A2(3.2) or later, if you use the ft switchover all command in the Admin context of the active ACE, it advertises an RST to the client and server. The connections close on the client and server. However, the connections still exist on the ACE. This issue is seen only with software version A2(3.1).

To workaround this issue, use the ft switchover all command in the Admin context of the standby ACE running software version A2(3.2) or later.



Note To upgrade your ACE software to version A2(1.0) or higher, your ACE must be running software version 3.0(0)A1(5a) or higher.



Caution We strongly recommend that you do not make any CLI changes when the ACE modules in a redundant configuration are running different software versions. Unexpected results may occur. Remove any new feature commands before performing a downgrade on the ACE.

Before you upgrade your ACE software, be sure that your ACE configurations meet the upgrade prerequisites in the following sections:

Changing the Admin Password

Changing the www User Password

Checking Your Configuration for FT Priority and Preempt

Creating a Checkpoint

Updating Your Application Protocol Inspection Configurations

Changing the Admin Password

Before you upgrade to software version A2(1.0) or higher, you must change the default Admin password, if you have not already done so. Otherwise, after you upgrade the ACE software, you will be able to log in to the ACE only through the console port or through the supervisor engine of the Catalyst 6500 series switch or the Cisco 7600 series router. For details about changing the Admin password, see the Cisco Application Control Engine Module Administration Guide.

Changing the www User Password

Before you upgrade to software version A2(1.0) or higher, you must change the default www user password if you have not already done so. Otherwise, after you upgrade the ACE software, the www user will be disabled and you will not be able to use Extensible Markup Language (XML) to remotely configure an ACE until you change the default www user password. For details about changing the www user password, see the Cisco Application Control Engine Module Administration Guide.

Checking Your Configuration for FT Priority and Preempt

If you want the currently active ACE to remain active after the software upgrade, be sure that the active ACE has a higher priority than the standby (peer) ACE and that the preempt command is configured. To check the redundant configuration of your ACEs, use the show running-config ft command. The preempt command is enabled by default and does not appear in the running-config file.

Creating a Checkpoint

We strongly recommend that you create a checkpoint in the running-configuration file of each context in your ACE. A checkpoint creates a snapshot of your configuration that you can later roll back to in case a problem occurs with an upgrade and you want to downgrade the software to a previous release. Use the checkpoint create command in Exec mode in each context for which you want to create a configuration checkpoint and name the checkpoint. For details about creating a checkpoint and rolling back a configuration, see Cisco Application Control Engine Module Administration Guide. For information about downgrading your ACE, see the "Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration" section.

Updating Your Application Protocol Inspection Configurations

Because the ACE version A2(1.0) or higher software has stricter error checks for application protocol inspection configurations than A1(x) software versions, be sure that your inspection configurations meet the guidelines that follow. The error checking process in A2(1.0) or higher software denies misconfigurations in inspection classifications (class maps) and displays error messages. If such misconfigurations exist in your startup- or running-configuration file before you load the A2(1.0) or higher software, the standby ACE in a redundant configuration may boot up to the STANDBY_COLD state. For information about redundancy states, see the Cisco Application Control Engine Module Administration Guide.

If the class map for the inspection traffic is generic (match . . . any or class-default is configured) so that noninspection traffic is also matched, the ACE displays an error message and does not accept the inspection configuration. For example:

switch/Admin(config)# class-map match-all TCP_ANY
switch/Admin(config-cmap)# match port tcp any
 
switch/Admin(config)# policy-map multi-match FTP_POLICY
switch/Admin(config-pmap)# class TCP_ANY 
switch/Admin(config-pmap-c)# inspect ftp
Error: This class doesn't have tcp protocol and a specific port
 
   

The following examples show some of the generic class-map match statements and an ACL that are not allowed in A2(1.0) or higher inspection configurations:

match port tcp any

match port udp any

match port tcp range 0 65535

match port udp range 0 65535

match virtual-address 192.168.12.15 255.255.255.0 any

match virtual-address 192.168.12.15 255.255.255.0 tcp any

access-list acl1 line 10 extended permit ip any any

For application protocol inspection, the class map must have a specific protocol (related to the inspection type) configured and a specific port or range of port numbers.

For HTTP, FTP, RTSP, Skinny, and ILS protocol inspection, the class map must have TCP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port tcp eq www
 
   

For SIP protocol inspection, the class map must have TCP or UDP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port tcp eq 124
 
   

or

host1/Admin(config-cmap)# match port udp eq 135
 
   

For DNS inspection, the class map must have UDP as the configured protocol and a specific port or range of ports. For example, enter the following commands:

host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match port udp eq domain

For ICMP protocol inspection, the class map must have ICMP as the configured protocol. For example, enter the following commands:

host1/Admin(config)# access-list ACL1 extended permit icmp 192.168.12.15 255.255.255.0 
192.168.16.25 255.255.255.0 echo
 
   
host1/Admin(config)# class-map match-all L4_CLASS
host1/Admin(config-cmap)# match access-list ACL1
 
   

Downgrading Your ACE Software from Version A2(1.0) or Higher to 3.0(0)A1(6.x) in a Redundant Configuration

If you need to downgrade your ACE software from version A2(1.0) or higher to an earlier version, use the procedure that follows. You can downgrade your ACE from software version A2(1.0) or higher to 3.0(0)A1(6.1) or higher. Downgrading your ACE software to a software version below 3.0(0)A1(6.1) is not supported and not recommended. We recommend that you downgrade to the highest 3.0(0)A1(6.x) software version that is available. This procedure assumes that your ACEs are configured as redundant peers to ensure that there is no disruption to existing connections during the downgrade process. In the following procedure, the active ACE is referred to as ACE-1 and the standby ACE is referred to as ACE-2.


Caution We strongly recommend that you do not make any CLI changes when the ACE modules in a redundant configuration are running different software versions. Unexpected results may occur. Remove any new feature commands before performing a downgrade on the ACE.

This section contains the following topics:

Before You Begin

Downgrade Procedure

Before You Begin

Before you downgrade your ACE software, ensure that the following conditions exist:

Identical versions of 3.0(0)A1(6.x) software images reside in the image: directory of both ACEs.

The active ACE has a higher priority than the standby ACE and preempt is enabled on the FT group if you want the active ACE to remain active after the downgrade procedure.

Downgrade Procedure

To downgrade your A2(1.0) or higher software in a redundant configuration, perform the following steps:


Step 1 If you have created checkpoints in your 3.0(0)A1(6.x) running-configuration files (highly recommended), roll back the configuration in each context on each ACE to the check-pointed configuration. For example:

host1/Admin# checkpoint rollback CHECKPOINT_ADMIN
host1/Admin# changeto C1
host1/C1# checkpoint rollback CHECKPOINT_C1
 
   

Do the same on the other ACE. For information about creating checkpoints and rolling back configurations, see Chapter 4, Managing the ACE Software.

Step 2 Configure ACE-1 to automatically boot from the 3.0(0)A1(6.x) image. To set the boot variable and configuration register to 1, use the boot system image: and config-register commands in configuration mode. For example, enter the following command:

host1/Admin# config
host1/Admin(config)# boot system image:c6ace-t1k9-mzg.3.0.0_A1_6_3.bin
host1/Admin(config)# config-register 1
host1/Admin(config)# exit
host1/Admin# 
 
   

You can set up to two images through the boot system command. If the first image fails, the ACE tries to boot from the second image.


Note Use the no boot system image: command to remove the configured A2(1.x) or higher boot variable.


Step 3 Verify that the boot variable was synchronized to ACE-2 by entering the following command on ACE-2:

host1/Admin# show bootvar
BOOT variable = "disk0:c6ace-t1k9-mzg.3.0.0_A1_6_3.bin"
Configuration register is 0x1
host1/Admin#
 
   

Step 4 Use the show ft group detail command to verify the state of each module. Upgrade the ACE that has its Admin context in the STANDBY_HOT state (ACE-2) first by entering the reload command.When ACE-2 loads the startup-configuration file, you may observe a few errors if you did not roll back the configuration to a checkpoint. These errors are harmless and occur because the 3.0(0)A1(6.x) software does not recognize the A2(1.x) or higher commands in the startup-configuration file. After ACE-2 boots up, it may take a few minutes to reach the STANDBY_HOT state again. At this time, configuration synchronization is disabled, but the connections through ACE-1 are still being replicated to ACE-2.

host1/Admin# reload
This command will reboot the system
Save configurations for all the contexts. Save? [yes/no]: [yes]
 
   

Step 5 Perform a graceful failover of all contexts from ACE-1 to ACE-2 by entering the ft switchover all command in Exec mode on ACE-1. ACE-2 becomes the new active ACE and assumes mastership of all active connections with no interruption to existing connections.

host1/Admin# ft switchover all
 
   

Step 6 Reload ACE-1 with the same 3.0(0)A1(6.x) software version as ACE-2. Again, you may observe a few errors as ACE-1 loads the startup-configuration file.

host1/Admin# reload
 
   

After ACE-1 boots up, it assumes the role of standby and enters the STANDBY_HOT state (this may take several minutes). You can verify the states of both ACEs by entering the show ft group detail command in Exec mode. Because both ACE-1 and ACE-2 are running the same version of software now, configuration mode is enabled. The configuration is synchronized from ACE 2 (currently active) to ACE-1. If ACE-1 is configured with a higher priority and preempt is configured on the FT group, ACE-1 reasserts mastership after it has received all configuration and state information from ACE-2, making ACE-2 the new standby. ACE-1 becomes the active ACE once again.

Step 7 Perform manual cleanup in the running-configuration files of both ACEs to remove unnecessary version A2(1.0) or higher configuration elements. For example, you may need to remove a service policy from an interface that was part of the version A2(1.x) or higher configuration that is no longer needed in version 3.0(0)A1(6.x).

Step 8 Enter the write memory all command in both ACEs to save the running-configuration files in all configured contexts to their respective startup-configuration files. This action will eliminate future errors when the ACEs reload their startup-configuration files.


Downgrading Effects on the kal-ap primary oos Command (Downgrade from version A2(3.1) to A2(3.0))

In case of a downgrade from version A2(3.1) to A2(3.0), ensure that the following conditions exist:

Case 1

The following example shows the kal-ap primary-oos configuration without the kal-ap-tag before the downgrade:

policy-map multi-match PoM-L47

class CM-VIP

loadbalance vip inservice

loadbalance policy PM-LB

kal-ap primary-oos

After the downgrade to A2(3.0), the output of the kal-ap primary-oos command configuration will be as shown in the following example:

policy-map multi-match PoM-L47

class CM-VIP

loadbalance vip inservice

loadbalance policy PM-LB

kal-ap tag primary-oos

Due to the kal-ap primary-oos configuration ACE module expects kal-ap keepalives from GSS with a specific tag value. This will not affect the normal behavior of the module. There will be no impact if there is no probing from GSS.

Before the downgrade, if ACE module responds with the default setting without the tag setting for GSS probing, It is required to revert the configuration after the downgrade as the requests from GSS will not be coming along with the primary-oos tag value.

Case2

The following example shows the kal-ap-tag configuration:

policy-map multi-match PoM-L47

class CM-VIP

loadbalance vip inservice

loadbalance policy PM-LB

kal-ap primary-oos

kal-ap-tag def

If the kal-ap-tag was configured earlier, the output configuration after the downgrade to A2(3.0) will be as shown in the following example:

policy-map multi-match PoM-L47

class CM-VIP

loadbalance vip inservice

loadbalance policy PM-LB

kal-ap-tag def

ACE Documentation Set

In addition to this document, the ACE documentation set includes the following publications:

Document Title
Description

Cisco Application Control Engine Module Hardware Installation Note

This guide provides information for installing the ACE into the Catalyst 6500 series switch and the Cisco 7600 series router.

Cisco Application Control Engine Module Getting Started Guide

This guide describes how to perform the initial setup and configuration tasks for the ACE.

Cisco Application Control Engine Module Administration Guide

This guide describes how to perform administration tasks on the ACE, including initial setup, establish remote access, configure class maps and policy maps, manage the ACE software, configure SNMP, define system message logging, configure redundancy, and upgrade your ACE software.

Cisco Application Control Engine Module Virtualization Configuration Guide

This guide provides instructions on how to operate your ACE in a single-context or in multiple-contexts. Multiple-contexts use the concept of virtualization to partition your ACE into multiple virtual devices or contexts.

Cisco Application Control Engine Module Routing and Bridging Configuration Guide

This guide provides instructions for configuring the routing and bridging features of the ACE. This guide provides a routing overview and describes how to perform ACE configuration tasks, including:

Configuring VLANs

Configuring routing

Configuring bridging

Configuring Address Resolution Protocol (ARP)

Configuring Dynamic Host Configuration Protocol (DHCP)

Cisco Application Control Engine Module Server Load-Balancing Configuration Guide

This guide describes how to perform ACE server load-balancing configuration tasks, including:

Server health monitoring

Real servers and server farms

Stickiness

Class maps and policy maps to load-balance traffic to real servers in server farms

Firewall load balancing

TCL scripts

Cisco Application Control Engine Module Security Configuration Guide

This guide describes how to perform ACE security configuration tasks, including:

Security access control lists (ACLs)

User authentication and accounting using a TACACS+, RADIUS, or LDAP server

Application protocol and HTTP deep packet inspection

TCP/IP normalization and termination parameters

Network address translation (NAT)

Cisco Application Control Engine Module SSL Configuration Guide

This guide describes how to perform ACE SSL configuration tasks, including:

SSL certificates and keys

SSL initiation

SSL termination

End-to-end SSL

Cisco Application Control Engine Module System Message Guide

Describes how to configure system message logging on the ACE. This guide lists and describes the system log messages generated by the ACE.

Cisco Application Control Engine Module Command Reference

This reference provides an alphabetical list of all command line interface (CLI) commands including syntax, options, and related commands.

Cisco CSM-to-ACE Conversion Tool User Guide

Describes how to use the CSM-to-ACE conversion tool to migrate Cisco Content Switching Module (CSM) running-configuration or startup-configuration files to the ACE.

Cisco CSS-to-ACE Conversion Tool User Guide

Describes how to use the CSS-to-ACE conversion tool to migrate Cisco Content Services Switches (CSS) running-configuration or startup-configuration files to the ACE.

Cisco Application Control Engine (ACE) Module Troubleshooting Guide, Release A2(x)

Describes the procedures and methodology in wiki format to troubleshoot the most common problems that you may encounter during the operation of your ACE.


Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS Version 2.0.