Table Of Contents
Release Note for the Cisco Application Control Engine Module
Ordering an Upgrade License and Generating a Key
Upgrade Path for ACE Software Version 3.0(0)A1(6.3b)
Routing Configuration Operating Consideration
Software Version 3.0(0)A1(6.3c) Resolved Caveats and Open Caveats
Software Version 3.0(0)A1(6.3c) Resolved Caveats
Software Version 3.0(0)A1(6.3c) Open Caveats
Software Version 3.0(0)A1(6.3b) Open Caveats and Resolved Caveats
Software Version 3.0(0)A1(6.3b) Open Caveats
Software Version 3.0(0)A1(6.3b) Resolved Caveats
Software Version 3.0(0)A1(6.3a) Open Caveats and Resolved Caveats
Software Version 3.0(0)A1(6.3a) Open Caveats
Software Version 3.0(0)A1(6.3a) Resolved Caveats
Software Version 3.0(0)A1(6.3) Open Caveats, Resolved Caveats, and Command Changes
Software Version 3.0(0)A1(6.3) Open Caveats
Software Version 3.0(0)A1(6.3) Resolved Caveats
Software Version 3.0(0)A1(6.3) Command Changes
Software Version 3.0(0)A1(6.2a) Operating Considerations
Software Version 3.0(0)A1(6.2a) Open Caveats
Software Version 3.0(0)A1(6.2a) Resolved Caveats
Software Version 3.0(0)A1(6.2a) Command Changes
Software Version 3.0(0)A1(6.1) Open Caveats and Resolved Caveats
Software Version 3.0(0)A1(6.1) Open Caveats
Software Version 3.0(0)A1(6.1) Resolved Caveats
Operating Considerations for 3.0(0)A1(5a)
Software Version 3.0(0)A1(5a) Open Caveats
Software Version 3.0(0)A1(5a) Resolved Caveats
Software Version 3.0(0)A1(5a) Command Changes
Software Version 3.0(0)A1(4a) Open Caveats
Software Version 3.0(0)A1(4) Open Caveats, Resolved Caveats, and Command Changes
Software Version 3.0(0)A1(4) Open Caveats
Software Version 3.0(0)A1(4) Resolved Caveats
Software Version 3.0(0)A1(4) Command Changes
Software Version 3.0(0)A1(3b) Resolved PSIRT
Obtaining Documentation and Submitting a Service Request
Release Note for the Cisco Application Control Engine Module
June 20, 2008
Note
The most current Cisco documentation for released products is also available on Cisco.com.
Contents
This release note applies to the following software versions for the Cisco Application Control Engine Module (ACE), models ACE10 (ACE10-6500-K9) and ACE20 (ACE20-MOD-K9):
•
3.0(0)A1(6.3c)
•
3.0(0)A1(6.3b) (See the "Upgrade Path for ACE Software Version 3.0(0)A1(6.3b)" section.)
•
3.0(0)A1(6.3a)
•
3.0(0)A1(6.3)
•
3.0(0)A1(6.2a)
•
3.0(0)A1(6.1)—Safe Harbor release.
•
3.0(0)A1(5a)
•
3.0(0)A1(4a)—Introduces support for the ACE20 module. This version also provides support for the ACE10 module on the Cisco 7600 series router.
•
3.0(0)A1(4)
•
3.0(0)A1(3b)
Note
Software version 3.0(0)A1(6.2) is no longer available for download. To update your ACE module, use software version 3.0(0)A1(6.2a) or greater. For information regarding version 3.0(0)A1(6.2), refer to the field notice at http://www-tac.cisco.com/Support_Library/field_alerts/fn63040.html.
The ACE10 and the ACE20 are module options for the Catalyst 6500 series switch and require Cisco IOS Release 12.2(18)SXF4 or later with a Multilayer Switch Feature Card (MSFC3) and the following models of Supervisor 720 engines: WS-SUP720, WS-SUP720-3B, and WS-SUP720-3BXL.
The ACE10 and the ACE20 modules require Cisco IOS Release 12.2(33)SXH or later running on a Catalyst 6500 series switch with a Multilayer Switch Feature Card (MSFC3) and the following models of Supervisor 720-10GE engines: VS-S720-10G-3C and VS-S720-10G-3CXL.
The ACE10 and the ACE20 are also module options for the Cisco 7600 series router and require either Cisco IOS Release 12.2(18)SXF4 or 12.2(33)SRB or later with the following models of Supervisor engines: WS-SUP720, WS-SUP720-3B, or WS-SUP720-3BXL. Note that Cisco IOS Release 12.2(33)SXH runs only on the Catalyst 6500 series switch, therefore, the Supervisor 720-10GE engines are not supported in the Cisco 7600 series router.
The ACE10 and the ACE20 modules require Cisco IOS Release 12.2(33)SRC or later running on a Cisco 7600 series router with the Cisco Route Switch Processor (RSP) 720.
For information on the ACE features and commands, refer to the ACE documentation located at: http://www.cisco.com/en/US/products/ps6906/tsd_products_support_model_home.html.
This release note contains the following sections:
•
Ordering an Upgrade License and Generating a Key
•
Upgrade Path for ACE Software Version 3.0(0)A1(6.3b)
•
Routing Configuration Operating Consideration
•
Software Version 3.0(0)A1(6.3c) Resolved Caveats and Open Caveats
•
Software Version 3.0(0)A1(6.3b) Open Caveats and Resolved Caveats
•
Software Version 3.0(0)A1(6.3a) Open Caveats and Resolved Caveats
•
Software Version 3.0(0)A1(6.3) Open Caveats, Resolved Caveats, and Command Changes
•
Software Version 3.0(0)A1(6.1) Open Caveats and Resolved Caveats
•
Software Version 3.0(0)A1(4a) Open Caveats
•
Software Version 3.0(0)A1(4) Open Caveats, Resolved Caveats, and Command Changes
•
Software Version 3.0(0)A1(3b) Resolved PSIRT
•
Obtaining Documentation and Submitting a Service Request
ACE Software Features
The ACE performs high-performance server load balancing (SLB) among groups of servers, server farms, firewalls, and other network devices, based on Layer 3 as well as Layer 4 through Layer 7 packet information. The ACE can also terminate and initiate SSL-encrypted traffic, which enables it to perform intelligent load balancing while ensuring secure end-to-end encryption. The module is capable of internetworking speeds of 4 Gigabits per second (Gbps) by default, and can achieve speeds of 8 Gbps with the purchase of an upgrade license.
The Cisco ACE provides the following features, functions and benefits:
•
Application infrastructure control—Virtual partitioning provides a way to create resource segmentation and isolation, allowing the ACE to act like it is several individual virtual modules within a single physical module.
•
Centralized control with decentralized management using template-based or customizable user permissions for each virtual partition—The Role-Based Access Control (RBAC) feature allows you to specify administrative roles and restrict administrators to specific functions within the module or virtual partitions.
•
Multitiered redundancy, availability, and scalability—Provides maximum protection for your critical business by offering three types of high availability. All of these availability modes provide rapid, stateful application redundancy with replication of connection state and sticky tables.
–
Inter-chassis: An ACE in one Catalyst 6500 series switch is protected by an ACE in a peer Catalyst 6500 series switch.
–
Intra-chassis: An ACE in a Catalyst 6500 series switch is protected by another ACE in the same Catalyst 6500 series switch (the Catalyst 6500 series switch has redundancy built in).
–
Inter-partition: The ACE supports high availability between virtual partitions configured across two modules to allow specific partitions to fail over without affecting the other partitions and applications on a given module.
•
Protects the data center and critical applications from malicious traffic with the following features:
–
HTTP deep packet inspection-HTTP header, URL, and payload
–
Bidirectional Network Address Translation (NAT) and Port Address Translation (PAT)
–
Support for static, dynamic and policy-based NAT/PAT
–
Access control lists (ACLs) to selectively allow traffic between ports
–
TCP connection state tracking
–
Virtual connection state for User Datagram Protocol (UDP)
–
Sequence number randomization
–
TCP header validation
–
TCP window size checking
–
Unicast Reverse Path Forwarding (URPF) checking at session establishment
•
Integrated hardware-accelerated protocol control— Efficient inspection and filtering of data center protocols such as HTTP, Real-Time Streaming Protocol, Domain Name System, FTP, and ICMP.
•
Large, scalable ACLs with a maximum of 256,000 elements— Permits both front-end scalability (number of users/client applications) and back-end scalability (number of servers/server farms).
•
SSL acceleration—The Cisco ACE solution integrates SSL acceleration technology to offload the encryption and decryption of SSL traffic from external devices, thereby allowing the Cisco ACE to look deeper into encrypted data and apply security and content switching policies.
Available ACE Licenses
By default, the ACE supports virtualization with one Admin context and five user contexts, 4 gigabits per second (Gbps) module bandwidth, and 1,000 SSL transactions per second (TPS). You can increase the number of default user contexts, module bandwidth, and SSL TPS by purchasing the following licenses:
•
ACE-VIRT-020—20 virtual contexts.
•
ACE-VIRT-050—50 virtual contexts.
•
ACE-VIRT-100—100 virtual contexts.
•
ACE-VIRT-250—250 virtual contexts.
•
ACE-08G-LIC—8 Gbps bandwidth (available when you initially purchase an ACE).
•
ACE-16G-LIC—16 Gbps bandwidth
If you purchase an ACE with a bandwidth of 4 Gbps, you can upgrade the module bandwidth to:
–
8 Gbps by using the ACE-UPG1-LIC license
–
16 Gbps by using the ACE-UPG2-LIC license.
•
ACE-SSL-5K-K9—SSL with 5,000 TPS.
•
ACE-SSL-10K-K9—SSL with 10,000 TPS (3.0(0)A1(3) or later).
•
ACE-SSL-15K-K9—SSL with 15,000 TPS (3.0(0)A1(3) or later).
You can upgrade virtualization in increments, provided that you do not exceed the limits of the ACE (a maximum of 250 contexts), by using the following licenses:
•
ACE-VIRT-UP1—upgrades 20 to 50 contexts
•
ACE-VIRT-UP2—upgrades 50 to 100 contexts
•
ACE-VIRT-UP3—upgrades 100 to 250 contexts
You can upgrade SSL in 5,000 TPS increments up to a maximum of 15,000 TPS by using the following SSL upgrade licenses:
•
ACE-SSL-UP1-K9—Upgrades SSL from 5,000 TPS to 10,000 TPS (3.0(0)A1(3) or later).
•
ACE-SSL-UP2-K9—Upgrades SSL from 10,000 TPS to 15,000 TPS (3.0(0)A1(3) or later).
You can also obtain an ACE demo licence for each type of virtualization, bandwidth, or SSL TPS licence, including upgrade increments for contexts. A demo license is valid for only 60 days. At the end of this period, you will need to update the demo license with a permanent license to continue to use the ACE software. To view the expiration of the demo license, use the show license usage command in Exec mode. If you need to replace the ACE module, you can copy and install the licenses onto the replacement module.
Note
You can access the license and show license commands only in the Admin context. You must have the Admin role in the Admin context to perform the tasks of installing, removing, and updating the license.
Ordering an Upgrade License and Generating a Key
This section describes the process to order an upgrade license and to generate a license key for your ACE. To order an upgrade license:
1.
Order one of the licenses from the list in the "Available ACE Licenses" section using any of the available Cisco ordering tools on Cisco.com.
2.
When you receive the Software License Claim Certificate from Cisco, follow the instructions that direct you to the Cisco.com website. As a registered user of Cisco.com, go to this URL:
http://www.cisco.com/go/license
3.
Enter the Product Authorization Key (PAK) number found on the license certificate as your proof of purchase.
4.
Provide all the requested information to generate a license key.
5.
After the system generates the license key, you will receive a license key e-mail with an attached license file and installation instructions. Save the license key e-mail in a safe place in case you need it in the future (for example, to transfer the license to another ACE).
For information about installing and managing ACE licenses, refer to Chapter 3, Managing ACE Software Licenses, in the Cisco Application Control Engine Module Administration Guide.
Upgrade Path for ACE Software Version 3.0(0)A1(6.3b)
We recommend that you do not attempt to upgrade from version 3.0(0)A1(6.3b) to version A2(1.0). Instead, the upgrade path for version 3.0(0)A1(6.3b) will be the version A2(1.1) maintenance release when it becomes available.
In a redundant configuration during an upgrade from ACE software version 3.0(0)A1(6.3b) to version A2(1.0), if the active ACE is running version 3.0(0)A1(6.3b) and the standby ACE is running version A2(1.0), redundancy does not function properly. The reason for this behavior is that the standby ACE does not detect the activeACE as being compatible. The result of this incompatibility is that the standby ACE never receives the replicated configuration from the active ACE and connection replication fails. The standby ACE never reaches the STANDBY_HOT state, which negates the intent of redundancy.
As a workaround, if you must upgrade (in an emergency) from version 3.0(0)A1(6.3b) to version A2(1.0) before version A2(1.1) is available, then you must reload both ACEs and then boot each ACE with version A2(1.0).
Routing Configuration Operating Consideration
The ACE requires a route back to the client before it can forward a request to a server. If the route back is not present, the ACE cannot establish a flow and drops the client request. Make sure that you configure the appropriate routing to the client network on the ACE VLAN where the client traffic enters the ACE module.
ACE Documentation Set
In addition to this document, the ACE documentation set includes the following publications:
Software Version 3.0(0)A1(6.3c) Resolved Caveats and Open Caveats
This ACE software release contains a single software change that has limited customer applicability. This change supports the use of the backslash (\) character in usernames for remote authentication only (not local). If you are looking for more features and resolved caveats, we recommend that you consider a recent release of the ACE A2(1.x) software series (for example, A2(1.1)). For more information about software version A2(1.1), see the following URL:
The following sections contain the resolved caveat and the open caveats in software version 3.0(0)A1(6.3c):
•
Software Version 3.0(0)A1(6.3c) Resolved Caveats
•
Software Version 3.0(0)A1(6.3c) Open Caveats
Software Version 3.0(0)A1(6.3c) Resolved Caveats
The following resolved caveat applies to software version 3.0(0)A1(6.3c):
•
CSCsq69858—When the ACE is configured for TACACS+ authentication, a username that is defined on the TACACS server with a backslash (\) does not work because the ACE interprets the backslash character as a Tab character.
Software Version 3.0(0)A1(6.3c) Open Caveats
The open caveats for software version 3.0(0)A1(6.3c) are the same as those for software version 3.0(0)A1(6.3b). For details, see the "Software Version 3.0(0)A1(6.3b) Open Caveats" section.
Software Version 3.0(0)A1(6.3b) Open Caveats and Resolved Caveats
The following sections contain the open and resolved caveats and command changes in software version 3.0(0)A1(6.3b):
•
Software Version 3.0(0)A1(6.3b) Open Caveats
•
Software Version 3.0(0)A1(6.3b) Resolved Caveats
Software Version 3.0(0)A1(6.3b) Open Caveats
The following open caveats apply to software version 3.0(0)A1(6.3b):
•
CSCsh70258—SSL initiation configured on an ACE may fail when it is used with Microsoft IIS 6.0 that is configured to accept client certificates. The additional SSL communication from the server to the ACE causes the ACE to experience an internal error. Workaround: In the IIS Web Server Properties > Directory Security > Edit menu, specify Ignore client certificates.
•
CSCsj41909—The ACE fails with the following message appearing in the syslog:
%ACE-5-199006: Orderly reload started at xxxxxxx by System.The reload reason is NP 1 Failed: NP Process Crashed. Workaround: None
•
CSCsj64833—The network utility traceroute does not work for a configured ACE IP interface if the underlying protocol is UDP or TCP. Workaround: ICMP traceroute will work. The default of most traceroute utilities is UDP so a command line option might be necessary. For Linux, use the -I option.
•
CSCsj68643—The following log messages may appear sporadically in the ACE log:
–
can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.
–
can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.
These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.
•
CSCsj70522—In a redundant configuration with multiple contexts and logging enabled for the console and the buffer on both ACEs, log messages appear only on the standby console when a probe to a server failed. No new messages are observed in the buffers on either ACE. After entering the changeto command to another context and then back to the original context, log messages appear on the active ACE console, but the log buffer is not updated. Workaround: None.
•
CSCsj74250—When you configure the TACACS server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, there is a key mismatch when attempting to authenticate. Workaround: Paste the properly encrypted key into the running-config.
•
CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, then subsequently try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in RSA format instead of RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.
•
CSCsj94366—While attempting to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: console configuration can only be done on console. Workaround: None.
•
CSCsk04298—When a switchover occurs in a Layer 7 load-balancing persistence-rebalance configuration, connections with multiple GETs may not proceed. Workaround: None.
•
CSCsk15979—With UDP and TCP class maps sharing the same VIP and both having the idle timer set to infinite, a series of connections are made and allowed to sit idle for several hours. When a change is made to the UDP class map to time out these connections in 60 seconds, the ACE clears the TCP and UDP connections. Workaround: None.
•
CSCsk16083—Changing the file used for hash probes does not fail the probe if the server is a Microsoft Windows 2000 server. Workaround: None.
•
CSCsk34767—CSCsh63231 added FTP inspection statistics to the show stats inspect command output, but the statistics cannot be cleared using the clear stats inspect command. Workaround: None.
•
CSCsk34843—The following messages may be observed on the console connection:
"Data bus error, epc == 2b5008e0, ra == 2b5008c8badvaddr 0x00005f00 cause 0x8cd8fd18 status 0x00007fff<1>A_SCD_BUS_ERR_STATUS 0x810c0000A_BUS_ERR_DATA_0 0x0A_BUS_ERR_DATA_1 0x0A_BUS_ERR_DATA_2 0xffffb12a00000000A_BUS_ERR_DATA_3 0xffff000000000000A_BUS_L2_ERRORS 0x0A_BUS_MEM_IO_ERRORS 0x20000 "Workaround: None.
•
CSCsk36611—If you are using Internet Explorer (IE), an SSL rehandshake may fail if the total length of the SSL certificate chain is greater than 4024 bytes. When this condition exists, the ACE creates two SSL records. The first record has a total length field indicating 4024 bytes, but containing a certificate item with a specified length greater than 4024 bytes. The second record contains a new SSL record header and the remaining portion of the previous SSL record. Workaround: Use Firefox or another browser.
•
CSCsk44035—When persistence-rebalance is enabled and there is an extremely high load on the ACE, buffer allocation failures and slow end-user responses may occur, and the ACE may reboot. Workaround: None.
•
CSCsk53147—When you configure SIP inspection, set up SIP calls at approximately 125 connections per second (cps), and enable system logging with console logging, the ACE may become unresponsive. This behavior can also happen with other high-rate data plane syslogs such as HTTP inspection. Workaround: Do not enable console logging when high-rate data plane syslogs are configured.
•
CSCsk54917—The ACE may reboot when there are a significant number of concurrent connections (for example, 200,000 open connections) in use and you have the following functions configured on the ACE:
–
HTTP inspection
–
IP address sticky group in an SLB policy
–
Persistence rebalance
The ACE reboots because multiple Micro Engines become unresponsive in the IXP network processor on the ACE. Workaround: None.
•
CSCsk61805—UDP probes may stay alive even when the ACE receives ICMP host unreachable messages. This behavior applies only to host unreachable messages. The UDP probe behavior is correct with port unreachable messages. Workaround: None.
•
CSCsk63774—Current connection statistics in the show serverfarm name command output may not coincide with the show serverfarm command output. This behavior may exist even with a minimal amount of traffic. Workaround: None.
•
CSCsk68396—In a firewall load-balancing configuration where the firewalls are positioned between two ACEs and you configure the mac-sticky command on the ACEs, an ICMP error packet from a router located between the second ACE and the real server is forwarded to a firewall that is different from the firewall that received the original IP packet that generated the ICMP error. Workaround: None.
•
CSCsk72920, CSCsk70822—Heavy SSL traffic passing through the ACE when operating in an end-to-end SSL configuration may cause a particle buffer to have a reference count of zero, causing the module to reboot. Workaround: None.
•
CSCsk74766—While copying and pasting the commands necessary to configure sticky server farms for a load-balancing policy map, the ACE may display the following message: Error: Add link failed! This behavior is intermittent. Workaround: Manually configure the commands needed to add the sticky server farm to the load-balancing policy map instead of copying and pasting the commands.
•
CSCsk95828—Checkpoint rollback issues occur with the banner and hostname or SSH keys. Workaround: None.
•
CSCsl15321—After reloading with an HTTP inspection configuration, the ACE drops HTTP packets even when the module is configured to allow them to pass. Workaround: Reapply the configuration.
•
CSCsl32269—After configuring a VIP while running in one-arm mode, you may observe that the traffic to the VIP fails and the ACE resets the connection after completing the back-end three-way handshake. Workaround: None. To resolve this condition, you must reboot the ACE.
•
CSCsl35473—Because the DNS inspection hash table entry timeout is fixed at approximately 100 seconds, when DNS inspection is configured and there are many unanswered requests, the hash table may become full. When the table is full, the ACE stops inspecting new DNS traffic. Workaround: None.
•
CSCsl36877—The ACE may become unresponsive under the following conditions:
–
End-to-end SSL configuration
–
Traffic rate of 1000 TPS or more
–
Certificates and keys of any size
–
Variable page size of 1 KB to 8 KB (occurs more frequently) or a fixed page size of 8 KB
On the ACE console, you may see messages similar to the following:
–
n2_intr: ARB_UNIT_ERR 0x00000008
–
n2_intr: EFL_IND_RPTR 0x000327fc
–
n2_intr: EFL_IND_RPTR_TYPE 0x00000120
Workaround: None
•
CSCsl38419— Modifying or changing the SSL proxy under a Layer 7 policy map does not take effect. Workaround: Delete the Layer 3 service policy from the interface and then re-add it.
•
CSCsl42531—If you change the load-balancing predictor from least connections to round robin or the opposite with traffic running, the load-balancing receive queue (lbrx) may get stuck, which may cause the ACE to reload. This behavior may occur when servers are transitioning between the MAX-CONNS and OPERATIONAL states. Workaround: Do not change the load-balancing predictor configuration several times in a row with traffic running and servers transitioning in and out of the MAX-CONNS state.
•
CSCsl46334—With a high rate of Layer 7 load-balanced traffic flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse configured, the ACE may start dropping traffic after a few hours. Workaround: None.
•
CSCsl57828—With load-balanced TCP flows running through the ACE, the module may become unresponsive and may eventually reset if the client or the server advertises a maximum segment size (MSS) value that is less than 1460 bytes (the ACE default value) via the MSS option or if the client or the server does not send an MSS value. You may observe this behavior when the ACE configuration includes the following features:
–
Layer 4 load balancing and HTTP inspection
–
Layer 7 load balancing and the persistence rebalance command
Workaround: Configure the following Layer 4 connection parameter-map and associate it with the Layer 4 multi-match policy map as follows:
parameter-map type connection TCPset tcp wan-optimization rtt 0policy-map multi-match TESTclass TESTconnection advanced-options TCP•
CSCsl60440—When the predictor leastconns, server-conn reuse, and persistence-rebalance commands are configured, if successive GET requests result in the same policy being selected, then the ACE does not increment the connection counts for the real server and does not set up predictor weight values. However, when the server-side connection closes, the ACE incorrectly performs cleanup on every REUSE CLOSE message, which results in negative weights for the real servers and incorrect traffic distribution for real servers in the server farm. Workaround: Disable either the persistence-rebalance or the server-conn reuse command.
•
CSCsl62013—Entering the show logging internal command may cause the syslog daemon (syslogd) to perform a core dump, which, in turn, may cause the ACE module to reboot. Workaround: None.
•
CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health check(s). Workaround: None.
•
CSCsl74755—ACE sockets may become depleted when you repeatedly (approximately 250 attempts) access the ACE using SSH. For example, entering the ssh admin@192.168.1.123 sh ver command may cause the ACE to generate the following message to the client:
Received disconnect from 192.168.1.123: 2: Could not create socket pairs: Too many open files in system.In some cases, this condition generates the following message on the ACE console:
socket: Limiting socket usage, no more sockets, Max allowed is 512 and current_usage is 512Workaround: Avoid excessive SSH logins. After the condition occurs, you must reboot the ACE to clear the resource depletion. No other workaround is currently available.
•
CSCsl77474—Configuring the shared-vlan-hostid 16 command causes any interfaces configured on shared VLANs using that ID to become unreachable. These interfaces are in the various user contexts on the ACE, and failing traffic includes ARPs, probes, and so on. In this case, the interface address becomes completely unreachable. Any interface created on a shared VLAN while the shared-vlan-hostid 16 command is enabled is also unreachable. The show arp command displays a MAC address of 00:12:43:dc:ab:00 or higher on such interfaces. Workaround: Do not configure the shared-vlan-hostid 16 command. You can use any of the other available host IDs (1 through 15) for the shared-vlan-hostid command.
The ACE randomly assigns a host ID to shared VLANs if you do not configure a host ID. The ACE never choose a host ID of 16 when performing this random assignment. You will observe this behavior only if you configure the shared-vlan-hostid 16 command on your ACE.
•
CSCsl82712—When the ACE is configured for Layer 7 load balancing, an HTTP method request that exceeds the default (2000 bytes) or configured maximum HTTP header parse length is dropped and a TCP reset is sent to the client. Workaround: Increase the maximum HTTP header parse length in a parameter map using the set header-maxparse-length command.
•
CSCsl89772—With a UDP probe configured on port 69 (TFTP), the concurrent connections counter of the show resource usage command output keeps incrementing. Workaround: None.
•
CSCsl95565—When you are using SNMP with notifications enabled in multiple contexts and redundancy is configured, the standby ACE may reboot with a last reboot reason of service snmpd. When the ACE comes back up, it may be in an odd state with some of the contexts not functioning properly. Workaround: None.
•
CSCsm03018—When you enable logging for syslog message IDs 302022 to 302025, changing the logging level for syslog messages 302022 to 302025 from Level 6 to Level 4 also affects the logging level for messages 302028 to 3020311. Logging rate limiting does not work for message IDs 302028 to 302031 and you cannot change the logging level for messages 302028 to 302031directly because these syslogs are not supported by the relevant CLI commands. Workaround: None.
•
CSCsm10896—Enabling the persistence rebalance command while a server farm is configured with any of the hash-based load-balancing predictors may cause the ACE to ignore the hash-based predictor after the first HTTP GET request. Instead, the ACE continues to load balance requests to the same real server that it previously selected if the requests match the same Layer 7 policy. Otherwise, the ACE load balances requests to a new real server in the server farm. Workaround: None.
•
CSCsm12883—The ACE resets connections when the number of long-lived persistent HTTP connections is one less than the configured maximum value. Workaround: None.
•
CSCsm18847—If you change the logging level for syslog messages 302028 to 302031 when they are enabled using the logging fastpath command, the ACE still uses level 6 to rate-limit these syslogs. Workaround: None.
•
CSCsm35407—After you roll back a large ACE configuration to a blank context and enter the show running-config command, the ACE may perform a core dump. The core dump does not cause the ACE to reload and there is no adverse impact to the module. Workaround: None.
•
CSCsm43541—The ACE may become unresponsive because of a buffer corruption issue on the crypto chip that is specifically related to SSL traffic. Workaround: None.
•
CSCso09642—When the active ACE, which is configured with a single context in a redundant configuration, reboots, the standby ACE becomes active and reboots. Then, the active ACE (previously the standby) reboots. The last boot reason on both units is >NP 0 Failed : NP ME Hung. This behavior can cause a core dump file to be generated.
The running-configuration file contains commands similar to the following:
policy-map multi-match FIRST_POLICYclass FIRST_DATAloadbalance vip inserviceloadbalance policy WHATEVERnat dynamic 1 vlan 100interface vlan 100nat pool 1 10.0.0.1 10.0.0.3 netmask 255.255.255.0nat pool 1 10.0.0.58 10.0.0.83 netmask 255.255.255.0NAT is configured without PAT on multiple ranges. This issue appears to occur very rarely even when using this configuration on affected sites. Workaround: None.
•
CSCso12223—Source (client) NAT operations stop working on the ACE and the show nat-fabric policies command output displays invalid entries in the List of NAT object IDs field. Workaround: Reload the affected ACE module.
•
CSCso12722—When you configure the ACE for SSL termination and a client sends a POST request, the request does not fully traverse the ACE to the real server. This behavior results in a timeout from the real server, which is waiting for the rest of the POST. Workaround: None.
•
CSCso13099—The ACE does not adjust the time correctly to Daylight Saving Time, even though the supervisor engine clock was adjusted correctly. When Daylight Saving Time occurs, the ACE does not update its clock to the correct time zone. Workaround: Enter the following command:
host1/Admin(config)# clock timezone CDT -5 0•
CSCso15332—The ACE logs the probe failed syslog message for the wrong real server in the wrong context. The first context configuration includes the following commands:
context 1logging host 10.0.0.1 udp/514probe https PROBE1probe http PROBE2rserver RS1ip address 192.168.1.1inserviceserverfarm SF1probe PROBE1rserver RS1inserviceserverfarm SF2probe PROBE2rserver RS2inserviceThe second context configuration includes the following commands:
context 2logging host 10.0.0.1 udp/514probe http C2PROBE2-8880port 8880rserver RS100ip address 172.16.1.1inserviceserverfarm SF100probe C2PROBE2-8880rserver RS100inserviceThe ACE is configured to do the following:
•
Use the same syslog host in two contexts
•
Use HTTP and HTTPS probes in both contexts
•
Log probe failures in both contexts
Syslog messages that describe port 8880 probe failures for 192.168.1.1 are received on the syslog host from context 2. The real servers in the second context are unavailable when the behavior begins, and later, those real servers become available and the incorrect logging stops. Workaround: None.
•
CSCso18391—When you perform a checkpoint rollback and enter the no nat dynamic n vlan nnn CLI command, the ACE displays the following messages:
Generating configuration....Errors while applying diff, please see log below.Failed Command:policy-map multi-match xxxxxclass xxxxxxno nat dynamic 1 vlan 150Failure Reason:Error: Called API timed outThis behavior appears to be related to larger configurations with many regular expressions (regexes). Workaround: Enter the no nat dynamic command before you perform the rollback operation.
•
CSCso25654—With the UDP probe interval set to 2 seconds, UDP probes take longer than expected to enter the failed state. Workaround: Use a time interval that is greater than or equal to 5 seconds for UDP probes.
•
CSCso28789—A syslog server that is receiving messages from the ACE fails to display messages that are formatted incorrectly. A few ACE syslog messages are missing a colon after the syslog message ID number. An example of an incorrectly formatted message is as follows:
%ACE-3-251008 Health probe failed for server....The format should include a colon after the ID number as follows:
%ACE-3-251008: Health probe failed for server....Workaround: View the local syslog buffer on the ACE by using the show logging history command.
Software Version 3.0(0)A1(6.3b) Resolved Caveats
The following resolved caveats apply to software version 3.0(0)A1(6.3b):
•
CSCsj21032—The output of the show ft group status command shows that the FT group is in the STANDBY_CONFIG state for more than an hour. The FT group eventually moves to the STANDBY_HOT state. A very large configuration in a context can increase the time that it takes to complete a configuration synchronization. If your system has a large number of contexts and some of them have very large configurations, the overall configuration synchronization time for all contexts to complete can be lengthy. Workaround: None. Wait for the configuration synchronization operation to finish.
•
CSCsj29467—UDP traceroute traffic and other traffic destined to an ACE VIP that is not part of a local subnet may cause forwarding loops until the time to live (TTL) expires. If you define an ACE VIP (class map) that is not part of a local subnet, you configure the class map for a specific TCP port or any TCP port, and you perform a UDP traceroute to this VIP, the ACE forwards this traffic to its default gateway. The default gateway, via a static route or a routing protocol, will attempt to forward the request back to the ACE, which may cause forwarding loops. This behavior is observed for VIPs that are configured with an IP address, a protocol, and a port and is not seen for VIPs that have only an IP address.
Perform one of the following workarounds:
•
Configure the ACE VIP with any protocol (match virtual-address 192.168.12.15 any)
•
Configure an additional match statement with UDP and a specific port (match virtual-address 192.168.12.15 udp eq sip)
•
Configure UDP and any port (match virtual-address 192.168.12.15 udp any)
Each of these workarounds cause the VIP to respond to the UDP traceroute with an ICMP Dest Port Unreachable message. If all real servers are down, there will be no response to the traceroute.
Alternatively, you can define an ACL to block all UDP traffic destined to the TCP VIP. This workaround causes the UDP traceroute to fail or to go unanswered.
•
CSCsk02170—If you configure a less-specific route before a more-specific route, ACE-originated traffic may not use the more specific route. For example, if you configure the following routes in the order shown:
Admin/host(config)# ip route 1.0.0.0 255.0.0.0 10.0.0.1Admin/host(config)# ip route 1.1.1.1 255.255.255.255 192.168.0.1The output of the show ip route command displays the route for 1.1.1.1, but the ACE may continue to use the 10.0.0.1 next hop. This behavior is dependent on the order in which the routes are configured on the ACE and affects ACE-originated traffic only.
Workaround: To restore routing as expected from the show ip route command output, perform the following steps:
1.
Configure a less-specific dummy route.
2.
Remove the less-specific route.
This sequence does not affect the routing table, but it does cause the cache to refresh, which corrects the routing problem.
•
CSCsk63774—Current connection statistics in the show serverfarm name command output may not coincide with the show serverfarm command output. This behavior may exist even with a minimal amount of traffic. Workaround: None.
•
CSCsl34397—When the ACE is running in bridge mode and SNMP notifications are enabled, the ACE may reboot with an SNMP daemon (snmpd) core dump. Workaround: None.
•
CSCsl82712—When the ACE is configured for Layer 7 load balancing, an HTTP method request that exceeds the default (2000 bytes) or configured maximum HTTP header parse length is dropped and a TCP reset is sent to the client. Workaround: Increase the maximum HTTP header parse length in a parameter map using the set header-maxparse-length command.
•
CSCsl95565—When you are using SNMP with multiple contexts and redundancy is configured, the standby ACE may reboot with a last reboot reason of service snmpd. When the ACE comes back up, it may be in an odd state with some of the contexts not functioning properly. Workaround: None.
•
CSCsm10390—An ACE process produced a core dump, but the compressed core dump file was not saved in the core: directory for analysis. When the ACE was reloaded, the generated core file was lost, leaving no way to analyze the problem that caused the reload. This behavior was a single occurrence that may be related to a previously failed ephemeral process such as a script or a shell invocation that had occurred over the previous several months of uninterrupted operation of the ACE module. Workaround: None.
•
CSCsm17268—In a redundant configuration, when you add a single line to an existing configuration (for example, Admin(config)# snmp community public ro) and the ACE performs a subsequent incremental synchronization, the snmp community command appears in the running-config on the peer, but the process seems to cause an internal failure that causes the peer transition to the STANDBY_COLD state. The running-config sync status is reported as "Peer in Cold State. Incremental Sync Failure: snmp config sync to sby failed." Workaround: If possible, avoid the use of the snmp command.
•
CSCsm37228—After executing TACACS related commands on the active ACE in a redundant configuration, the tacacs process fails on the standby device (both contexts), resulting in an ACE reload. Workaround: None.
•
CSCsm40004—When you are creating a certificate signing request (CSR) at the ACE CLI using the csr-generate command, you cannot use the space character in the State value. Workaround: Use the state abbreviation.
•
CSCsm52480—All IPv6 multicast packets are dropped even though the ACE module is configured as follows:
access-list acl_name ethertype permit ipv6This symptom occurs only with IPv6 multicast packets. It does not occur with IPv6 unicast packets. Workaround: None.
•
CSCsm90293—With an SMTP probe configured on the ACE, a new mail server rejects the probe as syntactically invalid because of the use of an underscore (_) that breaks the new rules in RFC 2821 and causes the probed server to never becomes active.
The probe configuration is as follows:
probe smtp SMTPPROBEdescription Generic SMTP applicationprobe interval 30passdetect interval 30receive 30expect status 220 220This behavior is not observed with older mail servers that still adhere to or permit the SMTP command arguments accepted by RFC 821. Workaround: None.
•
CSCso02922—If the disk0: directory of the Admin context consumes all the ACE flash disk space, an SSL certificate cannot be imported into a user context. As a result, some of the directories that are needed for a context to work cannot be created.
Workaround:
1.
Free up some flash disk space by deleting unnecessary files in the disk0: directory.
2.
Remove and readd the context that was impacted.
Software Version 3.0(0)A1(6.3a) Open Caveats and Resolved Caveats
The following sections contain the open and resolved caveats and command changes in software version 3.0(0)A1(6.3):
•
Software Version 3.0(0)A1(6.3a) Open Caveats
•
Software Version 3.0(0)A1(6.3a) Resolved Caveats
Software Version 3.0(0)A1(6.3a) Open Caveats
The following open caveats apply to software version 3.0(0)A1(6.3a):
•
CSCsh70258—SSL initiation configured on an ACE may fail when it is used with Microsoft IIS 6.0 that is configured to accept client certificates. The additional SSL communication from the server to the ACE causes the ACE to experience an internal error. Workaround: In the IIS Web Server Properties > Directory Security > Edit menu, specify Ignore client certificates.
•
CSCsj41909—The ACE fails with the following message appearing in the syslog:
%ACE-5-199006: Orderly reload started at xxxxxxx by System.The reload reason is NP 1 Failed: NP Process Crashed. Workaround: None
•
CSCsj64833—The network utility traceroute does not work for a configured ACE IP interface if the underlying protocol is UDP or TCP. Workaround: ICMP traceroute will work. The default of most traceroute utilities is UDP so a command line option might be necessary. For Linux, use the -I option.
•
CSCsj68643—The following log messages may appear sporadically in the ACE log:
–
can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.
–
can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.
These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.
•
CSCsj70522—In a redundant configuration with multiple contexts and logging enabled for the console and the buffer on both ACEs, log messages appear only on the standby console when a probe to a server failed. No new messages are observed in the buffers on either ACE. After entering the changeto command to another context and then back to the original context, log messages appear on the active ACE console, but the log buffer is not updated. Workaround: None.
•
CSCsj74250—When you configure the TACACS server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, there is a key mismatch when attempting to authenticate. Workaround: Paste the properly encrypted key into the running-config.
•
CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, then subsequently try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in RSA format instead of RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.
•
CSCsj94366—While attempting to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: console configuration can only be done on console. Workaround: None.
•
CSCsk02170—If you configure a less-specific route before a more-specific route, ACE-originated traffic may not honor the more specific route. For example, if you configure the following routes in the order shown:
Admin/host(config)# ip route 1.0.0.0 255.0.0.0 10.0.0.1Admin/host(config)# ip route 1.1.1.1 255.255.255.255 192.168.0.1The output of the show ip route command displays the route for 1.1.1.1, but the ACE may continue to use the 10.0.0.1 next hop. This behavior is dependent on the order in which the routes are configured on the ACE and affects ACE-originated traffic only.
Workaround: To restore routing as expected from the show ip route command output, perform the following steps:
1.
Configure a less-specific dummy route.
2.
Remove the less-specific route.
This sequence has no net effect on the routing table, but it does cause a refresh of the cache information which corrects the routing problem.
•
CSCsk04298—When a switchover occurs in a Layer 7 load-balancing persistence-rebalance configuration, connections with multiple GETs may not proceed. Workaround: None.
•
CSCsk15979—With UDP and TCP class maps sharing the same VIP and both having the idle timer set to infinite, a series of connections are made and allowed to sit idle for several hours. When a change is made to the UDP class map to time out these connections in 60 seconds, the ACE clears the TCP and UDP connections. Workaround: None.
•
CSCsk16083—Changing the file used for hash probes does not fail the probe if the server is a Microsoft Windows 2000 server. Workaround: None.
•
CSCsk34767—CSCsh63231 added FTP inspection statistics to the show stats inspect command output, but the statistics cannot be cleared using the clear stats inspect command. Workaround: None.
•
CSCsk34843—The following messages may be observed on the console connection:
"Data bus error, epc == 2b5008e0, ra == 2b5008c8badvaddr 0x00005f00 cause 0x8cd8fd18 status 0x00007fff<1>A_SCD_BUS_ERR_STATUS 0x810c0000A_BUS_ERR_DATA_0 0x0A_BUS_ERR_DATA_1 0x0A_BUS_ERR_DATA_2 0xffffb12a00000000A_BUS_ERR_DATA_3 0xffff000000000000A_BUS_L2_ERRORS 0x0A_BUS_MEM_IO_ERRORS 0x20000 "Workaround: None.
•
CSCsk36611—If you are using Internet Explorer (IE), an SSL rehandshake may fail if the total length of the SSL certificate chain is greater than 4024 bytes. When this condition exists, the ACE creates two SSL records. The first record has a total length field indicating 4024 bytes, but containing a certificate item with a specified length greater than 4024 bytes. The second record contains a new SSL record header and the remaining portion of the previous SSL record. Workaround: Use Firefox or another browser.
•
CSCsk44035—When persistence-rebalance is enabled and there is an extremely high load on the ACE, buffer allocation failures and slow end-user responses may occur, and the ACE may reboot. Workaround: None.
•
CSCsk53147—When you configure SIP inspection, set up SIP calls at approximately 125 connections per second (cps), and enable system logging with console logging, the ACE may become unresponsive. This behavior can also happen with other high-rate data plane syslogs such as HTTP inspection. Workaround: Do not enable console logging when high-rate data plane syslogs are configured.
•
CSCsk54917—The ACE may reboot when there are a significant number of concurrent connections (for example, 200,000 open connections) in use and you have the following functions configured on the ACE:
–
HTTP inspection
–
IP address sticky group in an SLB policy
–
Persistence rebalance
The ACE reboots because multiple Micro Engines become unresponsive in the IXP network processor on the ACE. Workaround: None.
•
CSCsk61805—UDP probes may stay alive even when the ACE receives ICMP host unreachable messages. This behavior applies only to host unreachable messages. The UDP probe behavior is correct with port unreachable messages. Workaround: None.
•
CSCsk63774—Current connection statistics in the show serverfarm name command output may not coincide with the show serverfarm command output. This behavior may exist even with a minimal amount of traffic. Workaround: None.
•
CSCsk68396—In a firewall load-balancing configuration where the firewalls are positioned between two ACEs and you configure the mac-sticky command on the ACEs, an ICMP error packet from a router located between the second ACE and the real server is forwarded to a firewall that is different from the firewall that received the original IP packet that generated the ICMP error. Workaround: None.
•
CSCsk72920, CSCsk70822—Heavy SSL traffic passing through the ACE when operating in an end-to-end SSL configuration may cause a particle buffer to have a reference count of zero, causing the module to reboot. Workaround: None.
•
CSCsk74766—While copying and pasting the commands necessary to configure sticky server farms for a load-balancing policy map, the ACE may display the following message: Error: Add link failed! This behavior is intermittent. Workaround: Manually configure the commands needed to add the sticky server farm to the load-balancing policy map instead of copying and pasting the commands.
•
CSCsk95828—Checkpoint rollback issues occur with the banner and hostname or SSH keys. Workaround: None.
•
CSCsl15321—After reloading with an HTTP inspection configuration, the ACE drops HTTP packets even when the module is configured to allow them to pass. Workaround: Reapply the configuration.
•
CSCsl32269—After configuring a VIP while running in one-arm mode, you may observe that the traffic to the VIP fails and the ACE resets the connection after completing the back-end three-way handshake. Workaround: None. To resolve this condition, you must reboot the ACE.
•
CSCsl34397—When the ACE is running in bridge mode and SNMP notifications are enabled, the ACE may reboot with an SNMP daemon (snmpd) core dump. Workaround: None.
•
CSCsl35473—Because the DNS inspection hash table entry timeout is fixed at approximately 100 seconds, when DNS inspection is configured and there are many unanswered requests, the hash table may become full. When the table is full, the ACE stops inspecting new DNS traffic. Workaround: None.
•
CSCsl36877—The ACE may become unresponsive under the following conditions:
–
End-to-end SSL configuration
–
Traffic rate of 1000 TPS or more
–
Certificates and keys of any size
–
Variable page size of 1 KB to 8 KB (occurs more frequently) or a fixed page size of 8 KB
On the ACE console, you may see messages similar to the following:
–
n2_intr: ARB_UNIT_ERR 0x00000008
–
n2_intr: EFL_IND_RPTR 0x000327fc
–
n2_intr: EFL_IND_RPTR_TYPE 0x00000120
Workaround: None
•
CSCsl38419— Modifying or changing the SSL proxy under a Layer 7 policy map does not take effect. Workaround: Delete the Layer 3 service policy from the interface and then re-add it.
•
CSCsl42531—If you change the load-balancing predictor from least connections to round robin or the opposite with traffic running, the load-balancing receive queue (lbrx) may get stuck, which may cause the ACE to reload. This behavior may occur when servers are transitioning between the MAX-CONNS and OPERATIONAL states. Workaround: Do not change the load-balancing predictor configuration several times in a row with traffic running and servers transitioning in and out of the MAX-CONNS state.
•
CSCsl46334—With a high rate of Layer 7 load-balanced traffic flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse configured, the ACE may start dropping traffic after a few hours. Workaround: None.
•
CSCsl57828—With load-balanced TCP flows running through the ACE, the module may become unresponsive and may eventually reset if the client or the server advertises a maximum segment size (MSS) value that is less than 1460 bytes (the ACE default value) via the MSS option or if the client or the server does not send an MSS value. You may observe this behavior when the ACE configuration includes the following features:
–
Layer 4 load balancing and HTTP inspection
–
Layer 7 load balancing and the persistence rebalance command
Workaround: Configure the following Layer 4 connection parameter-map and associate it with the Layer 4 multi-match policy map as follows:
parameter-map type connection TCPset tcp wan-optimization rtt 0policy-map multi-match TESTclass TESTconnection advanced-options TCP•
CSCsl60440—When the predictor leastconns, server-conn reuse, and persistence-rebalance commands are configured, if successive GET requests result in the same policy being selected, then the ACE does not increment the connection counts for the real server and does not set up predictor weight values. However, when the server-side connection closes, the ACE incorrectly performs cleanup on every REUSE CLOSE message, which results in negative weights for the real servers and incorrect traffic distribution for real servers in the server farm. Workaround: Disable either the persistence-rebalance or the server-conn reuse command.
•
CSCsl62013—Entering the show logging internal command may cause the syslog daemon (syslogd) to perform a core dump, which, in turn, may cause the ACE module to reboot. Workaround: None.
•
CSCsl68531—In bridge mode, a real server in a transparent server farm may stop accepting connections after another real server in the same server farm fails probe health check(s). Workaround: None.
•
CSCsl74755—ACE sockets may become depleted when you repeatedly (approximately 250 attempts) access the ACE using SSH. For example, entering the ssh admin@192.168.1.123 sh ver command may cause the ACE to generate the following message to the client:
Received disconnect from 192.168.1.123: 2: Could not create socket pairs: Too many open files in system.In some cases, this condition generates the following message on the ACE console:
socket: Limiting socket usage, no more sockets, Max allowed is 512 and current_usage is 512Workaround: Avoid excessive SSH logins. After the condition occurs, you must reboot the ACE to clear the resource depletion. No other workaround is currently available.
•
CSCsl77474—Configuring the shared-vlan-hostid 16 command causes any interfaces configured on shared VLANs using that ID to become unreachable. These interfaces are in the various user contexts on the ACE, and failing traffic includes ARPs, probes, and so on. In this case, the interface address becomes completely unreachable. Any interface created on a shared VLAN while the shared-vlan-hostid 16 command is enabled is also unreachable. The show arp command displays a MAC address of 00:12:43:dc:ab:00 or higher on such interfaces. Workaround: Do not configure the shared-vlan-hostid 16 command. You can use any of the other available host IDs (1 through 15) for the shared-vlan-hostid command.
The ACE randomly assigns a host ID to shared VLANs if you do not configure a host ID. The ACE never choose a host ID of 16 when performing this random assignment. You will observe this behavior only if you configure the shared-vlan-hostid 16 command on your ACE.
•
CSCsl82712—When the ACE is configured for Layer 7 load balancing, an HTTP method request that exceeds the default (2000 bytes) or configured maximum HTTP header parse length is dropped and a TCP reset is sent to the client. Workaround: Increase the maximum HTTP header parse length in a parameter map using the set header-maxparse-length command.
•
CSCsl89772—With a UDP probe configured on port 69 (TFTP), the concurrent connections counter of the show resource usage command output keeps incrementing. Workaround: None.
•
CSCsl95565—When you are using SNMP with multiple contexts and redundancy is configured, the standby ACE may reboot with a last reboot reason of service snmpd. When the ACE comes back up, it may be in an odd state with some of the contexts not functioning properly. Workaround: None.
•
CSCsm03018—When you enable logging for syslog message IDs 302022 to 302025, changing the logging level for syslog messages 302022 to 302025 from Level 6 to Level 4 also affects the logging level for messages 302028 to 3020311. Logging rate limiting does not work for message IDs 302028 to 302031 and you cannot change the logging level for messages 302028 to 302031directly because these syslogs are not supported by the relevant CLI commands. Workaround: None.
•
CSCsm10896—Enabling the persistence rebalance command while a server farm is configured with any of the hash-based load-balancing predictors may cause the ACE to ignore the hash-based predictor after the first HTTP GET request. Instead, the ACE continues to load balance requests to the same real server that it previously selected if the requests match the same Layer 7 policy. Otherwise, the ACE load balances requests to a new real server in the server farm. Workaround: None.
•
CSCsm12883—The ACE resets connections when the number of long-lived persistent HTTP connections is one less than the configured maximum value. Workaround: None.
•
CSCsm17268—In a redundant configuration, when you add a single line to an existing config (for example, Admin(config)# snmp community public ro) and the ACE performs a subsequent incremental synchronization, the snmp community command appears in the running-config on the peer, but the process seems to cause an internal failure, that makes the peer transition to the STANDBY_COLD state. The running-config sync status is reported as Peer in Cold State. Incremental Sync Failure: snmp config sync to sby failed. Workaround: If possible, avoid the use of the snmp command while this behavior is under investigation.
•
CSCsm18847—If you change the logging level for syslog messages 302028 to 302031 when they are enabled using the logging fastpath command, the ACE still uses level 6 to rate-limit these syslogs. Workaround: None.
•
CSCsm35407—After rolling back a large ACE configuration to a blank context and entering the show running-config command, the ACE may perform a core dump. The core dump does not cause the ACE to reload and there is no adverse impact to the module. Workaround: None.
Software Version 3.0(0)A1(6.3a) Resolved Caveats
The following resolved caveats apply to software version 3.0(0)A1(6.3a):
•
CSCsj12610—After running XML and SNMP scripts that target the ACE control plane, the ACE may stop responding to supervisor engine heartbeats. After the heartbeats time out, the supervisor engine reloads the ACE. Workaround: If possible, do not run XML or SNMP scripts.
•
CSCsk35629—After approximately 1,000 successful logins, the ACE may refuse any additional login attempts via the CLI, console, XML, or HTTP and displays the "Internal CLI error: Success" or the "Internal CLI error: no such file" error. This behavior is caused by a file descriptor leak in the AAA daemon. Workaround: None.
•
CSCsk54917—With redundancy, application protocol inspection, sticky, and the persistence-rebalance command configured and up to 200,000 concurrent connections, a network processor (NP) may become unresponsive and cause the ACE to reload. Workaround: Omit either redundancy or application inspection from the above configuration combination.
•
CSCsl31145, CSCsl36877—With at least 50,000 active SSL connections per network processor in the ACE, the module may become unresponsive and you may observe the following show command output values:
–
The output of the show np [1 | 2] me-stats -u command indicates that XTOME is utilized at 100 percent
–
Delta values in the show crypto hardware command output are 0
–
The Delta value of the FastQ Transmit Backpressure field in the show np 1 me-stats -s fp command output is very high
Workaround: None.
•
CSCsl42384—When the ACE is terminating a connection, is in zero window-mode (because of a delayed ACK from the client or the server is transmitting faster than the client can respond), and the server keeps sending ACKs and/or data, the ACE may unnecessarily shrink the TCP window. This behavior can cause end hosts to become confused and in-flight segments to be incorrectly dropped because of failing window checks on the ACE. Workaround: None.
•
CSCsl48103—When the ACE is configured for TACACS+ authentication with a user context and the Cisco ACS sends the cisco-av-pair* attribute before the ACE custom shell attribute, you cannot log in to the ACE via TACACS+ and use the Admin role. Workaround: Do not use the ACE TACACS+ authentication for an Admin role. If you must use TACACS+ authentication for an Admin role, do not configure the Cisco ACS to send the cisco-av-pair* attribute.
•
CSCsl60797—With the persistence-rebalance and the conn-limit max commands configured and persistent GETs over live connections being received, the ACE load balances the requests to the same real servers if they are in the OPERATIONAL state. After a real server reaches the MAXCONNS state, then the ACE load balances subsequent GETs to a different real server in the same server farm. Because of this behavior, if all the real servers reach the MAXCONNS state at one time, then the ACE resets subsequent connections if it continues to receive persistent GETs. Workaround: None.
•
CSCsl67984—A user context configuration is lost when a bulk config sync is triggered in the Admin context. This issue may occur under the following conditions:
–
The Admin context is active on one peer and a user context is active on the other peer.
–
You bring the FT VLAN down so that both the Admin and the user contexts enter the ACTIVE/STANDBY_COLD state.
–
You make a change in the Admin context configuration on either ACE.
–
You bring the FT VLAN back up, which triggers a bulk config sync for the Admin context.
Workaround: Configure both the Admin and the user context FT groups to be active on a single peer.
•
CSCsl72808—Microsoft Outlook Web Access (OWA) Gnome Evolution-Exchange sends TLS v1.1 in the initial Client Hello. Because the ACE does not support TLS Version 1.1, the ACE does not respond and the connection times out and terminates. Workaround: Clients connecting to the ACE should use TLS Version 1.0.
•
CSCsl79785—With HTTP inspection URL logging configured and Layer 7 traffic being received at a rate of 20,000 connections per second, the ACE starts dropping all connections in the inbound connection manager (ICM) because of the drop buffer threshold limit. This behavior occurs in less than five minutes after starting traffic and the ACE does not recover. Workaround: None.
•
CSCsl87663—When FTP inspection is configured, the ACE permits the dynamically negotiated data channel to pass through bypassing the security checks based on ACLs. However, this bypass behavior applies incorrectly only to inbound ACLs configured on the ingress interface and not to any outbound ACLs configured on the egress interface. As a result, if the configuration has outbound ACLs that deny the data channel traffic, FTP connections will fail, despite the application inspection configuration. Workaround: Remove the outbound ACLs that deny the data channel traffic.
•
CSCsl95191—If you remove a service policy (either globally or from an interface) and then re-add it or if you remove a NAT pool from an interface and then re-add it, some of the NAT fabric entries are lost. Workaround: Remove the relevant class map and the policy map and then re-add them to the configuration.
•
CSCsl99468—Clients who connect through the ACE using the default set tcp buffer-share size and set ack-delay values (32 KB and 200 ms, respectively) may experience a slow response time. Approximately 95 percent of the responses are returned in 30 to 40 ms. However, the remaining 5 percent of responses are returned in approximately 250 ms.
Workaround: You can minimize (but not eliminate) the percentage of delayed packets by increasing the TCP buffer-share size. You can minimize the delay lengths by decreasing the TCP ACK delay value. Because each situation is affected by different variables, the best values for the set tcp buffer-share and the set ack-delay commands need to be determined by testing response times with various values.
You can reduce some of the delay by allowing the TCP window to continuously update and send a window update when the window size is below 1 maximum segment size (MSS) rather than when the window size reaches 0. You can reduce additional delay by changing the ACE behavior so that an ACK is not delayed if the unupdated window size is below 2 MSS rather than waiting for the window update. This action prevents a latency if the unupdated window is below 2 MSS and the updated window is above 2 MSS.
•
CSCsm02006—With static sticky and a backup server configured on the ACE, when the primary server goes down, the ACE incorrectly loadbalances a flow intended for the primary server to a new server instead of the backup server. Workaround: None.
•
CSCsm03614, CSCsm27262—With at least 50,000 active SSL connections per IXP in the ACE, you may observe the following show command output values:
–
The output of the show np [1 | 2] me-stats -u command indicates that XTOME is utilized at 100 percent
–
TX Buffers used or RX Buffers used values in the show crypto hardware command output are nonzero
Workaround: None.
•
CSCsm15827—In a redundant configuration with 20 or more contexts, you may observe that the standby ACE is stuck in the INIT/CONFIG state because of redundancy message corruption. This behavior can occur either during bootup or after a switchover is performed. This issue is intermittent because a freed buffer is later reused. Workaround: none.
•
CSCsm20608—The ACE may reload unexpectedly when the SSL traffic loads surpass the normal high-performance levels of the module.
•
CSCsm22766—After taking a real server out of service or putting a real server back into service, the ACE may become unresponsive or may silently reload and sk_buff allocation failure messages may appear on the console prior to the reload. This behavior, which occurs when the network buffers are not being properly freed, can cause a leak in the network buffers. After eight hours, the leak can cause the system to become unresponsive or to reload. Workaround: None.
Software Version 3.0(0)A1(6.3) Open Caveats, Resolved Caveats, and Command Changes
The following sections contain the open and resolved caveats and command changes in software version 3.0(0)A1(6.3):
•
Software Version 3.0(0)A1(6.3) Open Caveats
•
Software Version 3.0(0)A1(6.3) Resolved Caveats
•
Software Version 3.0(0)A1(6.3) Command Changes
Software Version 3.0(0)A1(6.3) Open Caveats
The following open caveats apply to software version 3.0(0)A1(6.3):
•
CSCsh70258—SSL initiation configured on an ACE may fail when it is used with Microsoft IIS 6.0 that is configured to accept client certificates. The additional SSL communication from the server to the ACE causes the ACE to experience an internal error. Workaround: In the IIS Web Server Properties > Directory Security > Edit menu, specify Ignore client certificates.
•
CSCsj12610—A script that is running the show version command periodically causes the ACE to reload after 30 hours. Workaround: None.
•
CSCsj41909—The ACE fails with the following message appearing in the syslog:
%ACE-5-199006: Orderly reload started at xxxxxxx by System.The reload reason is NP 1 Failed: NP Process Crashed. Workaround: None
•
CSCsj64833—The network utility traceroute does not work for a configured ACE IP interface if the underlying protocol is UDP or TCP. Workaround: ICMP traceroute will work. The default of most traceroute utilities is UDP so a command line option might be necessary. For Linux, use the -I option.
•
CSCsj68643—The following log messages may appear sporadically in the ACE log:
–
can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.
–
can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.
These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.
•
CSCsj70522—In a redundant configuration with multiple contexts and logging enabled for the console and the buffer on both ACEs, log messages appear only on the standby console when a probe to a server failed. No new messages are observed in the buffers on either ACE. After entering the changeto command to another context and then back to the original context, log messages appear on the active ACE console, but the log buffer is not updated. Workaround: None.
•
CSCsj74250—When you configure the TACACS server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, there is a key mismatch when attempting to authenticate. Workaround: Paste the properly encrypted key into the running-config.
•
CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, then subsequently try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in RSA format instead of RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.
•
CSCsj94366—While attempting to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: console configuration can only be done on console. Workaround: None.
•
CSCsk04298—When a switchover occurs in a Layer 7 load-balancing persistence-rebalance configuration, connections with multiple GETs may not proceed. Workaround: None.
•
CSCsk15979—With UDP and TCP class maps sharing the same VIP and both having the idle timer set to infinite, a series of connections are made and allowed to sit idle for several hours. When a change is made to the UDP class map to time out these connections in 60 seconds, the ACE clears the TCP and UDP connections. Workaround: None.
•
CSCsk16083—Changing the file used for hash probes does not fail the probe if the server is a Microsoft Windows 2000 server. Workaround: None.
•
CSCsk34767—CSCsh63231 added FTP inspection statistics to the show stats inspect command output, but the statistics cannot be cleared using the clear stats inspect command. Workaround: None.
•
CSCsk34843—The following messages may be observed on the console connection:
"Data bus error, epc == 2b5008e0, ra == 2b5008c8badvaddr 0x00005f00 cause 0x8cd8fd18 status 0x00007fff<1>A_SCD_BUS_ERR_STATUS 0x810c0000A_BUS_ERR_DATA_0 0x0A_BUS_ERR_DATA_1 0x0A_BUS_ERR_DATA_2 0xffffb12a00000000A_BUS_ERR_DATA_3 0xffff000000000000A_BUS_L2_ERRORS 0x0A_BUS_MEM_IO_ERRORS 0x20000 "Workaround: None.
•
CSCsk36611—If you are using Internet Explorer (IE), an SSL rehandshake may fail if the total length of the SSL certificate chain is greater than 4024 bytes. When this condition exists, the ACE creates two SSL records. The first record has a total length field indicating 4024 bytes, but containing a certificate item with a specified length greater than 4024 bytes. The second record contains a new SSL record header and the remaining portion of the previous SSL record. Workaround: Use Firefox or another browser.
•
CSCsk44035—When persistence-rebalance is enabled and there is an extremely high load on the ACE, buffer allocation failures and slow end-user responses may occur, and the ACE may reboot. Workaround: None.
•
CSCsk54917—The ACE may reboot when there are a significant number of concurrent connections (for example, 200 K open connections) in use and you have following functions configured on the ACE:
–
HTTP inspection
–
IP address sticky group in an SLB policy
–
Persistence rebalance
The ACE reboots because multiple Micro Engines become unresponsive in the IXP network processor on the ACE. Workaround: None.
•
CSCsk72920, CSCsk70822—Heavy SSL traffic passing through the ACE when operating in an end-to-end SSL configuration may cause a particle buffer to have a reference count of zero, causing the module to reboot. Workaround: None.
•
CSCsk95828—Checkpoint rollback issues occur with the banner and hostname or SSH keys. Workaround: None.
•
CSCsl15321—After reloading with an HTTP inspection configuration, the ACE drops HTTP packets even when the module is configured to allow them to pass. Workaround: Reapply the configuration.
•
CSCsl32269—After configuring a VIP while running in one-arm mode, you may observe that the traffic to the VIP fails and the ACE resets the connection after completing the back-end three-way handshake. Workaround: None. To resolve this condition, you must reboot the ACE.
•
CSCsl34397—The ACE may reboot with an snmpd core dump. Workaround: None.
•
CSCsl36877—The ACE may become unresponsive under the following conditions:
–
End-to-end SSL configuration
–
Traffic rate of 1000 TPS or more
–
Certificates and keys of any size
–
Variable page size of 1 KB to 8 KB (occurs more frequently) or a fixed page size of 8 KB
On the ACE console, you may see messages similar to the following:
–
n2_intr: ARB_UNIT_ERR 0x00000008
–
n2_intr: EFL_IND_RPTR 0x000327fc
–
n2_intr: EFL_IND_RPTR_TYPE 0x00000120
Workaround: None
•
CSCsl42531—If you change the load-balancing predictor from least connections to round robin or the opposite with traffic running, the load-balancing receive queue (lbrx) may get stuck, which may cause the ACE to reload. This behavior may occur when servers are transitioning between the MAX-CONNS and OPERATIONAL states. Workaround: Do not change the load-balancing predictor configuration several times in a row with traffic running and servers transitioning in and out of the MAX-CONNS state.
•
CSCsl46334—With a high rate of Layer 7 load-balanced traffic flowing in multiple contexts or a high rate of Layer 7 traffic with server connection reuse configured, the ACE may start dropping traffic after a few hours. Workaround: None.
•
CSCsl48103—When the ACE is configured for TACACS authentication in a user context, a user with the Admin role cannot log in to the ACE if the Cisco Secure ACS sends the cisco-av-pair attribute in the TACACS Authorization Response before the ACE custom shell attribute.
Use one of the following workarounds:
–
Do not use ACE TACACS authentication for an Admin role.
–
Do not send the cisco-av-pair attribute via Cisco ACS.
Software Version 3.0(0)A1(6.3) Resolved Caveats
The following resolved caveats apply to software version 3.0(0)A1(6.3):
•
CSCse90603—Autocomplete does not work on checkpoint names. When issuing the checkpoint rollback and checkpoint delete commands, the "?" does not display the existing checkpoint names. Workaround: Use the show checkpoint all command to locate the name for the checkpoint rollback and checkpoint delete commands.
•
CSCsg62851—Occasionally, the ACE inserts a zero-length cookie value when cookie insertion is enabled. This behavior may occur when the same server farm (and, therefore, the same real server) is part of two different sticky groups. Workaround: None.
•
CSCsh27991—When sending a large ping of 50 KB to management IP addresses, all management and through-the-ACE communication may fail. This condition occurs when you configure the fragment chain limit value greater than the default value. Workaround: Set a low fragment chain limit value on the interfaces with the ICMP management policies.
•
CSCsh53739—In a server farm, a real server may enter a state where its status is ARP_FAILED in the output of the show serverfarm command. The ACE can still communicate with the server, but the ACE does not load balance traffic to this server. Workaround: Remove and then readd the real server configuration.
•
CSCsi40704—With Real-time Streaming Protocol (RTSP) inspection enabled and RTSP traffic flowing, the ACE may become unresponsive and may generate a core dump file, which indicates that the CMCLOSE microengine is disabled. This behavior occurs when a client and a server initiate a Real-time Transport Control Protocol (RTCP) connection at nearly the same time. Workaround: None.
•
CSCsi79974—With the logging trap 7 command, the logging buffered 6 command, and the logging host command configured, the ACE is not sending level 6 syslog messages to any location (console, buffer, or syslog server) for the control plane (CP) or the data plane (DP). Workaround: None.
•
CSCsj52843—A syslog for a real server state change (UP or DOWN) that is caused by a health probe failure is not generated when the probe is configured under the real server. If the health probe is configured under the server farm instead of under the real server, then the syslogs for real server state changes (4442001 and 4442002) are generated correctly. Workaround: Configure the health probe under each server farm to which the real server belongs.
•
CSCsj88500—If a large configuration contains many different service-policy actions, some of the show service-policy command output may be missing. The ACE may not display some of the actions associated with the service policy. Workaround: None.
•
CSCsk12848—Installing or uninstalling the ACE-250CTX-08G-SSL-20K.lic license causes one of the peers in a redundant configuration to enter an error state. The show ft group detail command output shows that the FT group is stuck in the STANDBY_CONFIG state. The FT group fails to go to the STANDBY_HOT state. This behavior occurs because the installing or uninstalling of the license internally generates multiple requests to the ACE to restart configuration synchronization. Because of the timing of these requests, the FT group remains in the incorrect state. Workaround: Remove and then reconfigure the FT group to put it into the correct state.
•
CSCsk27578—If you change an HTTP inspection policy action (for example, from permit to reset) while the ACE is inspecting HTTP traffic, the Conn Mismatch Errors counter of the HTTP microengine will increase and the inspectHttp process may fail, which causes a reload of the ACE. This behavior is more likely to occur when HTTP traffic is arriving at a rate of more than 25 K requests per second. Workaround: None.
•
CSCsk35523—In a redundant configuration, the configuration manager process fails while configuring and unconfiguring 250 contexts multiple times. Workaround: None.
•
CSCsk38275—With a Layer 7 load-balancing configuration and a large number of concurrent Layer 7 load-balancing connections, the ACE may reboot. Workaround: None.
•
CSCsk44854—With secondary cookie sticky and HTTP inspection configured, traffic may stop flowing after about 9 K connections are processed because the ACE does not release the buffers. Workaround: None.
•
CSCsk48871—While performing a Trivial File Transfer Protocol (TFTP) copy operation to upload a dump file from the core: directory to a TFTP server, the ACE may become unresponsive. Workaround: None.
•
CSCsk49756—In an SSL termination configuration with the match source-address command configured in a class map and a heavy volume of traffic, the ACE may reboot. Workaround: None.
•
CSCsk56416—Heavy SSL traffic passing through the ACE when operating in an end-to-end SSL configuration may cause the microengines in the IXP network processor to run too long, causing the module to reboot. Workaround: None.
•
CSCsk59228—With the inspect dns command configured, approximately 4 M concurrent connections (mostly DNS), and 1 M NAT operations, the ACE may become unresponsive if the network processor becomes overloaded and stops processing messages on the internal queues. Workaround: Remove the inspect dns command from the configuration.
•
CSCsk66975—When dynamic NAT is configured for the client IP addresses and FTP strict inspection is configured for inspecting FTP traffic, the translations (xlates) created for the data channel are leaked when active FTP is used (the server initiates a data channel to the client), the output of the show xlate command displays xlates with a nonzero reference count even when there are no connections, and entering the clear xlate command prevents any new connections from being established from the client side because the xlate goes into a dump state. Workaround: Reboot the ACE and change the inspect ftp strict command to the inspect ftp command.
•
CSCsk72011—SSL connections may fail in the ACE because of internal errors. When an SSL connection failure occurs, you may observe the following internal error on the ACE IXP network processor console:
buf_atomic_decrement_ref_cnt Illegal ref_cnt value of 0 detected a8b76670 d 1 e 0Workaround: None.
•
CSCsk77129—When you add a real server that is in the PROBE-FAILED state to a server farm that is configured with the least-connections predictor, the real server receives all the new incoming connections because of the incorrect initialization of the weighted connection count. Workaround: Use the round-robin predictor.
•
CSCsk82829—With end-to-end SSL configured, an SSL microengine malfunction may cause the ACE to stop servicing any connections. Workaround: None.
•
CSCsk85110—During connection replication processing where the ACE is loadbalancing Layer 4 connections in a redundant configuration, the ACE may perform a core dump. Workaround: None.
•
CSCsk86070—When you enter the class class_name insert-before class_name command, the ACE may log the following message:
%ACE-1-106028: WARNING: Unknown error while processing access-group.The ACE may also log the following message if the debug access-list merge errors and the debug access-list merge info commands are enabled:
acl: (ctx:0) ACL-MERGE add acl: instance:1 feature:SECURITYprotocol:IPv4 priority:0 ace-count:1 sec-level:0x7context:0 acl: (ctx:0) ACL-MERGE-ERROR:Duplicate lineno: lineno already exists in insert_ace_in_acl ../security/acl/acl_merge.c:3728 acl: (ctx:0) ACL-MERGE-ERROR:insertion in list failed in acl_merge_add_acl_to_list ../security/acl/acl_merge.c:4967Workaround: Enter the write memory command and then reload the ACE. Alternatively, you can delete the interface on which the offending service policy is configured using the no interface vlan vlan_id command and then reconfigure the interface with the IP addresses, the service-policy, and so on. You must perform these actions for every interface on which the service policy is configured.
•
CSCsk89562—If you configure dynamic NAT with PAT on an ACE for ICMP traffic, you may observe that a large number of NAT pool allocations fail (as displayed by the show np 1 me-stats -socm | i NAT command) and that ICMP ports (ICMP IDs) are not released to the NAT pool for reuse when the connections close. Eventually, the number of active NAT pool allocations (NAT Pool Alloc and NAT Pool Free counters) may increase beyond the number of active connections indicating that the NAT resources are not being reused. When no ports are available, NAT allocation failures result and the ACE forwards the traffic unNATed. This behavior has not been observed with UDP or TCP connections.
Workaround: If possible, avoid NAT of ICMP traffic. If NAT of ICMP traffic is necessary for your application or if NAT resources have already been leaked, schedule periodic reboots of the ACE to free NAT pool resources. If you schedule the reboots before the ports are totally depleted, then you will not observe any adverse network effect. When the difference between the NAT Pool Alloc and the NAT Pool Free counters is approximately 32,000, the ACE has reached the upper limit of resources that may be leaked before an adverse network effect is noticed and NAT Pool Alloc failures associated with unNATed connections occur. Rebooting the ACE before the difference between the NAT Pool Alloc and the NAT Pool Free counters is 30,000 will protect your network from any unexpected event.
•
CSCsk96968—When SSL termination is configured on the ACE and a connection parameter-map that includes the nagle feature is applied, you may observe a delay in the time that is required to complete an SSL handshake. This behavior should not cause the connection to fail. For connections where a large amount of data will be passed, the time required to send the data will be much greater than the delay caused by completing the SSL handshake. In these cases, you may not notice a delay. However, for connections with only a small amount of data, the handshake delay may be more apparent.
Workaround: If you observe the handshake delay and consider it to be a problem, disable nagle in the connection parameter map.
•
CSCsk98100—Changing the IP address of a VIP causes the new VIP to stop responding to ICMP pings even though the new VIP is configured to respond to ICMP pings with the loadbalance vip icmp-reply active command. Workaround: Disable and then reenable the loadbalance vip icmp-reply active command associated with the traffic class.
•
CSCsk98229—The system uptime value resets after 49 days and 17 hours. Workaround: None.
•
CSCsk98342—With persistence-rebalance enabled, the ACE continues to load balance connections to a previously selected server even when it is not part of the same server farm. If you remove a server from a server farm to which some initial GET requests in a persistence-rebalance enabled connection were being loadbalanced, the ACE continues to send connections to that server and ignores the configuration change. Workaround: Disable persistence-rebalance or do not change the ACE configuration in the middle of a connection.
•
CSCsk98697—With a large number of concurrent Layer 7 load-balancing connections in multiple contexts with heavy continuous traffic, the ACE may reboot. Workaround: None.
•
CSCsk99824—Performance with 2 K SSL keys is very low. Workaround: None.
•
CSCsl03150—SSL traffic may cause the ACE to stop servicing any connections. This behavior was caused by an SSL service that was configured and the SSL traffic that was sent to that service. Workaround: None.
•
CSCsl06233—With server connection reuse configured and a very heavy traffic rate, the ACE may become unresponsive. Workaround: None.
•
CSCsl08681—The back-end and end-to-end SSL connections may fail because of an incorrect error code. This behavior affects only SSL connections that are initiated from the ACE. SSL termination connections are not affected. Workaround: None.
•
CSCsl14127—With end-to-end SLL and chain groups configured, a gradual memory leak may occur in the two ACE IXPs. Workaround: None.
•
CSCsl22716—The CLI command that allow you to view the contents of the Hyperion registers currently requires the loading of the debug plug-in to access specific vsh_debug commands. Access to the internal Hyperion ASIC component registers may be useful as a troubleshooting aid to determine why the ACE may have become unresponsive to outside traffic. Workaround: None.
•
CSCsl24338—When using two ACE modules in a redundant configuration, a number of the configured VIP addresses listed in the routing table may be lost when the standby ACE becomes active. This behavior occurs with the following configurations on the ACE:
–
Multiple virtual IP (VIP) addresses and routes are configured on the active ACE.
–
Route Health Injection (RHI) is enabled as a Layer 3 and Layer 4 SLB policy action to advertise the availability of a VIP address.
To identify the lost VIP addresses, use the show ip route command to display the route entries. Examine the show output to determine if any advertised routes are missing. Workaround: Specify the shutdown command, followed by the no shutdown command, to reenable the VLAN interface associated with the missing VIP addresses. When a switchover occurs, the active ACE will reinject the missing routes.
•
CSCsl31301—In a Layer 7 configuration, the ACE does not retransmit a FIN in a TCP packet that does not have data in it. Workaround: The associated client-server connections will time out eventually. No action is required.
•
CSCsl32646—When you are using the least-connections load-balancing predictor, if a server goes out of service ungracefully and then comes back into service later, the ACE may not assign the appropriate number of new connections to the server. This behavior may impact other real servers in the server farm. Workaround: If you observe this condition, use the round-robin predictor instead of least connections. If least connections is necessary for your application, try changing the predictor to round-robin temporarily, wait 60 seconds, and then change back to least connections. If the problem does not resolve itself, use round-robin if possible.
•
CSCsl38632—Occasionally, uploading or downloading files over SSL may fail. This behavior may occur when the ACE is the receiver of TCP data. The TCP buffer-share feature implements TCP sliding windows on the ACE and can cause the ACE to advertise a zero window size to the sender of the data. If a duplicate ACK arrives at the ACE soon after the module advertised a zero window segment, then the ACE fails to reopen its advertised window and the connection stalls. Workaround: None.
•
CSCsl48846—With HTTP-cookie sticky configured, when the ACE parses the cookie value, the module occasionally rejects GETs from Internet Explorer (IE). The ACE sends a TCP RST to the client to close the connection and the Static Parse Errors counter in the show stats http command output increments for each RST sent by the ACE. The ACE does not reject GETs from Firefox. The difference between the two browsers is that IE sometimes sends a GET containing a cookie with the following form:
Cookie: COOKIE; COOKIE2=VALUE
^ Note that there is a space following the delimiting semicolon.
The ACE is configured to parse the value of COOKIE2 in the above cookie string.
You may observe this behavior only when you are using HTTP parsing of a cookie in which the above construct occurs. If the ACE is not performing cookie sticky or otherwise requiring that the cookie string be parsed, this type of traffic is not a problem. In that case, the ACE either routes or load balances the traffic, depending on your configuration.
Note that none of the following cookie constructs generates a RST:
Cookie: COOKIE1;COOKIE2=VALUE (no space following the delimiter)
Cookie: COOKIE1=;COOKIE2=VALUE (explicit equals; no space following the delimiter)
Cookie: COOKIE1=; COOKIE2=VALUE (explicit equals; space following the delimiter)
Workaround: Avoid the offending construct by using Firefox or use a non-Layer 7 load-balancing policy for this traffic.
Software Version 3.0(0)A1(6.3) Command Changes
Table 1 lists the commands and options that have been added in software version 3.0(0)A1(6.2).
Table 2 lists the commands and options that have changed in software version 3.0(0)A1(6.2).
Source NAT Using a VIP
The ACE now allows you to configure a virtual IP (VIP) address in the network address translation (NAT) pool for dynamic NAT and PAT. This action is useful when you want to source-NAT real server-originated connections (bound to the client) using the VIP address. This feature is specifically useful when there are a limited number of real world IP addresses on the client-side network. To perform PAT for different real servers that are source-NATed to the same IP address (VIP), you must configure the pat keyword in the nat-pool command.
For details about configuring dynamic NAT and PAT for source NAT, see the Cisco Application Control Engine Module Security Configuration Guide.
Software Version 3.0(0)A1(6.2a) Operating Considerations, Open Caveats, Resolved Caveats, and Command Changes
The following sections contain the operating considerations, open and resolved caveats, and command changes in software version 3.0(0)A1(6.2a):
•
Software Version 3.0(0)A1(6.2a) Operating Considerations
•
Software Version 3.0(0)A1(6.2a) Open Caveats
•
Software Version 3.0(0)A1(6.2a) Resolved Caveats
•
Software Version 3.0(0)A1(6.2a) Command Changes
Software Version 3.0(0)A1(6.2a) Operating Considerations
The following operating considerations apply to software version 3.0(0)A1(6.2a):
•
Support for the maximum SSL certificate chain length has been increased from 8000 bytes to 16000 bytes. However, if a browser performs an SSL rehandshake, SSL may fail if the certificate chain exceeds 4000 bytes. For details, see open caveat CSCsk36611.
•
Support has been added for extended validation (EV) SSL certificates. If you are using a 2048-bit RSA key with EV certificates, the chained certificate will likely exceed 4000 bytes. In this case, a rehandshake issued by a browser may fail. For details, see open caveat CSCsk36611. EV SSL certificates with 1024-bit RSA keys are not impacted by CSCsk36611.
Software Version 3.0(0)A1(6.2a) Open Caveats
The following open caveats apply to software version 3.0(0)A1(6.2a):
•
CSCse90603—Autocomplete does not work on checkpoint names. When issuing the checkpoint rollback and checkpoint delete commands, the "?" does not display the existing checkpoint names. Workaround: Use the show checkpoint all command to locate the name for the checkpoint rollback and checkpoint delete commands.
•
CSCsh27991—When sending a large Ping of 50 KB to management IP addresses, all management and through the ACE communication may fail. This condition occurs when you configure the fragment chain limit as greater than the default value. Workaround: Set a low fragment chain limit on the interfaces with the ICMP management policies.
•
CSCsj35046—Checkpoint rollback issues occur with the banner and hostname or SSH keys. Workaround: None.
•
CSCsj41909—The ACE fails with the following message appearing in the syslog:
%ACE-5-199006: Orderly reload started at xxxxxxx by System.The reload reason is NP 1 Failed: NP Process Crashed. Workaround: None
•
CSCsj64833—The network utility traceroute does not work for a configured ACE IP interface if the underlying protocol is UDP or TCP. Workaround: ICMP traceroute will work. The default of most traceroute utilities is UDP so a command line option might be necessary. For Linux, use the -I option.
•
CSCsj68643—The following log messages may appear sporadically in the ACE log:
–
can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.
–
can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.
These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.
•
CSCsj70522—In a redundant configuration with multiple contexts and logging enabled for the console and the buffer on both ACEs, log messages appear only on the standby console when a probe to a server failed. No new messages are observed in the buffers on either ACE. After entering the changeto command to another context and then back to the original context, log messages appear on the active ACE console, but the log buffer is not updated. Workaround: None.
•
CSCsj74250—When you configure the TACACS server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, there is a key mismatch when attempting to authenticate. Workaround: Paste the properly encrypted key into the running-config.
•
CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, then subsequently try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in RSA format instead of RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.
•
CSCsj88500—If a large configuration contains many different service-policy actions, some of the show service-policy command output may be missing. The ACE may not display some of the actions associated with the service policy. Workaround: None.
•
CSCsj94366—While attempting to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: console configuration can only be done on console. Workaround: None.
•
CSCsk04298—When a switchover occurs in a Layer 7 load-balancing persistence-rebalance configuration, connections with multiple GETs may not proceed. Workaround: None.
•
CSCsk15979—With UDP and TCP class maps sharing the same VIP and both having the idle timer set to infinite, a series of connections are made and allowed to sit idle for several hours. When a change is made to the UDP class map to time out these connections in 60 seconds, the ACE clears the TCP and UDP connections. Workaround: None.
•
CSCsk16083—Changing the file used for hash probes does not fail the probe if the server is a Microsoft Windows 2000 server. Workaround: None.
•
CSCsk34767—CSCsh63231 added FTP inspection statistics to the show stats inspect command output, but the statistics cannot be cleared using the clear stats inspect command. Workaround: None.
•
CSCsk34843—The following messages may be observed on the console connection:
"Data bus error, epc == 2b5008e0, ra == 2b5008c8badvaddr 0x00005f00 cause 0x8cd8fd18 status 0x00007fff<1>A_SCD_BUS_ERR_STATUS 0x810c0000A_BUS_ERR_DATA_0 0x0A_BUS_ERR_DATA_1 0x0A_BUS_ERR_DATA_2 0xffffb12a00000000A_BUS_ERR_DATA_3 0xffff000000000000A_BUS_L2_ERRORS 0x0A_BUS_MEM_IO_ERRORS 0x20000 "Workaround: None.
•
CSCsk34907—Config sync displays an incremental synchronization failure even though the configurations on the active and standby ACEs are in sync. Workaround: None
•
CSCsk36611—If you are using Internet Explorer (IE), an SSL rehandshake may fail if the total length of the SSL certificate chain is greater than 4024 bytes. When this condition exists, the ACE creates two SSL records. The first record has a total length field indicating 4024 bytes, but containing a certificate item with a specified length greater than 4024 bytes. The second record contains a new SSL record header and the remaining portion of the previous SSL record. Workaround: Use Firefox or another browser.
•
CSCsk44035—When persistence-rebalance is enabled and there is an extremely high load on the ACE, buffer allocation failures and slow end-user responses may occur, and the ACE may reboot. Workaround: None.
•
CSCsk54917—The ACE may reboot when there are a significant number of concurrent connections (for example, 200 K open connections) in use and you have following functions configured on the ACE:
–
HTTP inspection
–
IP address sticky group in an SLB policy
–
Persistence rebalance
The ACE reboots because multiple Micro Engines become unresponsive in the IXP network processor on the ACE. Workaround: None.
•
CSCsk56416—Heavy SSL traffic passing through the ACE when operating in an end-to-end SSL configuration may cause the Micro Engines in the IXP network processor to run too long, causing the module to reboot. Workaround: None.
•
CSCsk72011—SSL connections may fail in the ACE due to internal errors. When an SSL connection failure occurs, you may observe the following internal error on the ACE IXP network processor console:
buf_atomic_decrement_ref_cnt Illegal ref_cnt value of 0 detected a8b76670 d 1 e 0Workaround: None.
•
CSCsk72920—Heavy SSL traffic passing through the ACE when operating in an end-to-end SSL configuration may cause a particle buffer to have a reference count of zero, causing the module to reboot. Workaround: None.
•
CSCsk81763—With the ACE configured for end-to-end SSL connectivity, SSL traffic may stop and, after approximately 10 or 15 minutes, the ACE reboots. This behavior may occur when you use an RSA key pair that is less than 1024 bits (for example, 512 bits). Workaround: Specify a key pair that is greater than 1024 bits. For details, see the Cisco Application Control Engine Module SSL Configuration Guide.
•
CSCsl04618—The ACE may reboot unexpectedly due to memory allocation library corruption. Workaround: None.
•
CSCsl22716—The CLI for viewing the contents of the Hyperion registers currently requires the loading of the debug plug-in to access specific vsh_debug commands. Access to the internal Hyperion ASIC component registers may be useful as a troubleshooting aid to determine why the ACE may have become unresponsive to outside traffic. Workaround: None.
•
CSCsl23116—When using the ACE to perform SSL termination, the Internet Explorer web browser may become unresponsive when loading pages from the ACE. This behavior may occur when the ACE inserts multiple SSL records in a single frame. You may not encounter this issue when using other web browsers, such as Firefox. Workaround: Use Firefox or any browser, other than Internet Explorer.
•
CSCsl24338—When using two ACE modules in a redundant configuration, a number of the configured VIP addresses listed in the routing table may be lost when the standby ACE becomes active. This behavior occurs with the following configurations on the ACE:
–
Multiple virtual IP (VIP) addresses and routes are configured on the active ACE.
–
Route Health Injection (RHI) is enabled as a Layer 3 and Layer 4 SLB policy action to advertise the availability of a VIP address.
To identify the lost VIP addresses, use the show ip route command to display the route entries. Examine the show output to determine if any advertised routes are missing. Workaround: Specify the shutdown command, followed by the no shutdown command, to reenable the VLAN interface associated with the missing VIP addresses. When a switchover occurs, the active ACE will reinject the missing routes.
Software Version 3.0(0)A1(6.2a) Resolved Caveats
The following resolved caveats apply to software version 3.0(0)A1(6.2a):
•
CSCsg06046—With persistence-rebalance configured, if a client opens a single HTTP 1.1 persistent connection and sends multiple GETs with all requesting the same data, each GET causes the ACE to tear down the server flow, load balance to a new server, and establish a new server flow for each one.
•
CSCsh51846—After configuring the banner motd command, the banner appears for Telnet sessions. When accessing the ACE through SSH, the same banner is not present.
•
CSCsh77916, CSCsk47318, CSCsk49113—Random SSL traffic causes the ACE to unexpectedly reboot.
•
CSCsh80233—Enabling the strict option with the SSL parameter map close-protocol command does not change the behavior. When the strict option is enabled, the server or client sends a close notify message and expects to receive the same from its peer. However the ACE does not support this behavior even though the option is available.
•
CSCsi01207—When using two ACE modules in a redundant configuration, if you use the shared vlan hostid command to configure a specific bank of MAC addresses for an ACE, the MAC address allocated for a shared VLAN interface will be the same on both the active and the standby ACE. This behavior may cause MAC address moves in the Catalyst 6500 series switch supervisor engine or Layer 2 MAC address table, which may lead to traffic destined to the interface MAC address to be intermittently switched to the standby ACE.
•
CSCsi50665—The CSM-to-ACE tool generates invalid access control lists (ACLs). The ACL generated by the tool has an invalid any parameter.
•
CSCsi52045—When you use a Layer 7 load-balancing configuration with multiple connections per second occurring, the ACE may drop some packets. The ACE drops the packets when the connection moves from the proxied to the unproxied state after the ACE sends the first parsed request to the real server.
•
CSCsi82962—When persistence-rebalance is enabled and the TCP congestion window is full from the ACE TCP stack perspective, the ACE may not respect the negotiated MSS.
•
CSCsi92341—The show logging command may not display the latest syslog.
•
CSCsi98829—SSL connections to the ACE do not complete due to an SSL handshake failure. This failure occurs when the certificate on the ACE used for the SSL handshake has an exponent with an odd length.
•
CSCsj07696—The ACE may reboot during connection replication in a redundancy configuration. This issue occurs when the active and standby ACE switch over. Then, the previously active ACE becomes active again, and starts replicating connections to the standby ACE. The standby ACE may fail while updating the connection counts for replicated connections for a real server.
•
CSCsj14813—When the SSL cert/key pair is modified and the modification results in a mismatch, the configuration download fails.
•
CSCsj17220, CSCsj59659, CSCsk07539—The load-balancing receive queue (LBRX) may become stuck and does not drain.
•
CSCsj23996—An ACL download failure occurs due to duplicate line numbers. The ACL is not updated and traffic matches incorrect entries, which results in behavior such as source NAT using the wrong NAT pool.
•
CSCsj25440—When the radius-server, tacacs-server, and ldap-server commands are configured on the ACE, the no form of the commands fail and do not remove these entries.
•
CSCsj26023—The QNX qconn process may terminate unexpectedly and generate a core dump file.
•
CSCsj28335—An ACE configuration that includes Layer-7 load balancing and the persistence-rebalance command combined with several GET requests per a TCP connection may lead to throughput degradation.
•
CSCsj37029—Pushing the ACE beyond its performance capability may result in poor SSL performance with large GET or POST page sizes. Redundancy may also switch over multiple times under the same circumstances. When you use certain ciphers, redundancy may switch over multiple times.
•
CSCsj37653—Unexpected URL or inspection results occur when the ACE parses multibyte UTF8 encoding. This problem applies only when the ACE parses 2 or 3 byte characters.
•
CSCsj40587—In a redundancy configuration with sticky replication enabled and multiple contexts to load balance, the ACE may reboot and generate a core dump file when a switchover occurs.
•
CSCsj47861—If certain internal flags are non-zero, the demo license installation may fail with the following error message: Installing license... failed: Can't install this license with the current count.
•
CSCsj48884—When an FTP client sends a PORT command inside an FTP control channel to an FTP server, the server opens a connection to the client. This connection hits the ACE with the server IP address as the source. The ACE should replace the source IP address with the VIP address. However, if client NAT is configured on the server interface, the ACE gives priority to the policy NAT and translates the source IP address with the policy NAT instead of the VIP, thereby breaking the FTP data connection.
•
CSCsj52843—When you configure a health probe under a real server and the probe fails, the ACE does not generate a syslog for a real server state change (UP or DOWN). If you configure the health probe under a server farm, the ACE does generate syslogs for real server state changes (4442001 and 4442002).
•
CSCsj55836—With Layer 7 features configured and certain SSL traffic flowing, new connections may not be processed because the buffers have been consumed by the SSL process.
•
CSCsj66075—A real server may reject new connections intermittently even though it is in the OPERATIONAL state. This issue occurs when the current connections are greater than or equal to the minimum connections divided by two. Alternate connections from a single source using sequential port numbers typically fail.
•
CSCsj66157—The ACE may not process client RST packets. Connections persist until the idle timeout expires. This issue occurs when segments are lost prior to RST and they apply to connections in which there were unrecoverable packet loss beyond the ACE in the network.
•
CSCsj74292—With redundancy configured and the peer ACE unreachable, the running configuration is missing after rebooting the ACE. A failure to establish the redundancy Telnet connection to the peer blade during booting of the ACE causes the startup config to be bypassed. The redundancy heartbeats appear to flow between the two peers, but the TCP connection (FT TL connection) fails. The ACE attempts to bring up the TCP connection continuously and fails to timeout the FT groups to the Active state.
•
CSCsj74391—Upon reload of the ACE blade, a gratuitous ARP is not sent for static NAT addresses. This issue occurs because the MAC address for NAT or virtual addresses comes from a 16K pool and can change upon reload. Because the MAC address can change and the ACE does not send a gratuitous ARP for NAT addresses, a network down scenario can result.
•
CSCsj77194—When there are a large number of contexts in a configuration, the ACE may issue an erroneous XML response when you enter the show context command in the Admin.
•
CSCsj89960—Defining the same global static NAT address using a unique port number under different policy-maps generates the following error message: Cannot overlap VIP or NAT address configured in a shared interface.
•
CSCsj90095—During a GET or POST with a large URL header that spans many packets, the response may arrive at the client as out-of-order packets. However, the application is not affected. The TCP stack on the client or server resolves the out-of-order packets.
•
CSCsj91836—When RTSP inspection is enabled on the ACE, some RTSP control connection responses from the server are rejected by the client and no media flows. The problem is observed with a Darwin RTSP server (Quicktime RTSP client) when NAT or LB is configured between the client and server. RTSP connections that were made using a RealPlayer client and Helix Universal Server appear to function as expected. The problem occurs only if the string lengths of pre-NAT and post-NAT IP addresses are different (for example, if 10.0.60.5 is translated to 110.0.60.5). The problem does not occur if no NAT or load balancing is configured between the client and the server.
•
CSCsj97751—Removing the NAT pool terminates the nat_dnld process. The following message appears on the console when this problem occurs:
Service name:nat_dnld(912) has terminated on receiving signal 11When a NAT pool with multiple IP address ranges is removed, the no command to remove the last IP address range causes the failure of the nat_dnld process.
•
CSCsj97786—Removing a service policy from one VLAN interface removes it from other interfaces on which the service policy is configured.
•
CSCsj98311—Enabling redundancy on the Admin context or causing a bulk config sync in an already redundancy-enabled Admin context may cause the redundancy-enabled user contexts to lose their configuration.
•
CSCsk03039— When the ACE executes the show context command through XML, the ctx_vlans tag is not properly closed in the output.
•
CSCsk21143—When an ACE with a higher priority than the current active ACE becomes active, all connections through the ACE are lost. This problem occurred when preemption is configured and the higher priority ACE has the FT peer heartbeat interval command set to a value lower than 200 ms.
•
CSCsk26606—Internet Explorer (IE) closes a connection to the ACE after receiving the server hello and server certificates. Other browsers ignore this error. When the total length of all server and chain-group certificates is higher than 4,024 bytes, the ACE creates two SSL records. One record has a total length field indicating 4,024, but contains a certificate item with a specified length higher than 4,024. The other record has a new SSL record header and the remaining length of the previous SSL record.
•
CSCsk27032—When user contexts have TCP-based probes that are configured with an abortive close by sending a RST instead of graceful shutdown, probes in all contexts fail and never recover. The show conn command output displays connections in the established or time-wait state.
•
CSCsk30079—Operating the ACE beyond its buffering or performance capability may result in a redundancy switchover. When all buffers are used up in the ACE, occasionally, heart beat packets are dropped.
•
CSCsk30714—The ACE eventually stops passing SSL traffic because the buffers are depleted. High levels of SSL traffic can cause buffer depletion.
•
CSCsk30865—When the ACE is running under high stress with SSL configured, the ACE reboots.
•
CSCsk33251—The ACE fails to honor the sticky IP-netmask configuration directive for some, but not all, source IP addresses. For this problem, redundancy, sticky replication and least connections load balancing on the server farm was also configured.
•
CSCsk34973—Changing the class map under a policy map that uses FTP inspect does not change the behavior of the policy map. The original configuration is still used.
•
CSCsk44718, CSCsk46504—If a port channel goes down and then comes back up after a Catalyst 6500 series switch supervisor engine SSO failover, the ACE performs a core dump.
•
CSCsk50407—The ACE changes the source port of (port-NATs) load-balanced traffic arriving at the real servers. Certain applications on the servers require that the traffic arrive from a specific port, and this behavior by the ACE breaks those applications. The traffic experiencing the unwanted port NAT is sent by the client to the VIP address on the ACE. Routed traffic does not experience this problem.
•
CSCsk50946—When using a chain group created from extremely large certificates, the ACE fails. This problem occurs when extremely large certificates are used in a chain group. In this particular example, certificates based on 16K-bit RSA keys were used. There is no commercial CA in existence that supports certificates based on 16K-bit keys and 16K-bit RSA keys are not supported for SSL traffic.
•
CSCsk53677—After you configure the script file index filename command without the script file on the ACE, most show and configuration commands may cause the ACE to hang with either of the following errors:
–
Error: Called API timed out
–
asciigen::The following SAPs did not respond within the expected timeframe
asciigen::Pending SAPS-->506In a redundant configuration, the problem occurs on either or both the active or standby ACE depending on where the file is absent.
•
CSCsk64480—With match source-address commands in a Layer-7 policy for a VIP, any configuration change pertaining to the VIP can potentially corrupt the client map lists causing matches based on source addresses to fail.
•
CSCsk67825—The ACE uses the IP address from an unconfigured NAT pool for NAT. This problem occurs only when the NAT pool with the single instance is removed and then readded along with additional NAT pools with different IP addresses, but with the same NAT-pool ID. After adding multiple instances, if the initially configured NAT pool is removed again from the interface, then the ACE uses the IP address in the removed NAT pool.
•
CSCsk68825—When end-to-end SSL is configured with HTTP inspection and heavy traffic is sent to the ACE, the ACE reboots.
•
CSCsl23515—During an SSL handshake from the client, the ACE may fail to respond with a Server Hello message. The use of SSL termination on the ACE may lead to this behavior.
•
CSCsk91129—Application issues and possible timeouts may occur with connections to an HTTPS VIP address. For example, a GET request is made to obtain data from a server and the returned data is in excess of 125 KB. Though the ACE sends this data to the client, there may be indications that the client has missed some data and requests a retransmission of that data. When this occurs, the ACE may ignore the data retransmission request, which results in a connection timeout. Note that this connection timeout behavior may not occur with a lesser amount of returned data (less than 125 KB).
•
CSCsl24291—The ACE may fail to honor a RST (Reset) with data from a server even after a FIN (Finish) was previously received by the module. In this case, the ACE ignores the RST and retransmits data to the server.
Software Version 3.0(0)A1(6.2a) Command Changes
Table 3 lists the commands and options that have been added in software version 3.0(0)A1(6.2a).
Table 4 lists the commands and options that have changed in software version 3.0(0)A1(6.2a).
Software Version 3.0(0)A1(6.1) Open Caveats and Resolved Caveats
The following sections contain the open and resolved caveats in software version 3.0(0)A1(6.1):
•
Software Version 3.0(0)A1(6.1) Open Caveats
•
Software Version 3.0(0)A1(6.1) Resolved Caveats
Software Version 3.0(0)A1(6.1) Open Caveats
The following caveats apply to software version 3.0(0)A1(6.1):
•
CSCse90603—Autocomplete does not work on checkpoint names. When issuing the checkpoint rollback and checkpoint delete commands, the "?" does not display the existing checkpoint names. Workaround: Use the show checkpoint all command to locate the name for the checkpoint rollback and checkpoint delete commands.
•
CSCsh27991—When sending a large Ping of 50 KB to management IP addresses, all management and through the ACE communication may fail. This condition occurs when you configure the fragment chain limit as greater than the default value. Workaround: Set a low fragment chain limit on the interfaces with the ICMP management policies.
•
CSCsj41909—The ACE fails with the following message appearing in the syslog:
%ACE-5-199006: Orderly reload started at xxxxxxx by System.The reload reason is NP 1 Failed: NP Process Crashed. Workaround: None
•
CSCsj47861—If certain internal flags are non-zero, the demo license installation may fail with the following error message: Installing license... failed: Can't install this license with the current count. Workaround: To recover from this error state, reload the ACE.
•
CSCsj48884—When an FTP client sends a PORT command inside an FTP control channel to an FTP server, the server opens a connection to the client. This connection hits the ACE with the server IP address as the source. The ACE should replace the source IP address with the VIP address. However, if client NAT is configured on the server interface, the ACE gives priority to the policy NAT and translates the source IP address with the policy NAT instead of the VIP, thereby breaking the FTP data connection. Workaround: Create an ACL to explicitly deny the FTP data channel from the NAT policy.
•
CSCsj52843—When you configure a health probe under a real server and the probe fails, the ACE does not generate a syslog for a real server state change (UP or DOWN). If you configure the health probe under a server farm, the ACE does generate syslogs for real server state changes (4442001 and 4442002). Workaround: Configure the health probe under each server farm to which the real server belongs.
•
CSCsj55836—With Layer 7 features configured and certain SSL traffic flowing, new connections may not be processed because the buffers have been consumed by the SSL process. Workaround: None.
•
CSCsj59659—The ACE may reload and cause the network processor (NP) microengine (ME) to become unresponsive as shown in the core dump information. You may observe this issue when stickiness is enabled, all sticky table entries are in use, and heavy traffic that causes a lot of new sticky entries to be created is flowing. Workaround: If more sticky resources are available, allocate more sticky resources for the context.
•
CSCsj64833—The network utility traceroute does not work for a configured ACE IP interface if the underlying protocol is UDP or TCP. Workaround: ICMP traceroute will work. The default of most traceroute utilities is UDP so a command line option might be necessary. For Linux, use the -I option.
•
CSCsj66075—A real server may reject new connections intermittently even though it is in the OPERATIONAL state. This issue occurs when the current connections are greater than or equal to the minimum connections divided by two. Alternate connections from a single source using sequential port numbers typically fail. Workaround: None.
•
CSCsj66157—The ACE may not process client RST packets. Connections persist until the idle timeout expires. This issue occurs when segments are lost prior to RST and they apply to connections in which there were unrecoverable packet loss beyond the ACE in the network. Workaround: Turn off the random-sequence number using a connection parameter map.
•
CSCsj68643—The following log messages may appear sporadically in the ACE log:
–
can_wait_specific_msg: Aborting call (SAP 27, pid 959). Another thread is also waiting for a specific msg.
–
can_wait_specific_msg: Aborting call (SAP 27, pid 905). Another thread is also waiting for a specific msg.
These messages do not impact the operation of the ACE. The messages may be caused by more than one device accessing the ACE context through XML. Workaround: None.
•
CSCsj70522—In a redundant configuration with multiple contexts and logging enabled for the console and the buffer on both ACEs, log messages appear only on the standby console when a probe to a server failed. No new messages are observed in the buffers on either ACE. After entering the changeto command to another context and then back to the original context, log messages appear on the active ACE console, but the log buffer is not updated. Workaround: None.
•
CSCsj74250—When you configure the TACACS server key attribute on the ACE, the key should be encrypted in the show running-config command output. If it is not, there is a key mismatch when attempting to authenticate. Workaround: Paste the properly encrypted key into the running-config.
•
CSCsj74292—With redundancy configured and the peer ACE unreachable, the running configuration is missing after rebooting the ACE. A failure to establish the redundancy Telnet connection to the peer blade during booting of the ACE causes the startup config to be bypassed. The redundancy heartbeats appear to flow between the two peers, but the TCP connection (FT TL connection) fails. The ACE attempts to bring up the TCP connection continuously and fails to timeout the FT groups to the Active state. Workaround: Manually try to Telnet to the peer FT VLAN IP address. If successful, try to remove and then add the FT configuration. If Telneting to the peer fails, the connection to the peer needs to be debugged. Redundancy will not come up if basic connectivity to the peer fails.
•
CSCsj74391—Upon reload of the ACE blade, a gratuitous ARP is not sent for static NAT addresses. This issue occurs because the MAC address for NAT or virtual addresses comes from a 16K pool and can change upon reload. Because the MAC address can change and the ACE does not send a gratuitous ARP for NAT addresses, a network down scenario can result. Workaround: Reduce the ARP timeout on neighboring devices to 3 minutes or to a value less than the ACE reload time.
•
CSCsj77194—When there are a large number of contexts in a configuration, the ACE may issue an erroneous XML response when you enter the show context command in the Admin. Workaround: None.
•
CSCsj80265—With the ACE configured for TACACS+ authentication and SSHv1 management access and the SSH keys generated in RSA1 format, SSH fails to authenticate a user because of a bad password when you attempt to connect to the ACE using an SSH Client. You can connect to the ACE using Telnet and the session works. If you Telnet to the ACE with the same credentials (username and password) that you attempted to use with SSH, then subsequently try to connect to the ACE using SSH, the SSH session is established. Workaround: Use SSHv2 to connect to the ACE by generating the SSH key in RSA format instead of RSA1 format. For example, enter the following command: host1/Admin# ssh key rsa 1024 force.
•
CSCsj88500—If a large configuration contains many different service-policy actions, some of the show service-policy command output may be missing. The ACE may not display some of the actions associated with the service policy. Workaround: None.
•
CSCsj89960—Defining the same global static NAT address using a unique port number under different policy-maps generates the following error: Cannot overlap vip or NAT address configured in a shared interface! Workaround: None.
•
CSCsj90095—During a GET or POST with a large URL header that spans many packets, the response may arrive at the client as out-of-order packets. However, the application is not affected. The TCP stack on the client or server resolves the out-of-order packets. Workaround: None
•
CSCsj91836—When RTSP inspection is enabled on the ACE, some RTSP control connection responses from the server are rejected by the client and no media flows. The problem is observed with a Darwin RTSP server (Quicktime RTSP client) when NAT or LB is configured between the client and server. RTSP connections that were made using a RealPlayer client and Helix Universal Server appear to function as expected. The problem occurs only if the string lengths of pre-NAT and post-NAT IP addresses are different (for example, if 10.0.60.5 is translated to 110.0.60.5). The problem does not occur if no NAT or load balancing is configured between the client and the server.
The following workarounds resolve the issue, but they may be considered restrictive, depending on your network configuration:
–
Disable NAT and loadbalancing between the client and server.
–
Configure NAT or load balancing so that the IP address string length remains the same after translation (that is, from real IP address to mapped IP address or virtual IP address to real server IP address).
•
CSCsj94366—While attempting to modify the console settings using the CLI on the ACE running software version 3.0(0)A1(4a), the following error message appears: console configuration can only be done on console. Workaround: None.
•
CSCsj97786—Removing a service policy from one VLAN interface removes it from other interfaces on which the service policy is configured. Workaround: Do not delete the service policy from the first interface.
•
CSCsk03514—The ACE reboots, possibly associated with SSL traffic. This issue has only been observed in sustaining release A1.4(L). It also appears to be very similar to CSCsh64662, which has been resolved in this image. Workaround: None.
•
CSCsk04298—When a switchover occurs in a Layer 7 load-balancing persistence-rebalance configuration, connections with multiple GETs may not proceed. Workaround: None.
•
CSCsk07539—The ACE module occasionally reloads with the network processor (NP) microengine (ME) unresponsive as shown in the core dump information. You may observe this issue when sticky is enabled, all sticky table entries are in use, and heavy traffic that causes a lot of new sticky entries to be created is flowing. Workaround: If more sticky resources are available, allocate more sticky resources for the context.
•
CSCsk15979—With UDP and TCP class maps sharing the same VIP and both having the idle timer set to infinite, a series of connections are made and allowed to sit idle for several hours. When a change is made to the UDP class map to time out these connections in 60 seconds, the ACE clears the TCP and UDP connections. Workaround: None.
•
CSCsk16083—Changing the file used for hash probes does not fail the probe if the server is a Microsoft Windows 2000 server. Workaround: None.
•
CSCsk34767—CSCsh63231 added FTP inspection statistics to the show stats inspect command output, but the statistics cannot be cleared using the clear stats inspect command. Workaround: None.
•
CSCsk34843—The following messages may be observed on the console connection:
"Data bus error, epc == 2b5008e0, ra == 2b5008c8badvaddr 0x00005f00 cause 0x8cd8fd18 status 0x00007fff<1>A_SCD_BUS_ERR_STATUS 0x810c0000A_BUS_ERR_DATA_0 0x0A_BUS_ERR_DATA_1 0x0A_BUS_ERR_DATA_2 0xffffb12a00000000A_BUS_ERR_DATA_3 0xffff000000000000A_BUS_L2_ERRORS 0x0A_BUS_MEM_IO_ERRORS 0x20000 "Workaround: None.
•
CSCsk34907—Config sync displays an incremental synchronization failure even though the configurations on the active and standby ACEs are in sync. Workaround: None
•
CSCsk34973—Changing the class map under a policy map that uses FTP inspect does not change the behavior of the policy map. The original configuration is still used. Workaround: None.
•
CSCsk44718—If a port channel goes down and then comes back up after a Supervisor SSO failover, the ACE performs a core dump. Workaround: None.
Software Version 3.0(0)A1(6.1) Resolved Caveats
The following caveats apply to software version 3.0(0)A1(6.1):
•
CSCsg80881—After you remove UDP probes from the configuration, the ACE continues to send them. Workaround: None.
•
CSCsh55553—The ACE shows a successful HTTPS probe even when the ACE received an invalid response code. This issue occurred when HTTPS probes were sent to an IIS server that had client authentication enabled. Workaround: None.
•
CSCsh58404—The FTP probe only expects a return code of 150 to determine success. IIS servers send a 125 return code. Workaround: None.
•
CSCsh70061—If you configure an interface MTU of less than 512 bytes, the ACE may reboot while sending packets of greater than 512 bytes. Workaround: None.
•
CSCsi42736—An FTP probe can be successful even when the file is not present on the server. Workaround: None.
•
CSCsj30888—A reload of a non-ACE module in a Catalyst 6500 series switch may cause the ACE to stop passing traffic. Workaround: None.
•
CSCsj32962—A called API timed out after changes were made to a server farm. Workaround: None.
•
CSCsj38329—The ACE does not bridge IP fragments with a destination IP address of 255.255.255.255 and a special data pattern. Workaround: None.
•
CSCsj47437—Entering the show hyp reg command with a nonexistent Hyperion register number causes the ACE to hang, and the Catalyst 6500 series switch supervisor engine reloads the module because of a keepalive failure. Hyperion register numbers range from 0 to 0x6db, but there are some missing register numbers. The ACE now correctly checks for all nonexistent register numbers.
•
CSCsj56916, CSCsj93583—Connections may fail if the load-balancing policy has multiple Layer 7 class maps each with different match criteria, followed by the match source-address command. Workaround: Remove the match source-address command from one of the class maps.
•
CSCsj64745—When a parameter map exists on the active ACE and you use the show parameter-map command for this map on the active ACE, the standby ACE reboots. This issue occurs when the two ACE modules are not running at the same version of code. Workaround: Do not issue the command until both modules are running the same version.
•
CSCsj64871—The documentation needs to clarify the ACE handling of traceroute.
•
CSCsj74599, CSCsj76879—While rolling back a large configuration on the active ACE in a redundant configuration, the standby ACE reboots continuously. To stop the standby from rebooting, reload the active ACE. Workaround: Avoid rolling back a configuration in a redundant configuration.
•
CSCsj78793—The ACE may reboot with a core dump file when a real server in a server farm changes state to OPERATIONAL. This issue does not necessarily require a configuration change.
•
CSCsj96952—An ACE network processor may fail during real server weight list processing. This issue occurs when the least connections predictor is configured and is related to a timing issue that happens when all real servers and the associated VIP go down. Workaround: None.
•
CSCsj97767—A client using a TCP window size of greater than 65,535 while attempting to make an SSL-terminated connection with the ACE experiences significant delays in receiving responses. Setting the TCP window size on the client to any value that requires TCP window scaling (that is, greater than 65,535) causes this delay. Note that this is a client configuration, and not an ACE configuration directive. Workaround: Do not set the TCP window size on the client to any value that requires TCP window scaling.
•
CSCsk01396—Various connectivity issues occur beginning with probe failures. After several days, the ACE can not ping a directly-connected real server. However, the same real server can successfully ping the ACE. These issues sometimes recover spontaneously and connectivity is restored, but eventually they do not recover. The show resource usage all command displays that management connections are denied in the affected context in the millions and that current management connections are several thousand. However, the show conn command shows a low level of two connections to any local IP address on the ACE.
When the ACE is configured with a logging host on a remote subnet that does not accept syslog messages, it causes the return of ICMP error messages for each syslog message. Also, there may be a high volume of logging to the server, causing large numbers of errors to be sent.
Workaround: The ACE recovered when the logging host was removed from the configuration or the flood of ICMP error messages was stopped, as verified by the return of the current count for management connections to its normal level in the show resource usage all command. If the current count for management connections did not recover and the current count is much greater than the number of connections shown to all the ACE addresses in show conn command, reboot the ACE.
•
CSCsk03319—The fix for CSCsh55553 causes frequent SSL probe failures. This defect tracks the removal of CSCsh55553 from the code. Workaround: None.
•
CSCsk08018—In a redundant and bridged configuration, TCP-based probes fail to complete the three-way handshake in a user context in the new standby ACE after a switchover. Telnet and SSH from VSH still work. Workaround: To resolve the problem, enter the clear rtcache or the clear arp command.
•
CSCsk27032—When the ACE is in bridge mode and you are using a TACACS+ server, the ACE may leave the TACACS+ connections half-opened. This issue may cause all management resources to be consumed. Workaround: If you observe management connections being dropped, enter the clear arp command.
Software Version 3.0(0)A1(5a) Operating Considerations, Open Caveats, Resolved Caveats, and Command Changes
The following sections contain the open and resolved caveats and command changes in software version 3.0(0)A1(5a):
•
Operating Considerations for 3.0(0)A1(5a)
•
Software Version 3.0(0)A1(5a) Open Caveats
•
Software Version 3.0(0)A1(5a) Resolved Caveats
•
Software Version 3.0(0)A1(5a) Command Changes
Operating Considerations for 3.0(0)A1(5a)
With version 3.0(0)A1(5a), when you make a change to a chain-group certificate, the change takes effect after you respecify the associated chain group in the SSL proxy service using the chaingroup command in SSL proxy configuration mode. Prior to version 3.0(0)A1(5a), you had to remove the chain group from the SSL proxy service using the no chaingroup command, and then readd the chain group using the chaingroup command.
Prior to Version 3.0(0)A1(5a), when a bulk configuration synchronization (config sync) occurred in the Admin context and there was a difference between the active and the standby running-config files, the ACE rolled back the running-config on the standby, and then applied the active running-config to the standby. This action deleted and then re-added all the contexts and FT groups on the standby. As a result, all standby files (for example, SSL certs, keys, scripted probe files, and so on) were deleted and you had to manually reinstall them.
With Version 3.0(0)A1(5a), when a bulk sync occurs in the Admin context, the ACE compares the active and the standby running-config files on the standby, computes the difference (if any), and stores the difference in a file. Only if there is a difference, the ACE performs the following steps:
•
Sets all user FT groups to the MAINT_MODE_SBY_COLD state. This action causes the user FT groups to go to the STANDBY_COLD state. Any user FT groups that were active now become active on the standby.
•
Rolls back the configurations of all user contexts. This action clears the user context configurations by issuing a no command on each configuration line, but keeps the file system and files intact.
•
Applies the diff file that it computed previously.
•
Sets all user FT groups to the MAINT_MODE_OFF state. This action triggers the redundancy finite state machines (FSMs) for the user FT groups, causing them to go through a bulk sync, and eventually puts the user FT groups in the STANDBY_HOT state.
At this point, the Admin FT group is finished with config sync and transitions to the STANDBY_BULK state if the ACE applies the diff successfully, and then transitions to the STANDBY_HOT state.
If the ACE fails to apply the diff, the Admin FT group moves to the STANDBY_COLD state.
Note that a bulk sync in a user context is different now. Prior to version 3.0(0)A1(5a), if there was a diff, the ACE rolled back the standby running-config file and applied the active running-config file.
With Version 3.0(0)A1(5a), the ACE computes a diff and applies only the difference.
You may notice longer bulk sync times particularly in the Admin context. Prior to Version 3.0(0)A1(5a), the ACE deleted all the contexts with the no context context_name command. With Version 3.0(0)A1(5a), the ACE issues a no command on each configuration line in each context, which may take longer, especially when the context configurations are large.
Software Version 3.0(0)A1(5a) Open Caveats
The following caveats apply to software version 3.0(0)A1(5a):
•
CSCsc25259, CSCsi75907—Packets continue to be captured even after the packet capture feature is explicitly disabled. High CPU utilization may be observed. Workaround: Clear the connection that is being debugged or use an external packet capture tool.
•
CSCsd80505—Repeated Layer 4 performance tests provide a result of only 260Kcps. This issue occurs when a test exercises the ACE beyond its system limitation of 330Kcps. Workaround: Make sure that test parameters are within 330 Kcps.
•
CSCse08728—The ACE does not have a provision to back up all the contexts in bulk and restore them in a single operation. Workaround: Save the configurations for each context individually and repeat the process when restoring the configurations.
•
CSCse23076—With continuous RTSP traffic, you may observe streaming packet loss. This issue may be a minor effect of video or audio quality on the end host. Workaround: None.
•
CSCse64922—If the virtual server state is OUTOFSERVICE, the ACE may incorrectly update the dropped conns counter under the L7 loadbalance policy section of the show service-policy policy_name detail command. Workaround: None.
•
CSCse70920—The ACE does not indicate whether a connection was selected from the reuse pool in the connection setup syslog (%ACE-6-302022). Workaround: None.
•
CSCse72735—The ACE does not send traffic from one context to another (on the same module) over a firewall service module in transparent mode. Workaround: None.
•
CSCse79208—The no logging message message_id level level command does not disable message logging at the specified level. Workaround: Disable the message logging without the level option, then reenable it for the desired levels.
•
CSCse85906—In a redundant configuration, persistent connections established through the ACE context become unresponsive after a switchover. Workaround: None.
•
CSCse89405—The delta for application inspection statistics cannot be obtained using the command show np np me-stats "-M n -s appinspect" command. Workaround: None.
•
CSCse90603—Autocomplete does not work on checkpoint names. When issuing the checkpoint rollback and checkpoint delete commands, the "?" does not display the existing checkpoint names. Workaround: Use the show checkpoint all command to locate the name for the checkpoint rollback and checkpoint delete commands.
•
CSCse95074—The Total Count field value in the output of the show serverfarm name retcode [detail] command is not preserved for real servers that are taken out of service. The ACE resets the counter to zero. Workaround: None.
•
CSCse96401—When you are configuring the boot string, you may encounter the following error: "write_block failed after attempt, sr=0xb BF: error write". On subsequent reloads, autoboot may fail and the ACE displays the following errors:
–
"out of room in table, cannot set "ARGV0""
–
"Autoboot: failed, boot string not found" errors.
Workaround: At the rommon prompt, enter the following sequence of commands:
–
priv
–
erase
–
sync
–
reset
This command sequence clears the corruption of the boot flash configuration parameters. If the ACE module does not autoboot, use the boot disk0:image_name command to proceed.
•
CSCsf01854, CSCsd58768—Under high TPS load and with front-end SSL and six chained certificates configured, the ACE may reboot and go to rommon. The following message may occur on the console: "Buf alloc failed. Out of Buffers". Workaround: Use fewer than six certificates in the certificate chain.
•
CSCsf02492— If a new real server is added to a server farm that is configured with the least connections predictor and slowstart, the ACE may not apply the slowstart mode to the new server if these actions occurred:
–
You added the real server to the server farm
–
You changed an existing real server weight to the default value
Workaround: Change the weight of the existing real server before adding the new real server to the server farm.
•
CSCsf03118—In a redundant configuration with persistence-rebalance configured, if the second GET request in the same connection is load balanced to another real server in the server farm, the active ACE context may not replicate the connection to the standby context correctly. If a switchover occurs at this time, subsequent GET requests are forwarded to the original real server, which responds with a reset (RST). Workaround: None.
•
CSCsf06622—The ACE may reboot unexpectedly and go to rommon as the ACE prepares to service an issued reload command and the "last boot reason" output of show version command is set to process "reboot." This issue is benign. Workaround: None.
•
CSCsf06628—Under certain conditions, the hash value in the ACL deny syslog may not match internal hash values displayed in the output of the show access-list acl_name command. If you specify partial (but valid) keywords while configuring the entries in the ACL, the ACE computes the hash value on the partial configuration string. However, after a reboot, the ACE computes the hash value on the complete command string that is in the startup configuration. This will result in inconsistencies between the hash values shown in the syslog before reload and hash values shown in the show access-list acl_name CLI after reload. Workaround: Tab complete (or type entire keywords) in the CLI when configuring individual entries in the ACL.
•
CSCsf07297—If you try to modify a NAT configuration by cutting and pasting the configuration into the ACE console, the existing NAT pool entries are not correctly deleted. Workaround: Manually execute the following configuration sequence instead of using the cut-and-paste method:
–
Delete the current NAT dynamic statement
–
Delete the NAT pool
–
Add the new NAT pool
–
Add the new NAT dynamic statement
•
CSCsf30686—The TPS maximum is 25,000 when you configure the persistence-rebalance feature, in which each HTTP request within the connection is load balanced. Workaround: None.
•
CSCsg06046—With persistence-rebalance configured, if a client opens a single HTTP 1.1 persistent connection and sends multiple GETs with all requesting the same data, each GET causes the ACE to tear down the server flow, load balance to a new server, and establish a new server flow for each one. Workaround: None.
•
CSCsg23096—The show conn serverfarm command does not display FTP data connections. Workaround: None.
•
CSCsg49360—When using the no context command, you may delete a context without a warning. Workaround: Use this command cautiously.
•
CSCsg49375—Issuing the no context command deletes saved checkpoints and files from disk0. There is no way to recover them. Workaround: None.
•
CSCsg57448—A stress condition (100,000 total connection) to an SSL initiation VIP address generates resets (RSTs) for some of the connections to the front-end client. Workaround: None.
•
CSCsg74733 —In a routed network design, the network analysis module (NAM) cannot see all packets on VLANs allocated to the ACE although traffic is flowing correctly. Workaround: Use port mirroring and an external packet capturing tool.
•
CSCsg77694—Server load balancing fails when you configure a VIP address with a mask. Workaround: Configure static ARP or routes as needed on all upstream routers for all the IP addresses in the specified range.
•
CSCsg80625—The ACE may drop UDP packets addressed to a VIP address if the packet size is greater than 1,500 bytes. Workaround. Use a packet size that is less than 1,500 bytes.
•
CSCsg80881—After you remove UDP probes from the configuration, the ACE continues to send them. Workaround: None.
•
CSCsg88014—Copying an FTP image fails when overwriting the same filename. Workaround: Delete the file and recopy it.
•
CSCsg91127—A license mismatch may occur when a new resource class and a new context is added in an HA setup. Workaround: Delete the context and re-add it.
•
CSCsh19382—SSL bulk throughput is only 3.4 Gbps. Workaround: None.
•
CSCsh20546—If the maximum connections, persistence rebalance, and sticky features are all enabled on the ACE, the maximum connections feature does not work. Workaround: Do not use all three of these features on the same VIP/server farm.
•
CSCsh22888—If an SSL certificate is missing or incorrect, the standby ACE may reset on reboot. Workaround: Make sure that the standby ACE has the correct certificate.
•
CSCsh22894—The SSL handshake for all SSL connections going to an SSL service will fail because of an internal failure. The ACE sends an SSL fatal alert message before closing the connection. Workaround: If the key-certificate pair was self-generated, manually generate a new pair to replace the problem pair. The new pair may work.
•
CSCsh27991—When sending a large Ping of 50 KB to management IP addresses, all management and through the ACE communication may fail. This condition occurs when you configure the fragment chain limit as greater than the default value. Workaround: Set a low fragment chain limit on the interfaces with the ICMP management policies.
•
CSCsh42422—When configuring real servers and server farms, the ACE may not resolve the ARPs for the real servers. Workaround: Remove the real servers and the server farms from the configuration and then re-add them. This action causes the ACE to resolve the ARPs.
•
CSCsh54479—SSL throughput drops below 4 Gbps. Workaround: None.
•
CSCsh86146—If you have traffic flowing and you enter the ft switchover command (soft transition), the SSL traffic may remain on the standby ACE. Workaround: None.
•
CSCsi14578—The ACE delivers poor performance when balancing POSTs of large files over SSL terminated connections even at relatively low connection rates. This problem may be seen with only a single client (single source IP) and multiple connections to a single VIP on the ACE. Workaround: None.
•
CSCsi52045—When you use a Layer 7 load-balancing configuration with multiple connections per second occurring, the ACE may drop some packets. The ACE drops the packets when the connection moves from the proxied to the unproxied state after the ACE sends the first parsed request to the real server. Workaround: To prevent the ACE from unproxying connections and dropping packets, configure the set tcp wan-optimization rtt 0 command in a parameter map and apply it to the Layer 4 class under the Layer 4 multi-match policy.
Note
Using the set tcp wan-optimization rtt 0 command keeps Layer 7 connections proxied until they are closed. Therefore, you may observe that system performance degrades.
The following example shows a sample configuration with the set tcp wan-optimization rtt 0 command:
parameter-map type connection STAY_PROXIEDset tcp wan-optimization rtt 0policy-map multi-match LAYER4_POLICYclass LAYER4_CLASSconnection advanced-options STAY-PROXIED•
CSCsj08692—The standby ACE becomes unresponsive when the active ACE is reloaded.ok Workaround: None.
•
CSCsj12610—A script that periodically runs the show version command causes the ACE module to reload after 30 hours. Workaround: None.
•
CSCsj14813—When the SSL cert/key pair is modified and the modification results in a mismatch, the configuration download fails. Workaround: None.
•
CSCsj17220—The load-balancing receive queue (LBRX) may become stuck and does not drain. Workaround: Increase sticky resources.
•
CSCsj20832—The ACE becomes unresponsive due to an ME-Deadlock with Layer 4, Layer 7, and SSL traffic. Workaround: None.
•
CSCsj21032—Configuration synchronization for contexts with large configurations takes a long time on the standby. The API to compute the difference between the config on the standby and the configuration shipped from the active needs to be more efficient. If there is an error in applying the difference, there needs to be an API to roll back only the applied difference. Workaround: None.
•
CSCsj37029—Pushing the ACE beyond its performance capability may result in poor SSL performance with large GET or POST page sizes. Redundancy may also switch over multiple times under the same circumstances. When you use certain ciphers, redundancy may switch over multiple times. Workaround: None.
•
CSCsj38329—The ACE does not bridge IP fragments with a destination IP address of 255.255.255.255 and a special data pattern. Workaround: None.
Software Version 3.0(0)A1(5a) Resolved Caveats
The following caveats apply to software version 3.0(0)A1(5a):
•
CSCsc62866—Occasionally VSH core files appear in the core directory. This issue is benign and does not affect any functionality.
•
CSCse23330—With the Catalyst 6500 series switch supervisor engine in a blocked state, the ACE trunk interface does not block traffic. When the BPDU fixup is enabled, a misconfigured redundancy configuration resulting in an active-active state causes the spanning tree to put one of the ports in a blocking state to break the loop. If the blocking port is the ACE trunk interface, it does not go into the blocking state even though the ACE status reports it.
•
CSCse31852—For Layer 7 load-balanced connections, if the maximum segment size (MSS) sent by the client is greater than 1420 bytes, then the MSS value in the TCP SYN segment sent to the server is restricted to 1420 bytes. If the MSS sent by the client is less than 1420 bytes, then the ACE sends the same MSS to the server. This is seen only when normalization is enabled. Workaround: None
•
CSCse37639—If all the real servers under the primary server farm go down and a backup server farm is configured, the ACE does not load balance incoming connections to the backup server farm, even when all the real servers under the backup server farm are up, unless the aggregate-state option is configured.
•
CSCse39448—After an active ACE module that was reloaded comes back up in a redundant configuration (interchassis topology), traffic between two hosts may be dropped for approximately 10 seconds. You may observe this issue with preempt configured, which ensures that the newly reloaded ACE will become the active member of the FT group.
•
CSCse63796—For each real server, the ACE module supports 16 TCP option combinations in the reuse pool. The supported TCP options are:
–
Maximum segment size (MSS)
–
Selective acknowledgement (SACK)
–
Timestamp (TS)
–
Window scale (WS)
If all 16 combinations are exhausted and a new one has to be created, then the least recently used combination is replaced correctly, but the connections and proxy IDs present in that pool are not closed.
•
CSCse65505—If an incremental synchronization fails because a certificate or probe is not imported on the standby ACE, configuration synchronization between ACEs is not disabled. This issue occurs when using a certificate or a scripted probe in a redundant configuration. It is expected that the files containing the script or the certificates are imported on both the active and standby ACEs so that configuration synchronization occurs correctly. If the file is missing from the standby ACE, the synchronization cannot occur. Workaround: Use FTP to copy the file to both active and standby ACEs.
•
CSCse90214—In a redundant configuration, the peer context associated with an FT group may be configured with a higher priority, which causes a switchover when it is brought into service. With this sequence of events, connections that were established through the previously active context are lost. Workaround: Bring up the peer FT group with a priority and preempt configuration so it does not trigger an immediate switchover. Once the Active - Standby Hot states are reached and connections are replicated, the switchover may then be triggered by appropriate configuration changes or Exec commands.
•
CSCse95336—The transfer of a large page (10 MB) between a single client and server may generate several short record SSL errors and occasionally bad MAC record errors at the end of the connection. This condition occurs infrequently and is not easily observed.
•
CSCsf03222—In a redundant configuration where the active Admin context autosync state is being toggled and the standby context has autosync disabled, the running config in a user context (standby) may be lost as a result of a configuration synchronization issued by the active ACE. Workaround: Enable autosync of the running-configuration in the user context before enabling it for the Admin context.
•
CSCsf06159—In a redundant configuration with configuration synchronization enabled, the ACE may reboot and go to rommon when processing the configuration of a large ACL (100 K entries). This may occur even if there is no traffic through the Active module at the time of configuration.
•
CSCsf10038—In a redundant setup, connections will not be replicated during the upgrade from the following images:
–
Image 3.0(0)A1(2) to 3.0(0)A1(3), 3.0(0)A1(3b), or 3.0(0)A1(4)
–
Image 3.0(0)A1(3) or 3.0(0)A1(3b) to 3.0(0)A1(4)
The recommended upgrade procedure is as follows:
1.
Save the configuration on each ACE module by using the write mem all command.
2.
Copy the 3.0(0)A1(3), A1(3b) or 3.0(0)A1(4) image to both ACE modules using the copy ftp command.
3.
Configure the boot parameters to boot from the image. Use the show boot command to verify the boot configuration.
4.
Upgrade the ACE module that has the Admin context in the StandbyHot state first. Use the show ft group detail command to verify the state of the module. Reload this module. After the module has booted up, it may take four to five minutes to reach the StandbyHot state again. Note that the connections through the active ACE module will not be replicated to the standby module.
5.
Upgrade the other ACE module by reloading it.
At this point, a switchover takes place. The standby module becomes the new active module.
If preempt is set, the module is restored to the original active or standby state. Otherwise, the newly rebooted module comes up and becomes the standby module.
6.
Use the show ft group detail command to verify that the Active-StandbyHot state has been reached on the peer modules.
•
CSCsf15331—The ACE may reboot if more than 400 HTTPS probes are configured. Workaround: Configure fewer than 400 HTTPS probes. We also recommend that you limit the number of Concurrent-openssl connections to 50 or less.
•
CSCsf12347, CSCsg95237— A persistent connection may reset when a new HTTP request arrives after any change in the server farm configuration is completed.
•
CSCsf30183—The ACE may lose IP connectivity after loading SSL traffic (after approximately 10 minutes). If this condition occurs, the ACE stops switching traffic until you reload the module. This behavior is seen with export ciphers only. Workaround: None.
•
CSCsg06003—The show service-policy command displays multiple current connections under a NAT class map. However, the show xlate and show conn commands display no connections. This issue does not affect functionality. Workaround: None.
•
CSCsg10024—A Layer 7 FTP policy blocks the cdup command, but allows the cd command. This issue enables you to change to the root directory, negating any effect of blocking the cdup command. Workaround: None.
•
CSCsg10072—The inspect ftp strict command should mask the user 530 reply code with a 331 reply code and the SYST command with XXXXXXXXXX, but it only works for the user command.
•
CSCsg23016—The class map insert-before option does not work on a previously-configured class map. Workaround: Delete the class map and configure it again.
•
CSCsg24950—When changing from one context to another, the CLI prompt is carried across to the other context. For example, when configuring a server farm and then changing to a different context, the server farm prompt appears in the second context even though this context does not contain any server farms. Workaround: None.
•
CSCsg26146—When you are logged in as a Network-Admin user and you use XML to run show commands, you may receive the following error message:
<status code="110" text="XML_ERR_CHECKPOINT_MAX_LIMIT"><error_message>reached max checkpoint limit 10</error_message>.
When you are logged in as Admin, entering the same show commands does not produce an error. Workaround: None.
•
CSCsg40029—After a switchover, ARP RESPONSE collision messages may appear on the standby ACE for two minutes. This issue is benign and is a rare event. Workaround: None.
•
CSCsg42860—After the Catalyst 6500 series switch supervisor engine switches over, the ACE resets to 8 Gbps even though it was 16 Gbps before the supervisor switchover. Workaround: None.
•
CSCsg47106—The ACE may not stick on an HTTP cookie inserted with the cookie insert command and you configured the replicate sticky command in a redundancy configuration. This failure also causes return traffic to contain a Set Cookie: message, which updates the browser to use the newly chosen server. Workaround: Configure a static cookie on each sticky group that uses the cookie insert command after the redundancy status has stabilized. This cookie does not require a particular value. This workaround does not persist across reboots.
•
CSCsg55692—When you enable server connection reuse with persistence-rebalance, some connections may see a reset or timeout. Workaround: None.
•
CSCsg74903—When you enter the no logging console command in any context, console logging to all contexts stops. Workaround: None.
•
CSCsg87825—The VSH process core dumps on exiting. You can ignore this core file. Workaround: None.
•
CSCsg89266 —When you add new class maps to a policy map after the default class map configuration, the new class maps are not activated. Workaround: Remove the default class map and re-add it after you configure the new class maps.
•
CSCsg97750, CSCsh72122—Capture files that are generated when the capture buffer is copied to disk may not be sent by TFTP to a server when the capture file name starts with "ace." Workaround:
–
Do not use the word "ace" in the capture filename.
–
Instead of using the copy capture buffer_name command, use the show capture buffer_name filename command.
•
CSCsg98273—When the ACE receives a legitimate ICMP error packet, it recalculates the ICMP checksum but not the embedded IP header checksum before forwarding it to the next hop. If the NAT fixup is operated on the original IP header, some clients may flag this checksum error when it receives the ICMP error packet.
•
CSCsh02651—Transfer-encoding type chunked regex does not work for the HTTP inspection feature.
•
CSCsh10834—Host: tag contains unintelligible text, which results in an HTTP 400 bad request returned by the server.
•
CSCsh11119—Cookie insert may be configured successfully on the ACE, but later stops working, preventing client requests from sticking to their servers as configured.
•
CSCsh12487—The ACE may reboot if the VLAN is configured with the no normalization command and a TCP reset packet with a zero TCP Header Length enters the ACE. Workaround: Do not disable normalization.
•
CSCsh12673—When the ACE is configured with a dynamic NAT policy fails during a rollback, the following error message appears: "Error: Cannot delete nat pool when it is referenced in nat dynamic!" When removing an interface that refers to a NAT pool through the nat-pool command and is in use by a NAT dynamic policy, the following error message appears: "Cannot delete interface when it has nat-pool in use." Workaround: None.
•
CSCsh29765—If generic parsing is configured on the ACE, traffic may become stuck to one real server even when no pattern has been matched.
•
CSCsh12994, CSCsh62761, CSCsh88414—The no nat-pool command fails if you enter it immediately after the no nat dynamic command and the dynamic NAT policy fails during rollback with the following error: "Error: Cannot delete nat pool when it is referenced in nat dynamic!". When you remove the interface that refers to a NAT pool that is in use by a dynamic NAT policy, the following error appears: "Cannot delete interface when it has nat-pool in use."
•
CSCsh14278, CSCsh81195—A reset for an established connection was being accounted for as a connection failure. Workaround: None.
•
CSCsh17204—The ACE inserts an empty cookie value if the connection is load balanced to a backup server farm. Workaround: Do not configure a backup server farm with cookie-insert sticky.
•
CSCsh17540—Configuring a VIP that only allows certain source IP addresses to access it may cause the ACE to respond to an HTTP GET with a SYN-ACK, which prevents the page from loading. Workaround: None.
•
CSCsh19260—The ACE may reset if EXPORT ciphers are used. Workaround: None.
•
CSCsh19664—Using scripted probes may result in the use of management connection resources, thereby preventing the setup of management connections to the ACE or to subsequent probes. Workaround: None.
•
CSCsh19923—The end host (client or server) discards ICMP error packets received for TCP connections. This issue is caused by the ACE corrupting the ICMP checksum of ICMP error packets that are generated by intermediate routers and end hosts for TCP connections. End hosts that rely on these ICMP error packets for their functionality (for example PMTU Discovery) are affected. Workaround: This workaround applies only to the Type 3 Code 4 (Fragmentation needed but DF bit set) ICMP error. On the interface that receives this error packet, configure the ip df clear command in interface configuration mode.
•
CSCsh20636—While adding or removing contexts with large configurations and running at a rate of 4 Kcps, the interface manager (IFMGR) process fails. Workaround: None.
•
CSCsh23922—When you take a real server out of service, active long-lived HTTP connections may also be torn down for other real servers in the same server farm.
•
CSCsh27649—High CPU utilization has been observed with ICMP probes. The problem can occur in large configuration (100 contexts and 50 real servers in each context) when an ICMP probe is associated with each of the real servers. The ICMP probes failed because the Catalyst 6500 series switch supervisor engine VLANs were not configured. Workaround: None.
•
CSCsh27991—When sending a large Ping of 50 KB to management IP addresses, all management and through the ACE communication may fail. This condition occurs when you configure the fragment chain limit as greater than the default value. Workaround: Set a low fragment chain limit on the interfaces with the ICMP management policies.
•
CSCsh28128—When strict FTP inspection is enabled, the ACE resets FTP connections to an MSFTP server when a user logs into the module. Workaround: Disable the strict FTP inspection feature by using the no inspect ftp strict command in policy map class configuration mode. Instead of using strict FTP inspection, use the regular FTP inspection feature by entering the inspect ftp command in policy map class configuration mode.
•
CSCsh28624—When using the crypto configuration commands, a reference to a file that is unreadable or nonexistent may cause the ACE to reboot, generating a memory dump file.
•
CSCsh31720—When any additional real servers are disabled so that only one remains operational, connections may not get load balanced. Workaround: None.
•
CSCsh41184—The ACE may leave a half-closed incomplete HTTP connection in the ESTABLISHED state.
•
CSCsh41405—An SSL core dump and reload may occur during SSL testing. Workaround: None.
•
CSCsh41546—Clients running Internet Explorer 7 see large POSTs to SSL terminated sessions get a "Malformed multipart POST" error response from the server. This problem is caused by the ACE inserting 16 bytes of unknown data. Workaround: Use a client other than Internet Explorer 7, such as Internet Explorer 6.
•
CSCsh44015—IP traffic that is passing through the ACE always has the type of service (TOS) field cleared. Workaround: None.
•
CSCsh45160—The ACE becomes unresponsive when there is heavy traffic to or from the control plane. An example of heavy traffic is an TFTP download or SNMP or XML polling.
•
CSCsh49427, CSCsh76216—Configuring the default SSL parameter map on the ACE and receiving a large (greater than 64,000 byte) reply to a GET may cause a memory dump of the ACE microengines.
•
CSCsh52210—In an SSL configuration, an ACE redirect real server fails when trying to redirect traffic. Redirect traffic may be transmitted in clear text. Workaround: None.
•
CSCsh53251—When changing the predictor from round robin to least connections or the reverse, and also adding or removing a real server, the ACE may become unresponsive because of list corruption. Workaround: None.
•
CSCsh54615—Domain checking for show serverfarm, show rserver, and other show commands does not work. Users are allowed to access objects that are outside their domain. Workaround: None.
•
CSCsh55916—The expect status in an HTTP probe does not get added when the lower or upper limits use values 998 or 999. Workaround: None.
•
CSCsh56158—The ACE sends TCP segments that are larger than the client MSS. Workaround: Reenable normalization on the ingress interface.
•
CSCsh59599, CSCsi05380—With predictor leastconns and sticky configured, not all servers are selected because the connection count has an invalid value. Workaround: Trigger a re-download to the data plane.
•
CSCsh59748—When you enter the clear logging command, the ACE clears the buffer but not the memory the buffer is holding. Workaround: None.
•
CSCsh61341—When you configure a scripted probe and a server farm on the active ACE, the configuration is copied to the standby ACE in an incremental sync. The standby shows the server farm as operational even though the scripted probe remains in an init state. Workaround: None.
•
CSCsh62913—The ACE responds to the FTP syst command with a different number of X characters depending on the operating system of the FTP server. Workaround: None.
•
CSCsh63169—In a Layer 7 (L7) load-balancing configuration in which a policy-map is applied to multiple interfaces, if the default class is configured first on the policy map followed by the L7 match, the L7 policy map may not be applied properly to the second interface. Traffic that should hit the L7 match may hit the default class instead. Workaround: Configure the persistence-rebalance command in the parameter map HTTP configuration mode and then reboot the ACE.
•
CSCsh63231—The show stats inspect command displays only HTTP inspect statistics. It does not does not display the statistics for DNS, FTP, ICMP or RTSP because it is not implemented for these protocols. Workaround: None.
•
CSCsh64473—When you delete an interface, the ACE generates internal syslog error %ACE-3-440003: Deletion failed for Shadow IP Addr Table. Workaround: None.
•
CSCsh64549—Changing sticky resource allocation may cause the ACE to become unresponsive because load balancing utilization could reach 100%. Workaround: None.
•
CSCsh64662—The ACE may experience microengine memory dumps while forwarding SSL traffic when using ciphers RSA_WITH_3DES_EDE_CBC_SHA, RSA_WITH_AES_128_CBC_SHA, or RSA_WITH_AES_256_CBC_SHA. The issue is not seen when using cipher RSA_WITH_RC4_128_MD5.
•
CSCsh66336—If you configure the script file command prior to loading the script onto disk0:, the script does not get loaded into memory and the script code cannot be viewed. Workaround: Copy the script file onto disk0: before configuring the script file command.
•
CSCsh66714—When a VIP is taken out of service because of dynamic changes and later comes back into service, that information is not propagated to the data plane, causing the ACE to drop connections. Workaround: None.
•
CSCsh70061—If you configure an interface MTU of less than 512 bytes, the ACE may reload while sending packets of greater than 512 bytes. Workaround: None.
•
CSCsh71875, CSC27089—In a redundant configuration with sticky replication configured, a reset may occur on the standby ACE during an upgrade from software versions 3.0(0)A1(2), 3.0(0)A1(3), or 3.0(0)A1(3b) to 3.0(0)A1(4). Upgrading from all versions of 3.0(0)A1(3) and 3.0(0)A1(4) to 3.0(0)A1(5) is supported.
•
CSCsh76214—The output of the show stats connection command may display the Total Connections Destroyed counter as incrementing when timed-out connections are observed.
•
CSCsh77775—An old script file cannot be removed from memory or replaced with a new one. Workaround: Enter the no script file filename command and reboot the ACE module.
•
CSCsh78960—Incomplete connections are observed in the connection database. Workaround: Use the clear conn command to clear the incomplete connections.
•
CSCsh81061—The ACE may reboot while executing the show service-policy command if you configured a service policy with a large number of VIPs (virtual servers). Workaround: Change or split up the service-policy configuration into multiple service policies with fewer VIPs.
•
CSCsh84238—Occasionally, a redirect policy does not send HTTP 302 messages to the client. Workaround: None.
•
CSCsh88518—The ACE becomes unresponsive upon executing the ft switchover force command. Workaround: None.
•
CSCsh91449—When you try to copy a file from the ACE to an FTP or SFTP server, you may receive the following error: "Couldn't get handle: Permission denied or Couldn't open local file <path> for reading: permission denied." Workaround: None.
•
CSCsh92224—In a redundancy configuration, after you reboot the standby ACE module, some parts of the Admin context configuration on the module are deleted. This problem occurs when configuring the no ft auto-sync run and no ft auto-sync start commands in the Admin context. The deleted configuration in Admin context includes the SNMP configuration and additional contexts. Workaround: Do not use the no ft auto-sync run and no ft auto-sync start commands in the Admin context.
•
CSCsh97150—When you enter the show tacacs-server command, a TACACS+ server from the Admin context appears in another context that is not configured with that server. Workaround: None.
•
CSCsh97657—Class map configurations that exceed 128K regular expression states may cause the ACE to reboot from the configuration manager (cfgmgr) process. Alternatively, the ACE does not forward the traffic matching of one or more configured classes using the specified policy. Workaround: Do not exceed 128K regular expression resource states. In particular, avoid configuring class map matches that include a wildcard match at the beginning of or internal to the match string.
•
CSCsh97711— After you configure the aaa authentication login error-enable command in the active Admin context, the ACE does not replicate the configuration in the standby context. Workaround: None.
•
CSCsi05384—Adding or removing certs in a chaingroup that is applied to an SSL proxy server does not succeed. The CLI shows that the change has taken place, but the certs that have been changed are not sent to the client. Workaround: If you remove the chaingroup from the SSL proxy server and re-add it, the ACE sends the certs in the chaingroup to the client.
•
CSCsi05456—Adding or removing a real server associated with a VIP may cause connections to be sent to it even if the probe failed. Workaround: None.
•
CSCsi07482, CSCsh95001—Management connections and health probes fail when probes are configured with more than 1024 real servers. Workaround: None.
•
CSCsi14233, CSCsi72877—After the standby ACE is manually rebooted, SSL offload traffic does not work on the standby. The following symptoms are seen:
–
The show crypto file command displays the cert and key, but the crypto verify command fails (pass before reload), and the key disappears from the next show crypto file command output.
–
The cert storage directory does not exist for the context.
–
Commands disappeared from the running-config in the Admin context. Non-Admin contexts disappear as well as ip route commands.
Workaround: Replace the commands in the Admin context. Import missing crypto files again.
•
CSCsi15461—With a Layer 7 load-balancing configuration, connections may not close. Workaround: None.
•
CSCsi17037—An sslHs process core dump occurs when using SGC/Server Step-up certificates with the ACE. Workaround: None.
•
CSCsi20592—The ACE CLI becomes unresponsive after a critical process (system manager) fails. Workaround: None.
•
CSCsi23932—If you enter the clear sticky command on the active ACE, sticky entries on the active ACE are cleared, but remain on the standby ACE. Workaround: None.
•
CSCsi24886—Bulk sync fails when the ACE configuration has commands with multiple words and quotes.
•
CSCsi29549—Some file system commands, such as copy, may fail after a large number of users log in and log out of the ACE. Workaround: None.
•
CSCsi32632—When running a load-balancing configuration that is terminating SSL connections, a significant performance degradation is seen when transferring (using either POST or GET) large files (greater than 4 MB) with a relatively low connections per second rate. Workaround: None.
•
CSCsi39584—After a reboot, static sticky entries may not be created because of delayed resource allocation for the context. Workaround: None.
•
CSCsi39717, CSCsi28907—In a redundancy configuration where one ACE is running image A1.4, A1.4a, A1.4b or A1.4c and the other ACE is running an image A1.4d through A1.4l, configuration synchronization may fail. This problem is most likely to occur during an upgrade where one ACE has been upgraded to one of the later images listed and the other is still on an older image.
•
CSCsi40727—When changing from dynamic sticky to cookie insert, the ACE creates static entries, but does not clear old dynamic entries.
•
CSCsi49611—Processing SSL handshake messages may cause the ACE module to reload. Workaround: None.
•
CSCsi50387—The ACE reloads while running ssl-init or ssl end to end and transferring 1 KB pages. This causes SSL certificates and keys to be lost from user contexts.
•
CSCsi51985—This issue is seen upon reload with multiple user contexts that do not have interfaces configured. The existing connections are first reaped on the standby ACE before state sync is initiated from the active ACE. Connection reaping happens only when the context has configured interfaces, which causes a delay in completing the bulk sync. The redundant state of the ACEs is incorrectly reported as Active/Active until timeout.
•
CSCsi54336—When using the match source-address command in class map HTTP load balancing configuration mode, traffic from the matching class does not get matched to this class map. Instead, it may match to the default class map or gets dropped if the default class map is not configured. Workaround: None.
•
CSCsi59197—The ACE reboots with an SSL memory dump file while running a basic SSL configuration.
•
CSCsi62132—Sticky replication may not function with redundancy and stickiness configured in the Admin context.
•
CSCsi63998—On rare occasions, invalid addressing can cause the SSL or load-balancing process to perform a memory dump.
•
CSCsi70945—When non-load-balanced traffic traverses the ACE with the same source and destination ports, the ACE performs implicit port address translation (PAT). This behavior causes some sessions to terminate. Workaround: None.
•
CSCsi72278—When the /isan directory reaches 100%, the syslogd process hangs without performing a memory dump. Workaround: None.
•
CSCsi72304—Because of access logs, the /isan directory can reach 100%, which prevents further logging and other issues. Workaround: None.
•
CSCsi72622—With the no normalization command configured on a management interface, connection leak has been observed in the show resource usage command output. Workaround: None.
•
CSCsi72839—In a redundancy configuration where one ACE is running image A1.4, A1.4a, A1.4b or A1.4c and the other ACE is running image A1.4d through A1.4l, sticky database replication may fail. This problem is most likely to occur during an upgrade where one ACE has been upgraded to one of the later images listed and the other is still on an older image.
•
CSCsi79032, CSCsi73358, CSCsi62044—When 100 clients perform GETs/POSTs of large files over 2 MB in size, poor SSL performance is observed and clear text access on other contexts may be adversely affected. Workaround: If running in a redundant configuration, splitting the mastership between the ACEs could avoid this issue.
•
CSCsi87346—Files created using the copy capture command cannot be read from the compact flash. Workaround: Use the show capture command to display the captured packets. Alternatively, load the debug plugin and modify the permissions of the file in the /TN-HOME directory.
•
CSCsi87971—An unexpected reload of the ACE module while running SSL traffic produces an SSLHs core file. Workaround: None.
•
CSCsi88543—While upgrading the ACE, it becomes unresponsive because the IDMAP (the structure used for state replication and configuration synchronization) changed between the two software versions and was accessed incorrectly.
•
CSCsi92787—The SMTP server, whose health is monitored by an ACE SMTP probe, continuously logs the "500 5.5.2 bad chars in command" message. This is because SMTP commands sent by an SMTP probe end with an invalid trailing zero (null) character. Despite this, the probes are successful and the SMTP server stays in service. Workaround: None.
•
CSCsi94922—Configuring, unconfiguring, and reconfiguring more than 100 server farms caused the ACE to reload because of dangling pointers. Workaround: None.
•
CSCsi95597—Some VIPs on the ACE do not reply to ICMP ECHO requests as configured. Other VIPs still work well, and even the affected VIPs may serve content.
•
CSCsi96663—In an SSL environment, during an SSL re-handshake, if SSL data is received before the re-handshake completes, the ACE may become unresponsive. Workaround: None
•
CSCsj00990—VIPs on the ACE may send reset (RST) packets in response to HTTP or HTTPS requests. This situation occurs when multiple VIPs and service policies are configured. Changing one service policy configuration when another VIP is out of service may leave the out of service VIP in this state. Workaround: Changing the configuration of the VIPs that do not work as expected several times may help this situation.
•
CSCsj01761—The TACACS+ server key that you configure in the active context is not replicated correctly in the standby context. Workaround: None.
•
CSCsj01769—In a redundancy configuration, probes may get stuck either in the init state or failed/success state on the standby ACE after a configuration rollback. The problem may occur when a configuration rollback is done on the active ACE. Workaround: None.
•
CSCsj02257—In a redundancy configuration, when the checkpoint rollback blank and checkpoint rollback name commands were entered several times, a core dump file was created on the active ACE. Workaround: None.
•
CSCsj05983—When an inspect policy is applied to a class map that does not have a specific protocol or port, the ACE rejects the configuration with an error message. Workaround: None.
•
CSCsj10620—An ACE reload may occur when using the 3.0(0)A1(4a) sustaining image. Workaround: None.
•
CSCsj16667—When a probe fails (for example, if the server fails to send a SYN-ACK within the required time), the ACE may become unresponsive. If, during the interval between sending two consecutive probes, the probe configuration is changed and, if the probe encounters a failure, this condition occurs. Workaround: None.
•
CSCsj17127—Management connections and health probes may fail in some situations where there is a large number of unreachable real servers. Workaround: None.
Software Version 3.0(0)A1(5a) Command Changes
Table 5 lists the commands and options that have been added in software version 3.0(0)A1(5a).
Table 6 lists the commands and options that have changed in software version 3.0(0)A1(5a).
Software Version 3.0(0)A1(4a) Open Caveats
The following caveats apply to software version 3.0(0)A1(4a):
•
CSCsc62866—Occasionally VSH core files appear in the core directory. This issue is harmless. It does not affect any functionality.
•
CSCsd80505 —Repeated Layer 4 performance tests provide a result of only 260K CPS. This issue occurs when a test exercises a test beyond system limitations that is more than 330K CPS. Workaround: Make sure that test parameters are within 330KCPS.
•
CSCse39448—After an active ACE module that was reloaded comes back up in a redundant configuration (interchassis topology), traffic between two hosts may be dropped for approximately 10 seconds. You may observe this issue with preempt configured, which ensures that the newly reloaded ACE will become the active member of the FT group.
•
CSCse63796—For each real server, the ACE module supports 16 TCP option combinations in the reuse pool. The supported TCP options are:
–
Maximum segment size (MSS)
–
Selective acknowledgement (SACK)
–
Timestamp (TS)
–
Window scale (WS)
If all 16 combinations are exhausted and a new one has to be created, then the least recently used combination is replaced correctly, but the connections and proxy IDs present in that pool are not closed.
•
CSCse64922—If the virtual server state is OUTOFSERVICE, the ACE may incorrectly update the dropped conns counter under the L7 loadbalance policy section of the show service-policy policy_name detail command.
•
CSCse65505—If an incremental synchronization fails because a certificate or probe is not imported on the standby ACE, configuration synchronization between the ACEs is not disabled. This issue occurs when using a certificate or a scripted probe in a redundant configuration. It is expected that the files containing the script or the certificates are imported on both the active and standby ACEs so that configuration synchronization occurs correctly. If the file is missing from the standby ACE, the synchronization cannot occur. Workaround: Use FTP to copy the file to both active and standby ACEs.
•
CSCse70920—The ACE does not indicate whether a connection was selected from the reuse pool in the connection setup syslog (%ACE-6-302022).
•
CSCse72735—The ACE does not send traffic from one context to another (on the same module) over a firewall service module in transparent mode.
•
CSCse79208—The no logging message message_id level level command does not disable the logging of the message at the specified level. Workaround: Disable the logging of the message without the level option, then reenable it for the desired levels.
•
CSCse85906—In a redundant configuration, persistent connections established through the ACE context become unresponsive after a switchover.
•
CSCse89405—The delta for application inspection statistics cannot be obtained using the command show np np me-stats "-M n -s appinspect" command.
•
CSCse90214—In a redundant configuration, the peer context associated with an FT group may be configured with a higher priority and triggers a switchover when it is brought into service. With this sequence of events, connections that were established through the previously active context are lost. Workaround: Bring up the peer FT group with a priority and preempt configuration so it does not trigger an immediate switchover. Once the Active - Standby Hot states are reached and connections are replicated, the switchover may then be triggered by appropriate configuration changes or Exec commands.
•
CSCse90603—Auto complete does not work on checkpoint names. When issuing the checkpoint rollback and checkpoint delete commands, the "?" does not display the existing checkpoint names. Workaround: Use the show checkpoint all command to find the name for the checkpoint rollback and checkpoint delete commands.
•
CSCse95074—The Total Count field value in the output of the show serverfarm name retcode [detail] command is not preserved for real servers that are taken out of service. The ACE resets the counter to zero.
•
CSCse95336—The transfer of a large page (10 MB) between a single client and server may generate several short record SSL errors and occasionally bad MAC record errors at the end of the connection. This condition occurs infrequently and is not easily observed.
•
CSCse96401—When you are configuring the boot string, you may encounter the following error: "write_block failed after attempt, sr=0xb BF: error write". On subsequent reloads, autoboot may fail and the ACE displays the following errors:
–
"out of room in table, cannot set "ARGV0""
–
"Autoboot: failed, boot string not found"
Workaround: At the rommon prompt, enter the following sequence of commands:
–
priv
–
erase
–
sync
–
reset
This command sequence will clear the corruption of the boot flash configuration parameters. If the ACE module does not autoboot, use the boot disk0:image_name command to proceed.
•
CSCsf01854—Under high TPS load and with front-end SSL and six chained certificates configured, the ACE may reboot and go to rommon. "Buf alloc failed. Out of Buffers" messages may be observed on the console. Workaround: Use fewer than six certificates in the certificate chain.
•
CSCsf02492— If a new real server is added to a server farm that is configured with the least connections predictor and slowstart, the ACE may not put the new real server in slowstart mode if these actions occur:
–
Add the real server to the server farm
–
Change an existing real server weight to the default value
Workaround: Change the weight of the existing real server before adding the new real server to the server farm.
•
CSCsf03118—In a redundant configuration with persistence-rebalance configured, if the second GET request in the same connection is load balanced to another real server in the server farm, the active ACE context may not replicate the connection to the standby context correctly. If a switchover occurs at this time, subsequent GET requests are forwarded to the original real server, which responds with a reset (RST).
•
CSCsf03222—In a redundant configuration where the active Admin context autosync state is being toggled, the running config in a user context (standby) may be lost as a result of a configuration synchronization issued by the active ACE, if the standby context has autosync disabled. Workaround: Enable autosync of the running-configuration in the user context before enabling it for the Admin context.
•
CSCsf06159—In a redundant configuration with configuration synchronization enabled, the ACE may reboot and go to rommon on the configuration of a very large ACL (100 K entries). This may be seen even if there is no traffic through the Active module at the time of configuration.
•
CSCsf06622—The ACE may reboot unexpectedly and go to rommon as the ACE prepares to service an issued reload command and the "last boot reason" output of the show version command is set to process "reboot." This issue is harmless. Workaround: None.
•
CSCsf06628—Under certain conditions, the hash value in the ACL deny syslog may not match internal hash values displayed in the output of the show access-list acl_name command. If you specify partial (but valid) keywords while configuring the entries in the ACL, the ACE computes the hash value on the partial configuration string. However, after a reboot, the ACE computes the hash value on the complete command string because that is what is saved in the startup configuration. This will result in inconsistency between the hash values shown in the syslog before reload and hash values shown in the show access-list acl_name CLI after reload. Workaround: Tab complete (or type entire keywords) in the CLI when configuring individual entries in the ACL.
•
CSCsf07297—If you try to modify a NAT configuration by cutting-and-pasting the configuration into the ACE console, the existing NAT pool entries are not correctly deleted. Workaround: Manually execute the following configuration sequence instead of the cut-and-paste method:
–
Delete the current NAT dynamic statement
–
Delete the NAT pool
–
Add the new NAT pool
–
Add the new NAT dynamic statement
•
CSCsf10038—In a redundant setup, connections will not be replicated during the upgrade from the following images:
–
Image 3.0(0)A1(2) to 3.0(0)A1(3), 3.0(0)A1(3b), or 3.0(0)A1(4)
–
Image 3.0(0)A1(3) or 3.0(0)A1(3b) to 3.0(0)A1(4)
The recommended upgrade procedure is as follows:
1.
Save the configuration on each ACE module by using the write mem all command
2.
Copy the 3.0(0)A1(3), A1(3b)or 3.0(0)A1(4) image to both ACE modules using the copy ftp command
3.
Configure the boot parameters to boot from the image. Use the show boot command to verify the boot configuration.
4.
The ACE module that has the Admin context in the StandbyHot state should be upgraded first. Use the show ft group detail command to verify the state of the module. Reload this module. After the module has booted up, it may take four to five minutes to reach the StandbyHot state again. Note that the connections through the active ACE module will not be replicated to the standby module.
5.
Upgrade the other ACE module by reloading it.
6.
At this point, a switchover takes place. The standby module becomes the new active module.
7.
If preempt is set, the module is restored to the original active or standby state. Otherwise, the newly rebooted module comes up and becomes the standby module.
8.
Use the show ft group detail command to verify that the Active-StandbyHot state has been reached on the peer modules.
•
CSCsf15331—The ACE may reboot if more than 400 HTTPS probes are configured. Workaround: Configure less than 400 HTTPS probes. We also recommend that you limit the number of "Concurrent-openssl" connections to 50 or less.
•
CSCsf30686—The TPS maximum is 25, 000 when the persistence-rebalance feature is used, in which each HTTP request within the connection is load balanced. Workaround: None.
•
CSCsf12347— A persistent connection may reset when a new HTTP request arrives after any change in the server farm configuration is completed.
•
CSCsg06003 —The show service-policy command displays multiple current connections under a NAT class map. However, the show xlate and show conn commands display no connections. This issue does not affect functionality. Workaround: None.
•
CSCsg06046—With persistence-rebalance configured, if a client opens up a single HTTP 1.1 persistent connection and sends multiple GETs with all requesting the same data, each GET causes the ACE to tear down the server flow, load balance to a new server, and establish a new server flow for each one. Workaround: None.
•
CSCsg10024—A Layer 7 FTP policy blocks the cdup command, but allows the cd command. This issue enables you to change to the root directory, negating any effect of blocking the cdup command. Workaround: None.
•
CSCsg10072—The inspect ftp strict command should mask the user 530 reply code with a 331 reply code and the SYST command with XXXXXXXXXX, but it only works for the user command.
•
CSCsg23016—The class map insert-before option does not work on a previously-configured class map. Workaround: Delete the class map and configure it again.
•
CSCsg23096—The show conn serverfarm command does not display FTP data connections. Workaround: None.
•
CSCsg40029—After a switchover, ARP RESPONSE collision messages may appear on the standby ACE for two minutes. This issue is harmless and a rare event. Workaround: None.
•
CSCsg47106 —The ACE may not stick on an HTTP cookie inserted with the cookie insert command and using the replicate sticky command in a fault tolerant configuration. This failure also causes return traffic to contain a Set Cookie: which updates the browser to use the newly chosen server. Workaround: Configure a static cookie on each sticky group that uses the cookie insert command after the fault tolerant status has stabilized. This cookie does not require a particular value. This workaround does not persist across reboots.
•
CSCsg49360—When using the no context command, you can delete a context without a warning. This is a destructive command. Workaround: None.
•
CSCsg49375—Issuing the no context command deletes saved checkpoints and files from disk0. There is no way to recover them. Workaround: None.
•
CSCsg55692—When you enable server connection reuse with persistence-rebalance, some connections may see a reset or timeout. Workaround: None.
•
CSCsg57448—A stress condition (100,000 total connection) to an SSL initiation VIP address generates RSTs for some of the connections to the front-end client. Workaround: None.
•
CSCsg74733 —In a routed network design, the NAM cannot see all packets on VLANs allocated to the ACE although traffic is flowing correctly. Workaround: Use port mirroring and an external packet capturing tool.
•
CSCsg74903—Console logging stops if a blank checkpoint is used on other context. Workaround: Do not use a blank checkpoint.
•
CSCsg77694—Server load balancing fails when you configure a VIP address with a mask. Workaround: Configure static ARP or routes as needed on all upstream routers for all the IP addresses in the specified range.
•
CSCsg80625—The ACE may drop UDP packets addressed to a VIP address if the packet size is greater than 1,500 bytes. Workaround. Use packet size less than 1,500 bytes.
•
CSCsg80881—After you remove UDP probes from the configuration, the ACE continues to send them. Workaround: None.
•
CSCsg88014—Copying an FTP image fails when overwriting the same filename. Workaround: Delete the file and recopy it.
•
CSCsg89266 —When you add new class maps to a policy map after the default class map configuration, the new class maps are not activated. Workaround: Remove the default class map and re-add it after you configure the new class maps.
•
CSCsg91127—A license mismatch may occur when a new resource class and a new context is added in an HA setup. Workaround: Delete the context and re-add it.
•
CSCsg95237—High CPU utilization by the ACL-merge process occasionally occurs while performing the copy running-config startup-config command or when removing a real server from the service. Workaround: None. The CPU utilization should go down after a period of time.
•
CSCsg95379—A trace route does not work when initiated from a server behind the ACE when source NAT is configured for the server. The output displays only the destination IP address. Workaround: None.
•
CSCsg97750—Capture files that are generated when the capture buffer is copied to disk may not be sent by TFTP to a server when the capture file name starts with "ace." Workaround:
–
Do not to use the word "ace" in the capture filename.
–
Instead of the copy capture buffer_name command, use the show capture buffer_name filename command.
•
CSCsg98273—When the ACE receives a legitimate ICMP error packet, it recalculates the ICMP checksum but not the embedded IP header checksum before forwarding it to the next hop. If the NAT fixup is operated on the original IP header, some clients may flag this checksum error when it receives the ICMP error packet.
•
CSCsh02651—Transfer-encoding type chunked regex does not work for the HTTP inspection feature.
•
CSCsh12487—The ACE may reboot if the VLAN is configured with the no normalization command and a TCP reset packet with a zero TCP Header Length enters ACE. Workaround: Do not disable normalization.
•
CSCsh17204—The ACE inserts an empty cookie value if the connection is loadbalanced to a backup server farm. Workaround: Do not configure a backup server farm with cookie-insert sticky.
•
CSCsh19260—The ACE may reset if EXPORT ciphers are used. Workaround: None.
•
CSCsh19382—SSL bulk throughput is only 3.4 Gbps. Workaround: None.
•
CSCsh19923—The end host (client or server) discards ICMP error packets received for TCP connections. This issue is caused by the ACE corrupting the ICMP checksum of ICMP error packets that are generated by intermediate routers and end hosts for TCP connections. End hosts that rely on these ICMP error packets for their functionality, for example PMTU Discovery, are affected. Workaround: Applies only to the Type 3 Code 4 (Fragmentation needed but DF bit set) ICMP error. On the interface that receives this error packet, configure the ip df clear command in interface configuration mode.
•
CSCsh20546—If the maximum connections (maxconn), persistence-rebalance, and sticky features are all enabled on the ACE, the maxconn feature does not work. Workaround: Do not use all three of these features on the same VIP/server farm.
•
CSCsh22888—If an SSL certificate is missing or incorrect, the standby ACE may reset on boot. Workaround: Make sure that the standby ACE has the correct certificate.
•
CSCsh22894—SGC certificate does not work. Traffic using the SGC certificate is dropped. Workaround: None.
•
CSCsh23922—When you take an rserver out of service, active long lived HTTP connections may also be torn down for other rservers in the same server farm.
•
CSCsh88518—The ACE becomes unresponsive when executing the ft switchover force command. Workaround: None.
Software Version 3.0(0)A1(4) Open Caveats, Resolved Caveats, and Command Changes
The following sections contain the open and resolved caveats in software version 3.0(0)A1(4):
•
Software Version 3.0(0)A1(4) Open Caveats
•
Software Version 3.0(0)A1(4) Resolved Caveats
•
Software Version 3.0(0)A1(4) Command Changes
Software Version 3.0(0)A1(4) Open Caveats
The following caveats apply to software version 3.0(0)A1(4):
•
CSCsc62866—Occasionally VSH core files appear in the core directory. This issue is harmless. It does not affect any functionality.
•
CSCsd80505 —Repeated Layer 4 performance tests provide a result of only 260K CPS. This issue occurs when a test exercises a test beyond system limitations that is more than 330K CPS. Workaround: Make sure that test parameters are within 330KCPS.
•
CSCse39448—After an active ACE module that was reloaded comes back up in a redundant configuration (interchassis topology), traffic between two hosts may be dropped for approximately 10 seconds. You may observe this issue with preempt configured, which ensures that the newly reloaded ACE will become the active member of the FT group.
•
CSCse63796—For each real server, the ACE module supports 16 TCP option combinations in the reuse pool. The supported TCP options are:
–
Maximum segment size (MSS)
–
Selective acknowledgement (SACK)
–
Timestamp (TS)
–
Window scale (WS)
If all 16 combinations are exhausted and a new one has to be created, then the least recently used combination is replaced correctly, but the connections and proxy IDs present in that pool are not closed.
•
CSCse64922—If the virtual server state is OUTOFSERVICE, the ACE may incorrectly update the dropped conns counter under the L7 loadbalance policy section of the show service-policy policy_name detail command.
•
CSCse65505—If an incremental synchronization fails because a certificate or probe is not imported on the standby ACE, configuration synchronization between the ACEs is not disabled. This issue occurs when using a certificate or a scripted probe in a redundant configuration. It is expected that the files containing the script or the certificates are imported on both the active and standby ACEs so that configuration synchronization occurs correctly. If the file is missing from the standby ACE, the synchronization cannot occur. Workaround: Use FTP to copy the file to both active and standby ACEs.
•
CSCse70920—The ACE does not indicate whether a connection was selected from the reuse pool in the connection setup syslog (%ACE-6-302022).
•
CSCse72735—The ACE does not send traffic from one context to another (on the same module) over a firewall service module in transparent mode.
•
CSCse79208—The no logging message message_id level level command does not disable the logging of the message at the specified level. Workaround: Disable the logging of the message without the level option, then reenable it for the desired levels.
•
CSCse85906—In a redundant configuration, persistent connections established through the ACE context become unresponsive after a switchover.
•
CSCse89405—The delta for application inspection statistics cannot be obtained using the command show np np me-stats "-M n -s appinspect" command.
•
CSCse90214—In a redundant configuration, the peer context associated with an FT group may be configured with a higher priority and triggers a switchover when it is brought into service. With this sequence of events, connections that were established through the previously active context are lost. Workaround: Bring up the peer FT group with a priority and preempt configuration so it does not trigger an immediate switchover. Once the Active - Standby Hot states are reached and connections are replicated, the switchover may then be triggered by appropriate configuration changes or Exec commands.
•
CSCse90603—Auto complete does not work on checkpoint names. When issuing the checkpoint rollback and checkpoint delete commands, the "?" does not display the existing checkpoint names. Workaround: Use the show checkpoint all command to find the name for the checkpoint rollback and checkpoint delete commands.
•
CSCse95074—The Total Count field value in the output of the show serverfarm name retcode [detail] command is not preserved for real servers that are taken out of service. The ACE resets the counter to zero.
•
CSCse95336—The transfer of a large page (10 MB) between a single client and server may generate several short record SSL errors and occasionally bad MAC record errors at the end of the connection. This condition occurs infrequently and is not easily observed.
•
CSCse96401—When you are configuring the boot string, you may encounter the following error: "write_block failed after attempt, sr=0xb BF: error write". On subsequent reloads, autoboot may fail and the ACE displays the following errors:
–
"out of room in table, cannot set "ARGV0""
–
"Autoboot: failed, boot string not found" errors.
Workaround: At the rommon prompt, enter the following sequence of commands:
–
priv
–
erase
–
sync
–
reset
This command sequence will clear the corruption of the boot flash configuration parameters. If the ACE module does not autoboot, use the boot disk0:image_name command to proceed.
•
CSCsf01854—Under high TPS load and with front-end SSL and six chained certificates configured, the ACE may reboot and go to rommon. "Buf alloc failed. Out of Buffers" messages may be observed on the console. Workaround: Use fewer than six certificates in the certificate chain.
•
CSCsf02492— If a new real server is added to a server farm that is configured with the least connections predictor and slowstart, the ACE may not put the new real server in slowstart mode if these actions occur:
–
Add the real server to the server farm
–
Change an existing real server weight to the default value
Workaround: Change the weight of the existing real server before adding the new real server to the server farm.
•
CSCsf03118—In a redundant configuration with persistence-rebalance configured, if the second GET request in the same connection is load balanced to another real server in the server farm, the active ACE context may not replicate the connection to the standby context correctly. If a switchover occurs at this time, subsequent GET requests are forwarded to the original real server, which responds with a reset (RST).
•
CSCsf03222—In a redundant configuration where the active Admin context autosync state is being toggled, the running config in a user context (standby) may be lost as a result of a configuration synchronization issued by the active ACE, if the standby context has autosync disabled. Workaround: Enable autosync of the running-configuration in the user context before enabling it for the Admin context.
•
CSCsf06159—In a redundant configuration with configuration synchronization enabled, the ACE may reboot and go to rommon on the configuration of a very large ACL (100 K entries). This may be seen even if there is no traffic through the Active module at the time of configuration.
•
CSCsf06622—The ACE may reboot unexpectedly and go to rommon as the ACE prepares to service an issued reload command and the "last boot reason" output of show version command is set to process "reboot." This issue is harmless. Workaround: None.
•
CSCsf06628—Under certain conditions, the hash value in the ACL deny syslog may not match internal hash values displayed in the output of the show access-list acl_name command. If you specify partial (but valid) keywords while configuring the entries in the ACL, the ACE computes the hash value on the partial configuration string. However, after a reboot, the ACE computes the hash value on the complete command string because that is what is saved in the startup configuration. This will result in inconsistency between the hash values shown in the syslog before reload and hash values shown in the show access-list acl_name CLI after reload. Workaround: Tab complete (or type entire keywords) in the CLI when configuring individual entries in the ACL.
•
CSCsf07297—If you try to modify a NAT configuration by cutting-and-pasting the configuration into the ACE console, the existing NAT pool entries are not correctly deleted. Workaround: Manually execute the following configuration sequence instead of the cut-and-paste method:
–
Delete the current NAT dynamic statement
–
Delete the NAT pool
–
Add the new NAT pool
–
Add the new NAT dynamic statement
•
CSCsf10038—In a redundant setup, connections will not be replicated during the upgrade from the following images:
–
Image 3.0(0)A1(2) to 3.0(0)A1(3), 3.0(0)A1(3b), or 3.0(0)A1(4)
–
Image 3.0(0)A1(3) or 3.0(0)A1(3b) to 3.0(0)A1(4)
The recommended upgrade procedure is as follows:
1.
Save the configuration on each ACE module by using the write mem all command
2.
Copy the 3.0(0)A1(3), A1(3b)or 3.0(0)A1(4) image to both ACE modules using the copy ftp command
3.
Configure the boot parameters to boot from the image. Use the show boot command to verify the boot configuration.
4.
The ACE module that has the Admin context in the StandbyHot state should be upgraded first. Use the show ft group detail command to verify the state of the module. Reload this module. After the module has booted up, it may take four to five minutes to reach the StandbyHot state again. Note that the connections through the active ACE module will not be replicated to the standby module.
5.
Upgrade the other ACE module by reloading it.
6.
At this point, a switchover takes place. The standby module becomes the new active module.
7.
If preempt is set, the module is restored to the original active or standby state. Otherwise, the newly rebooted module comes up and becomes the standby module.
8.
Use the show ft group detail command to verify that the Active-StandbyHot state has been reached on the peer modules.
•
CSCsf15331—The ACE may reboot if more than 400 HTTPS probes are configured. Workaround: Configure less than 400 HTTPS probes. We also recommend that you limit the number of "Concurrent-openssl" connections to 50 or less.
•
CSCsf30686—The TPS maximum is 25, 000 when the persistence-rebalance feature is used, in which each HTTP request within the connection is load balanced. Workaround: None.
•
CSCsg06003 —The show service-policy command displays multiple current connections under a NAT class map. However, the show xlate and show conn commands display no connections. This issue does not affect functionality. Workaround: None.
•
CSCsg06046—With persistence-rebalance configured, if a client opens up a single HTTP 1.1 persistent connection and sends multiple GETs with all requesting the same data, each GET causes the ACE to tear down the server flow, load balance to a new server, and establish a new server flow for each one. Workaround: None.
•
CSCsg10024—A Layer 7 FTP policy blocks the cdup command, but allows the cd command. This issue enables you to change to the root directory, negating any effect of blocking the cdup command. Workaround: None.
•
CSCsg10072—The inspect ftp strict command should mask the user 530 reply code with a 331 reply code and the SYST command with XXXXXXXXXX, but it only works for the user command.
•
CSCsg23016—The class map insert-before option does not work on a previously-configured class map. Workaround: Delete the class map and configure it again.
•
CSCsg23096—The show conn serverfarm command does not display FTP data connections. Workaround: None.
•
CSCsg40029—After a switchover, ARP RESPONSE collision messages may appear on the standby ACE for two minutes. This issue is harmless and a rare event. Workaround: None.
•
CSCsg47106 —The ACE may not stick on an HTTP cookie inserted with the cookie insert command and using the replicate sticky command in a fault tolerant configuration. This failure also causes return traffic to contain a Set Cookie: which updates the browser to use the newly chosen server. Workaround: Configure a static cookie on each sticky group that uses the cookie insert command after the fault tolerant status has stabilized. This cookie does not require a particular value. This workaround does not persist across reboots.
•
CSCsg49360—When using the no context command, you can delete a context without a warning. This is a destructive command. Workaround: None.
•
CSCsg49375—Issuing the no context command deletes saved checkpoints and files from disk0. There is no way to recover them. Workaround: None.
•
CSCsg55692—When you enable server connection reuse with persistence-rebalance, some connections may see a reset or timeout. Workaround: None.
•
CSCsg57448—A stress condition (100,000 total connection) to an SSL initiation VIP address generates RSTs for some of the connections to the front-end client. Workaround: None.
•
CSCsg74733 —In a routed network design, the NAM cannot see all packets on VLANs allocated to the ACE although traffic is flowing correctly. Workaround: Use port mirroring and an external packet capturing tool.
•
CSCsg74903—Console logging stops if a blank checkpoint is used on other context. Workaround: Do not use a blank checkpoint.
•
CSCsg77694—Server load balancing fails when you configure a VIP address with a mask. Workaround: Configure static ARP or routes as needed on all upstream routers for all the IP addresses in the specified range.
•
CSCsg80625—The ACE may drop UDP packets addressed to a VIP address if the packet size is greater than 1,500 bytes. Workaround. Use packet size less than 1,500 bytes.
•
CSCsg80881—After you remove UDP probes from the configuration, the ACE continues to send them. Workaround: None.
•
CSCsg88014—Copying an FTP image fails when overwriting the same filename. Workaround: Delete the file and recopy it.
•
CSCsg89266 —When you add new class maps to a policy map after the default class map configuration, the new class maps are not activated. Workaround: Remove the default class map and re-add it after you configure the new class maps.
•
CSCsg91127—A license mismatch may occur when a new resource class and a new context is added in an HA setup. Workaround: Delete the context and re-add it.
•
CSCsg95237—High CPU utilization by the ACL-merged process occasionally occurs while performing the copy running-config startup-config command or when removing a real server from the service. Workaround: None. The CPU utilization should go down after a period of time.
•
CSCsg95379—A trace route does not work when initiated from a server behind the ACE when source NAT is configured for the server. The output displays only the destination IP address. Workaround: None.
•
CSCsg97750—Capture files that are generated when the capture buffer is copied to disk may not be sent by TFTP to a server when the capture file name starts with "ace." Workaround:
–
Do not to use the word "ace" in the capture filename.
–
Instead of the copy capture buffer_name command, use the show capture buffer_name filename command.
•
CSCsg98273—When the ACE receives a legitimate ICMP error packet, it recalculates the ICMP checksum but not the embedded IP header checksum before forwarding it to the next hop. If the NAT fixup is operated on the original IP header, some clients may flag this checksum error when it receives the ICMP error packet.
•
CSCsh02651—Transfer-encoding type chunked regex does not work for the HTTP inspection feature.
•
CSCsh12487—The ACE may reboot if the VLAN is configured with the no normalization command and a TCP reset packet with a zero TCP Header Length enters ACE. Workaround: Do not disable normalization.
•
CSCsh17204—The ACE inserts an empty cookie value if the connection is loadbalanced to a backup server farm. Workaround: Do not configure a backup server farm with cookie-insert sticky.
•
CSCsh19260—The ACE may reset if EXPORT ciphers are used. Workaround: None.
•
CSCsh19382—SSL bulk throughput is only 3.4 Gbps. Workaround: None.
•
CSCsh19923—The end host (client or server) discards ICMP error packets received for TCP connections. This issue is caused by the ACE corrupting the ICMP checksum of ICMP error packets that are generated by intermediate routers and end hosts for TCP connections. End hosts that rely on these ICMP error packets for their functionality, for example PMTU Discovery, are affected. Workaround: Applies only to the Type 3 Code 4 (Fragmentation needed but DF bit set) ICMP error. On the interface that receives this error packet, configure the ip df clear command in interface configuration mode.
•
CSCsh20546—If the maxconn, persistence-rebalance and sticky features are all enabled on the ACE, the maxconn feature does not work. Workaround: Do not use all three of these features on the same VIP/server farm.
•
CSCsh22888—If an SSL certificate is missing or incorrect, the standby ACE may reset on boot. Workaround: Make sure that the standby ACE has the correct certificate.
•
CSCsh22894—SGC certificate does not work. Traffic using the SGC certificate is dropped. Workaround: None.
•
CSCsh23922—When you take an rserver out of service, active long lived HTTP connections may also be torn down for other rservers in the same server farm.
•
CSCsh71875—In a redundant setup, a reset occurs during an upgrade from software versions 3.0(0)A1(2), 3.0(0)A1(3), or 3.0(0)A1(3b) to 3.0(0)A1(4).
•
CSCsh88518—The ACE becomes unresponsive when executing the ft switchover force command. Workaround: None.
Software Version 3.0(0)A1(4) Resolved Caveats
The following caveats apply to software version 3.0(0)A1(4):
•
CSCse53496—When using exportable ciphers (for example, EXP-RC4-MD5) with non-exportable key sizes (for example, 1536 or 2048), low concurrency traffic succeeds using ephemeral RSA step up. However, under stress loads, the same configuration causes connection failures with block cipher and bad MAC errors.
•
CSCse64261—Consecutive incoming connections with different MSS values are added to the same TCP option combination pool.
•
CSCse90987—The ACE may reboot and go to rommon when running front-end SSL with a large page size (1MB) in a high-throughput environment.
•
CSCse91186—Established connections to a real server are torn down if failaction purge has been configured and the real server reaches the MAXCONN state. The ACE treats the MAXCONNS state as an OUTOFSERVICE indication.
•
CSCse91284—The ACE may erroneously set the real server state to MAXCONNS and reject new connections, even if the total number of connections is less than the expected maximum connections value. This may happen if the number of connections on one of the network processors hits its individual maximum value.
•
CSCse96574—In a redundant configuration, with multiple bridge groups configured, TCP sessions through the active ACE context may become unresponsive upon switchover.
•
CSCse97571—In a redundant configuration with a one-arm topology, upon replication of connections, the standby ACE context starts sending packets with a source MAC address set to the alias virtual MAC address. This scenario causes the Catalyst 6500 series switch to direct packets in existing connections or new connections to the standby ACE context, which drops the packets.
•
CSCse98384—In a redundant configuration, the ACE may reboot and go to rommon with cookie sticky configured and 30 Kcps traffic. Just before the reboot, buffer allocation errors may be seen on the IXP console.
•
CSCse98813—Client SSL connections which attempt a handshake renegotiation with ACE SSL VIPs will fail. Workaround: If possible, disable client handshake renegotiation (either data based or time based). With Microsoft Internet Explorer, this may not be possible without altering the Windows registry.
•
CSCse98975—When using the ACE for transparent cache load balancing and load-balancing flows to a real server on the same VIP or a superset of the VIP, the ACE may send server response traffic directly to the client, rather than to the cache device. The caches are bypassed on the response traffic and the connections fail.
•
CSCse99442—In the output of the show resource usage command, the current, minimum allocated, and maximum allocated values for syslog rate and syslog buffer are displayed incorrectly.
•
CSCse99513—If an OUTOFSERVICE real server is brought to the OPERATIONAL state in a server farm configured with the least connections predictor and slowstart, the ACE sometimes does not place the real server in slowstart mode. You can observe this condition every alternate time that the real server is placed from the OUTOFSERVICE to the INSERVICE stats.
•
CSCsf02370—The syslog message generated after a sequence of deletions and additions of ACL entries may incorrectly have the ACL name set to Unknown and the ACE hash value set to 0. Workaround: Remove and add the service policy (or the security ACL) back to the interface.
•
CSCsf07333—In a redundant configuration (interchassis topology), mac-move messages that are associated with the ACE FT VLAN may be seen on the Catalyst 6500 series switch if the supervisor engine has the mac-address-table notification mac-move command configured and redundancy is enabled in the Admin context of the ACE. These messages are caused by the virtual MAC (VMAC) address, which is used as the source MAC by heartbeat messages exchanged between peer ACE modules. After every heartbeat, each switch claims the VMAC address to point via the ACE trunk interface. This issue does not have any impact on the rest of the network.
•
CSCsf08382—With 4 K or larger HTTP POSTs and front-end SSL termination configured, the ACE may appear to be unresponsive for approximately 12 seconds. Clients must wait a longer time while the ACE completes the TCP portion of the transaction (FIN, and so on).
•
CSCsf09910—In a redundant configuration with FTP inspection configured, FTP sessions established through the active ACE context could become unresponsive after one or more switchovers.
•
CSCsf11135—In a Layer 2 topology, if the default route of a real server points to the BVI alias IP address on the ACE, return traffic from the real server to the client network is dropped. Workaround: Change the default route of the real server to point to the switch or router between the client network and ACE.
•
CSCsf12347—A persistent connection may reset when a new HTTP request arrives after any change in the server farm configuration is completed.
•
CSCsg69147—Under certain circumstances, the MSFC in a Catalyst 6500 series switch using a SUP720 and Cisco IOS Release 12.2(18)SXF4 may reload an ACE10 module installed in the switch. The MSFC reports the following error message: "Module not responding to Keep Alive polling".
•
CSCsg98878—The ACE may reboot when a corrupt or garbage certificate is exported to the terminal. The corrupted or garbage certificate can occur if contexts are deleted and added in an HA setup.
Software Version 3.0(0)A1(4) Command Changes
Table 7 lists the commands and options that have been added in software version 3.0(0)A1(4).
Table 8 lists the commands and options that have been added in software version 3.0(0)A1(4).
Software Version 3.0(0)A1(3b) Resolved PSIRT
This DDTS is included in Cisco Security Response "Multiple vulnerabilities in OpenSSL library" published at http://www.cisco.com/warp/public/707/cisco-sr-20061108-openssl.shtml
This is the Cisco PSIRT response to the multiple security advisories published by The OpenSSL Project. The vulnerabilities are as follows:
* RSA Signature Forgery (CVE-2006-4339), described in
http://www.openssl.org/news/secadv_20060905.txt
* ASN.1 Denial of Service Attacks (CVE-2006-2937, CVE-2006-2940),
described in http://www.openssl.org/news/secadv_20060928.txt
* SSL_get_shared_ciphers() buffer overflow (CVE-2006-3738), also in
http://www.openssl.org/news/secadv_20060928.txt
* SSLv2 Client Crash (CVE-2006-4343) also in
http://www.openssl.org/news/secadv_20060928.txt
As of this publication there are no workarounds available for any of these vulnerabilities but it may be possible to mitigate some of the exposure.
This Security Response lists the status of each product or application when considered individually. However, in cases where multiple applications are running on the same computer, a vulnerability in one application or component can compromise the entire system. This compromise can then be leveraged against applications that would otherwise be unaffected. Therefore, users must consider all applications when determining their exposure to these vulnerabilities. Cisco strongly recommends that customers update all vulnerable applications and components to provide the greatest protection from the listed vulnerabilities.
Cisco will update this document in the event of any changes.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.
CCDE, CCENT, Cisco Eos, Cisco Lumin, Cisco Nexus, Cisco StadiumVision, the Cisco logo, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn is a service mark; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, IronPort, the IronPort logo, LightStream, Linksys, MediaTone, MeetingPlace, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0805R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
© 2008 Cisco Systems, Inc. All rights reserved.


