Table Of Contents
System Messages
This chapter lists the Cisco Cisco Application Control Engine (ACE) module system log messages. The messages are listed numerically by message code.
To view a list of the majority of variables used in ACE system log messages, refer toTable 1-2 in Chapter 1, Configuring System Message Logging. To view ACE system log messages listed by severity level, refer to Chapter 3, Messages Listed by Severity Level.
This chapter includes the following sections:
Messages 106021 to 111008
This section contains messages from 106021 to 111008.
106021
Error Message %ACE-1-106021: Deny protocol reverse path check from source_address to dest_address on interface interface_nameExplanation An attack is in progress. Someone is attempting to spoof an IP address on an inbound connection. Unicast RPF, also known as reverse route lookup, detected a packet that does not have a source address represented by a route and assumes that it is part of an attack on the ACE.
This message appears when you have enabled Unicast RPF with the ip verify reverse-path command (refer to the see the Cisco Application Control Engine Module Security Configuration Guide.. This feature works on packets input to an interface; if it is configured on the outside, then the ACE checks packets arriving from the outside.
The ACE looks up a route based on the source address. If an entry is not found and a route is not defined, then this system log message appears and the connection is discarded.
If there is a route, the ACE checks which interface to which it corresponds. If the packet arrived on another interface, it is either a spoof or there is an asymmetric routing environment that has more than one path to a destination. The ACE does not support asymmetric routing.
If the ACE is configured on an internal interface, it checks static route command statements or RIP, and if the source address is not found, then an internal user is spoofing their address.
Recommended Action Even though an attack is in progress, if this feature is enabled, no user action is required. The ACE repels the attack.
106023
Error Message %ACE-4-106023: Deny protocol number | name src incoming-interface:src-ip dst outgoing-interface:dst-ip by access-group "acl-name"Explanation An IP packet was denied by the ACL. This message displays even if you do not have the log option enabled for an ACL. If a packet hits an input ACL, the outgoing interface will not be known. In this case, the ACE prints the outgoing interface as undetermined. The source IP and destination IP addresses are the unmapped and mapped addresses for the input and ouput ACLs, respectively, when used with NAT.
Recommended Action If messages persist from the same source address, messages might indicate a foot-printing or port-scanning attempt. Contact the remote host administrators.
106028
Error Message %ACE-1-106028: WARNING: Access rule memory exhausted while processing Access-group. Incomplete rules are currently applied on interface interface-name. Manual roll back to a previous access rule configuration on this interface is needed.Explanation The access control list (ACL) compilation process has run out of memory, causing an inability to apply new ACL entries to the specified interface. The ACL configuration downloaded in hardware for that interface may not be in a known state due to this failure.
Recommended Action Remove some ACL entries to free up memory for the desired configuration. The affected interface needs to be removed and recreated to recover to known state
111008
Error Message %ACE-5-111008: User user executed the command stringExplanation This message is informational. The user entered a command that modified the configuration.
Recommended Action None required.
111009
Error Message %ACE-7-111009: User user executed cmd:stringExplanation This message is informational. The user entered a command that does not modify the configuration.
Recommended Action None required.
199006
Error Message %ACE-5-199006 : Orderly reload started at when by whom. Reload reason: reasonExplanation This message logs a reload record of the ACE and the reason for the reload.
The reason variable describes why the reload occurred. Possible reasons are:
•
reload command
•
•
•
The when variable specifies the time at which orderly reload operation begins.
The whom variable specifies the name of the user who issued the reload command. If the reload is caused by other reasons, "System" is specified.
Recommended Action None required.
Messages 212007 to 251013
This section contains messages from 212007 to 251013.
212007
Error Message %ACE-2-212007: SNMPD initialization failed while Variable1Explanation This is an SNMP message that is logged when the SNMP daemon fails to initialize. The SNMP daemon is created during device initialization.
The Variable1 string can be:
•
•
•
•
•
•
•
Recommended Action Reboot the ACE (see the Cisco Application Control Engine Module Administration Guide for details). If the SNMP daemon still fails to initialize, contact Cisco TAC and provide them with the output of show processes and show np commands.
212008
Error Message %ACE-3-212008: Failed while allocating memory in snmpdExplanation This is an SNMP message that is logged after a memory allocation failure in the SNMPD process. When this error occurs, SNMPD processes (for example, SNMP Get/GetNext responses, trap generation, or SNMP CLI) may be affected.
Recommended Action Check for the system memory using show system command. If the ACE is low on memory, reboot it (see the Cisco Application Control Engine Module Administration Guide for details). Otherwise contact the Cisco TAC and provide them with the output of the show system resources and show processes cpu memory commands.
251001
Error Message %ACE-3-251001: Probe configuration error, memory allocation failure.Explanation The ACE does not have enough memory to support the specified probe configuration. When the Config Manager sends a probe configuration to the Health Monitor module, the Health Monitor module needs to reserve memory to setup the probe. If there is not memory at the time Health Monitor is setting up the probe, the syslog message is sent. This syslog message exists on the ACE so the explanation and recommendation are intentionally kept the same.
Recommended Action Reduce the size of the probe configuration.
251002
Error Message %ACE-4-251002: The configured health probe script script-name for server A.B.C.D on port P is emptyExplanation This message is logged when an empty script is configured for scripted health probe for server A.B.C.D on port P.
Recommended Action Update the script file with appropriate probe information, unload, then reload the script (see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide for details). Alternatively, reconfigure the health probe to use a non-empty script.
251003
Error Message %ACE-3-251003: Could not load script script-name - File not foundExplanation This message is logged when the ACE is unable to find the script file it is required to load.
Recommended Action Create a new script file, unload the old file, then load the new file (see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide for details).
251004
Error Message %ACE-3-251004: Could not load script script-name - memory allocation failureExplanation This message is logged when the ACE does not have sufficient memory to load the specified script file.
Recommended Action Reduce the size of the configuration or unload any unused script files.
251005
Error Message %ACE-4-251005: Could not unload script script-nameExplanation This message is logged when the ACE is unable to load the specified script file due to an internal error.
Recommended Action Contact Cisco TAC if this error frequently occurs.
251006
Error Message %ACE-3-251006: Health probe failed for server A.B.C.D on port P, internal error: error messageExplanation The configured service on port P of server A.B.C.D. failed its health checks because the ACE encountered an internal error while performing the probe. Because the error is internal to the system, the real health of the server is unknown.
The error message string can be:
•
•
•
•
•
•
•
•
•
•
•
•
Recommended Action Contact Cisco TAC if this error frequently occurs.
251007
Error Message %ACE-3-251007: ICMP health probe failed for server A.B.C.D, internal error: error messageExplanation The configured service on port P of server A.B.C.D. failed its health checks because the ACE encountered an internal error while performing the ICMP probe. Because the error is internal to the system, the real health of the server is unknown.
The error message string can be:
•
•
•
•
•
Recommended Action Contact Cisco TAC if this error frequently occurs.
251008
Error Message %ACE-3-251008: Health probe failed for server A.B.C.D on port P, connectivity error: server open timeout (no SYN ACK)Explanation The configured service on port P of server A.B.C.D. failed its health checks because a probe was unable to reach the server due to network problem.
Recommended Action Verify network connectivity to the server, then reprobe the server.
251009
Error Message %ACE-3-251009: ICMP health probe failed for server A.B.C.D, connectivity error: error messageExplanation The configured real server A.B.C.D. failed its health checks because an ICMP health probe was unable to reach the server due to a network connectivity problem.
The error message string can be:
•
•
•
•
•
•
Recommended Action Verify network connectivity to the server, then reprobe the server.
251010
Error Message %ACE-3-251010: Health probe failed for server A.B.C.D on port P, error messageExplanation The configured service on port P of server A.B.C.D. failed its health checks because the server response is not as expected.
The error message string can be:
•
•
•
•
•
•
•
•
•
•
•
•
Recommended Action Check the service running on the affected server.
251011
Error Message %ACE-3-251011: ICMP health probe failed for server A.B.C.D, error message.Explanation The configured real server A.B.C.D. failed its health checks because the ICMP server response is not as expected.
The error message string can be:
•
•
•
•
•
•
•
•
Recommended Action Check the service running on the affected server.
251012
Error Message %ACE-3-251012: Could not load script script-name - Error reading script-fileExplanation This message is logged when the ACE is unable to read the script file it is attempting to load. The file may be corrupted.
Recommended Action Verify if the file contents are correct. If correct, unload, then reload the script file (see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide for details). If necessary, create a new script file. Unload the old file, then load the new file.
251013
Error Message %ACE-3-251013: Could not load script script-name - Error getting file sizeExplanation This message is logged when the ACE is unable to determine the script file size. Before a script file can be loaded, the ACE needs determine its size so the appropriate amount of memory can be allocated.
Recommended Action Verify if the file contents are correct. If correct, unload, then reload the script file (see the Cisco Application Control Engine Module Server Load-Balancing Configuration Guide for details). If necessary, create a new script file. Unload the old file, then load the new file.
Messages 302022 to 327001
This section contains messages from 302022 to 327001.
302022
Error Message %ACE-6-302022: Built TCP connection id for interface:real-address/real-port (mapped-address/mapped-port) to interface:real-address/real-port (mapped-address/mapped-port)Explanation This is informational message is logged when a TCP connection slot between two hosts is created.
Recommended Action None required.
302023
Error Message %ACE-6-302023: Teardown TCP connection id for interface:real-address/real-port to interface:real-address/real-port duration hh:mm:ss bytes bytes [reason]Explanation This is informational message is logged when a TCP connection slot between two hosts is terminated.
The reason variable presents the action that causes the connection to terminate. Set the reason variable to one of the TCP termination reasons listed in Table 2-1.
Recommended Action None required.
302024
Error Message %ACE-6-302024: Built UDP connection id for interface:real-address/real-port (mapped-address/mapped-port) to interface:real-address/real-port (mapped-address/mapped-port)Explanation A UDP connection slot between two hosts was added.
Recommended Action None required.
302025
Error Message %ACE-6-302025: Teardown UDP connection id for interface:real-address/real-port to interface:real-address/real-port duration hh:mm:ss bytes bytesExplanation A UDP connection slot between two hosts was deleted.
Recommended Action None required.
302026
Error Message %ACE-6-302026: Built ICMP connection for faddr/NATed_ID gaddr/icmp_type laddr/icmpIDExplanation An ICMP session was established.
Recommended Action None required.
302027
Error Message %ACE-6-302027: Teardown ICMP connection for faddr/NATed ID gaddr/icmp_type laddr/icmpIDExplanation An ICMP session was removed.
Recommended Action None required.
302028
Error Message %ACE-6-302028: Built TCP connection id for interface: real-address/real-port (mapped-address/mapped-port) to interface: real-address/real-port (mapped-address/mapped-port)Explanation A TCP connection slot between two hosts was created.
Recommended Action None required.
302029
Error Message %ACE-6-302029: Teardown TCP connection id for interface: real-address/real-port to interface: real-address/real-port duration hh:mm:ss bytes bytes [reason]Explanation A TCP connection between two hosts was terminated.
The reason variable presents the action that causes the connection to terminate. Set the reason variable to one of the TCP termination reasons listed in Table 2-1.
Recommended Action None required.
302030
Error Message %ACE-6-302030: Built UDP connection id for interface: real-address/real-port (mapped-address/mapped-port) to interface: real-address/real-port (mapped-address/mapped-port)Explanation A UDP connection slot between two hosts was added.
Recommended Action None required.
302031
Error Message %ACE-6-302031: Teardown UDP connection id for interface: real-address/real-port to interface: real-address/real-port duration hh:mm:ss bytes bytesExplanation A UDP connection slot between two hosts was deleted.
Recommended Action None required.
304001
Error Message %ACE-5-304001: user source_address Accessed {URL} dest_address: url.Explanation This is a URL message that is logged when the specified host attempts to access the specified URL.
Recommended Action None required.
313004
Error Message %ACE-4-313004: Denied ICMP type=icmp_type, from source_address on interface interface_name to dest_address:no matching sessionExplanation ICMP packets were discarded by the ACE because of security checks added by the stateful ICMP feature. These ICMP packets are discarded for any of the following reasons:
•
•
Recommended Action None required.
313006
Error Message %ACE-2-313006: ICMP Manager Initialization Failed. Reason : Variable1Explanation This message is logged when the ICMP Manager running on the Control Plane of the ACE fails to start.
The Variable1 string can be:
•
•
•
•
•
Recommended Action The ACE should automatically reboot the card. If not, try rebooting manually. If the problem still exists, contact Cisco TAC and provide them with the output of show tech-support command.
313007
Error Message %ACE-3-313007: ICMP Manager Memory Problem. Reason: Variable1Explanation Reports ICMP-related memory failures.
The Variable1 string can be:
•
•
•
•
Recommended Action Reboot the ACE (see the Cisco Application Control Engine Module Administration Guide for details). If the problem persists, contact Cisco TAC and provide them with the following command output:
•
•
322001
Error Message %ACE-3-322001: Deny MAC address MAC_address, possible spoof attempt on interface interfaceExplanation The ACE received a packet from the offending MAC address on the specified interface, but the source MAC address in the packet is statically bound to another interface in your configuration. This could be caused by either be a MAC-spoofing attack or a misconfiguration.
Recommended Action Check the configuration and take appropriate action by either finding the offending host or reconfiguring.
322002
Error Message %ACE-3-322002: ARP inspection check failed for arp {request|response} received from host MAC_address on interface interface. This host is advertising MAC Address MAC_address_1 for IP Address IP_address, which is {statically|dynamically} bound to MAC Address MAC_address_2.Explanation If ARP inspection is enabled, the ACE checks whether a new ARP entry advertised in the packet conforms to the statically configured or dynamically learned IP-MAC address binding before forwarding ARP packets. If this check fails, the ACE drops the ARP packet and generates this message. This situation can be caused by either ARP spoofing attacks in the network or an invalid configuration (IP-MAC binding).
Recommended Action If the cause is an attack, deny the host by using an ACL. If the cause is an invalid configuration, correct the binding (refer to the Cisco Application Control Engine Module Routing and Bridging Configuration Guide for details).
322003
Error Message %ACE-3-322003: ARP inspection check failed for arp {request|response} received from host MAC_address on interface interface. This host is advertising MAC Address MAC_address_1 for IP Address IP_address, which is not bound to any MAC Address.Error MessageExplanation If ARP inspection is enabled, the ACE checks whether a new ARP entry advertised in the packet conforms to the statically configured IP-MAC address binding before forwarding ARP packets. If this check fails, the the ACE drops the ARP packet and generates this message. This situation may be caused by either ARP spoofing attacks in the network or an invalid configuration (IP-MAC binding).
Recommended Action If the cause is an attack, deny the host by using an ACL. If the cause is an invalid configuration, correct the binding (refer to the Cisco Application Control Engine Module Routing and Bridging Configuration Guide for details).
327001
Error Message %ACE-3-327001: Detected Encap table Full when allocating encap entry for IP interface interface_nameExplanation The Encap table size is limited to 32,000 entries. This message is logged when trying to allocate an encap entry after the limit is reached.
Recommended Action Use the clear arp command to remove any unused or invalid table entries.
Messages 400000 to 442003
This section contains messages from 400000 to 442003.
400000
Error Message %ACE-4-400000: IDS:1000 IP Option Bad Option List from IP_address to IP_address on interface interface_nameExplanation Cisco Intrusion Detection System signature message.
Recommended Action Refer to the Cisco Intrusion Detection System User Guide.
405001
Error Message %ACE-4-405001: Received ARP {request | response} collision from IP_address/mac_address on interface interface_nameExplanation The ACE received an ARP packet, and the MAC address in the packet differs from the ARP cache entry.
Recommended Action This traffic might be legitimate, or it might indicate that an ARP poisoning attack is in progress. Check the source MAC address to determine where the packets are coming from and determine if the host is valid.
410001
Error Message %ACE-4-410001: Dropped UDP DNS packet_type from source_interface:source_address/source_port to dest_interface:dest_address/dest_port; error_length_type length length bytes exceeds max_length_type limit of maximum_length bytes.Explanation This message is printed when the domain-name length exceeds 255 bytes in a UDP DNS packet. (See RFC 1035 section 3.1.)
Recommended Action None required.
411001
Error Message %ACE-4-411001: Line protocol on interface interface_name changed state to upExplanation The status of the line protocol has changed from down to up.
Recommended Action None required.
411002
Error Message %ACE-4-411002: Line protocol on interface interface_name changed state to downExplanation The status of the line protocol has changed from up to down.
Recommended Action If this is an unexpected event on the interface, check the line.
411003
Error Message %ACE-4-411003: Configuration status on interface interface_name changed state to upExplanation The configuration status of the interface has changed from down to up.
Recommended Action If this is an unexpected event on the interface, check the line.
411004
Error Message %ACE-4-411004: Configuration status on interface interface_name changed state to downExplanation The configuration status of the interface has changed from up to down.
Recommended Action None required.
412001
Error Message %ACE-4-412001: MAC MAC_address moved from interface_1 to interface_2Explanation This message is generated when the ACE detects that a host was moved from one module interface to another. In a transparent ACE, mapping between the host (MAC) and the ACE port is maintained in a Layer 2 forwarding table. The table dynamically binds packet source MAC addresses to an ACE port. When movement of a host from one interface to another interface is detected during this binding process, this error message is generated.
Recommended Action The host move might be valid or the host move might be an attempt to spoof host MACs on other interfaces:
•
•
440002
Error Message %ACE-3-440002: Addition failed for variable 1Explanation This message is logged if there was an error for the SNMP Shadow Table Addition. SNMP Get / Get-Next requests might fail on the table name specified by variable 1.
Recommended Action Check the memory-related information in the system. Execute the show processes cpu memory command and locate the MemAlloc column in the output.
440003
Error Message %ACE-3-440003: Deletion failed for variable 2Explanation This message is logged if there was an error for the SNMP Shadow Table Deletion. Failure might result in a memory leak or wrong or non-existent values being returned for subsequent Get / Get - Next requests on the table name specified by variable 2.
Recommended Action Check the Memory related information in the system. Execute the show processes cpu memory command and locate the MemAlloc column in the output.
442001
Error Message %ACE-4-442001: Health probe probe name detected real_server_name (interface interface_name) in serverfarm sfarm_name changed state to UPExplanation This message is generated when the state of a real server changes from down to up.
Recommended Action None required.
442002
Error Message %ACE-4-442002: Health probe probe name detected real_server_name (interface interface_name) in serverfarm sfarm_name changed state to DOWNExplanation This message is generated when the state of a real server changes from up to down.
Recommended Action None required.
442003
Error Message %ACE-5-442003: Real Server real_server_name in serverfarm sfarm_name changed state to new stateExplanation Logs this message to report a real server state change.
The new state string can be:
•
•
•
Recommended Action None required.
Messages 504001 to 504002
This section contains messages from 504001 to 504002.
504001
Error Message %ACE-5-504001: Security context context-name was added to the systemExplanation A security context was successfully added to the system.
Recommended Action None required.
504002
Error Message %ACE-5-504002: Security context context-name was successfully removed from the systemExplanation A security context was successfully removed from the system.
Recommended Action None required.
Messages 615003 to 615004
This section contains messages from 615003 to 615004.
615003
Error Message %ACE-6-615003: VLAN VLAN-number not available for configuring an interfaceExplanation This message indicates that the specified VLAN number is no longer assigned to the ACE. If an interface is configured with that VLAN number on the module, it will be kept in a shutdown state. If an interface is already configured with that VLAN and is up, it will change state to shutdown.
Recommended Action If the VLAN specified in the log message is not required for the ACE, delete all interfaces that use this VLAN from the module configuration.
615004
Error Message %ACE-6-615004: VLAN VLAN-number available for configuring an interfaceExplanation This message indicates that the specified VLAN number is now assigned to the ACE. The module can use that VLAN to configure an interface and receive traffic on it.
Recommended Action To use the new VLAN, configure interfaces on the ACE using the new VLAN.
Messages 727001 to 728032
This section contains messages from 727001 to 728032.
727001
Error Message %ACE-1-727001: hostname: HA: Peer IP address is not reachable. Error: error str.Explanation This message is logged when an Active/Standby device cannot reach its redundant peer. This message is displayed on both devices and causes a switchover on the standby device. After the switchover occurs, both devices are no longer redundant. The error str value can be:
•
•
Recommended Action Verify connectivity between the peers. If a peer device is physically UP but connectivity is the problem, you may end up with two Active devices. If connectivity is lost due to peer going DOWN, reboot the peer to restore redundancy between the two devices.
727002
Error Message %ACE-1-727002: hostname: HA: FT interface interface name to reach peer IP address is down. Error: error strExplanation This message is logged when a peer device is not reachable on an FT Interface. In this situation the standby device does not switchover to Active, preventing two actives in the network. The error str value can be:
•
•
Recommended Action Verify connectivity between the two devices over the FT interface. Ping or Telnet to the peer IP address to confirm connectivity.
727003
Error Message %ACE-1-727003: hostname: HA: Mismatch in context names detected for FT group FTgroupID. Cannot be redundant.Explanation This message is logged when redundancy is enabled for a particular context, but both devices are unable to become Active/Standby because of a mismatch in context names.
Recommended Action Check the FT group configuration on both devices. Make sure both devices are associated with the same context.
727004
Error Message %ACE-1-727004: hostname: HA: Two actives have been detected for FT group FTgroupID.Explanation This message is an indication that both devices were detected to be Active for the same FT group. At this point, one of the two devices automatically relinquishes control and switches over to Standby.
Recommended Action None required.
727005
Error Message %ACE-1-727005: hostname: HA: Config replication failed for context ctx name. Error : error strExplanation This message is logged when a configuration could not be synced to the peer device due to the error condition returned by the error str value. The error str value can be:
•
•
•
•
•
•
•
•
•
Recommended Action Check the running and startup configurations on both devices. To recover, disable configuration sync, then manually apply the configuration on each device.
727006
Error Message %ACE-1-727006: hostname: HA: Peer is incompatible due to error str. Cannot be Redundant.Explanation This syslog is displayed when a peer device failed to become compatible. This can be a result of Software Relationship Graph (SRG) version inconsistency or mismatch in licenses between the devices. The error string indicates the reason for the failure.
The error str value can be:
•
•
Recommended Action Verify version and license compatibility on both the devices.
727007
Error Message %ACE-1-727007: hostname: HA: Module Initialization failure - Error Error str.Explanation This message is logged when there is an initialization error for one of the redundant modules. The error string indicates the reason for the failure.
The Error str value can be:
•
•
•
•
Recommended Action Contact Cisco TAC.
727008
Error Message %ACE-1-727008: hostname: HA: Failed to send heartbeats to peer. Internal error: Error strExplanation This message is displayed when the device is unable to send heartbeats to its peer due to an internal error. The error string indicates the reason for the failure.
The Error str value can be:
•
•
•
Recommended Action Contact Cisco TAC.
727009
Error Message %ACE-2-727009: hostname: HA: Communication failure for Peer Peer id Event: error strExplanation This informational message is logged when the device is unable to establish a TCP connection to the peer. The error str value is "Failed to establish TCP connection to Peer device."
Recommended Action Contact Cisco TAC.
727010
Error Message %ACE-2-727010: hostname: HA: Data replication failed for context ctx name. Error code error strExplanation This informational message is logged when data replication fails and data could not be successfully synced to the peer device. The next periodic sync will correct the failure and update the lost records. The error string indicates the reason for the failure.
The error str value can be:
•
•
Recommended Action None required.
727011
Error Message %ACE-2-727011: HA: Configuration replication for context ctx name will not happen. Error: Error strExplanation This message appears when configuration synchronization does not occur for a context. The error string indicates the reason for the failure.
The Error str value can be:
•
•
•
•
Recommended Action None required.
727012
Error Message %ACE-2-727012: hostname: HA: FT Group group ID changed state to NewState. Reason: reason str.Explanation This message displays the state transitions made by an HA state (redundancy) device for a context.
Values for the NewState variable are described in Table 2-2
.
Values returned for the reason str variable can be:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Recommended Action None required.
727013
Error Message %ACE-2-727013: hostname: HA: Peer Peer # is UP and reachable.Explanation This message indicates that the peer is now reachable. Heartbeats are flowing successfully between the two peers.
Recommended Action None required.
727014
Error Message %ACE-2-727014: hostname: HA: Heartbeats from Peer Peer id have become unidirectional.Explanation This message indicates that redundancy heartbeats from a peer have become unidirectional. That is, the peer cannot receive (only send) heartbeats. This problem occurs if one of the network processors has a problem.
Recommended Action Collect network processor drop counters, then contact Cisco TAC.
727015
Error Message %ACE-2-727015: hostname: HA: Detected mismatch in heartbeat interval from Peer peer id. Modified interval to interval.Explanation This message indicates that the redundancy heartbeat received from one peer differs from the value of the second peer. This condition can occur when you choose to dynamically change the heartbeat interval. The modified heartbeat interval that is displayed shows the adjusted interval. This value is the greater of the two values.
Recommended Action None required.
727016
Error Message %ACE-2-727016: hostname: HA: Replication for context ctx name has started. Status - status.This message indicates that the replication is being carried out to a peer. The status variable indicates the sync status. .
Values for the status variable can be:
•
•
•
•
•
•
Recommended Action None required.
727017
Error Message %ACE-2-727017: hostname: HA: FT Track track type track name is UP.Explanation This message indicates that FT track is UP.
The track type variable can be:
•
•
•
Recommended Action None required.
727018
Error Message %ACE-2-727018: hostname: HA: FT Track track type track name is DOWN.Explanation This message indicates that FT track is DOWN.
The track type variable can be:
•
•
•
Recommended Action None required.
727019
Error Message %ACE-5-727019: hostname: HA: Started alternate ping to IP address ip addrExplanation This message indicates that ICMP pings have started on the alternate interface to check the health of the peer. This process starts when heartbeats from peer are no longer received. The Standby device issues an alternate ping to the peer to determine whether the peer is still alive. If alive, it does not switchover, preventing two Active states on the network.
Recommended Action None required.
727020
Error Message %ACE-5-727020: hostname: HA: Stopped alternate ping to IP address ip addr.Explanation This message indicates that ICMP pings have stopped on the alternate interface. This occurs when heartbeats from the peer are received and the peer is up and reachable.
Recommended Action None required.
727021
Error Message %ACE-5-727021: hostname: HA: Peer is compatible.Explanation This message indicates that the two devices are in a compatible state and can be configured for redundancy.
Recommended Action None required.
727022
Error Message %ACE-5-727022: hostname: HA: Started sending heartbeats to peer Peer id interval value and count cntExplanation This message indicates that the redundancy connections to the peer have been successfully established and heartbeats have been started to peer with the configured interval and count.
The interval variable specifies interval in milliseconds. The count variable specifies the number of missed heartbeat intervals before the peer is declared Down.
Recommended Action None required.
727023
Error Message %ACE-5-727023: hostname: HA: Stopped sending heartbeats to peer Peer id.Explanation This message indicates that redundancy heartbeats to the peer have been stopped. This can occur if you deconfigure redundancy or make changes to basic connection parameters such as peer IP address.
Recommended Action None required.
728001
Error Message ACE-1-728001: Initialization failure (general) type variable1Explanation Initialization of the ACE load-balancing process is aborted due to a fundamental failure of a "general" nature (for example, lack of memory, failure to spawn threads, failure to establish a communication channel).
variable1 specifies the exact failure location in the codebase.
Recommended Action Record the syslog message, then reboot the ACE (see the Cisco Application Control Engine Module Administration Guide for details). Contact Cisco TAC with the recorded message text.
728002
Error Message ACE-1-728002: Initialization failure (sticky) type variable1Explanation Initialization of the ACE load-balancing process is aborted because of a fundamental failure in the "sticky" subsystem (for example, memory alignment failure, failure to spawn threads, failure to a establish communication channel.)
variable1 specifies the exact failure location in the codebase.
Recommended Action Record the syslog message, then reboot the ACE (see the Cisco Application Control Engine Module Administration Guide for details). Contact Cisco TAC with the recorded message text.
728003
Error Message ACE-1-728003: Initialization failure (sticky hash) variable1 entries, variable2 min, variable3 max type variable4Explanation Initialization of the ACE load-balancing process is aborted because of a fundamental failure when allocating entries for the sticky database (the database is not allocated.)
The variables displayed in this message represent the following:
•
•
•
•
Recommended Action Record the syslog message, then reboot the ACE (see the Cisco Application Control Engine Module Administration Guide for details). Contact Cisco TAC with the recorded message text.
728004
Error Message %ACE-5-728004: Internal communications notice (general) type variable1Explanation The ACE load-balancing process detects a spurious or unintelligible internal message that cannot be dispatched. Under a high load situation, message loss may occur.
variable1 specifies the exact failure location in the codebase.
Recommended Action If this message occurs frequently, or in conjunction with problems in load balancing under normal system load, contact Cisco TAC.
728005
Error Message %ACE-3-728005: Failed to transmit variable1 decision for connection from client variable2 type variable3Explanation A load-balancing decision was lost internally. No server connection can be initiated, and the identified client connection is reset. At this point, the client can attempt reconnection.
variable1 specifies the connection type. Possible values are:
•
•
•
variable2 specifies the address of client from whom the connection originated.
variable3 specifies the unique identifier for the line of code where the error was logged.
Recommended Action If this message occurs frequently, record the syslog message, then contact Cisco TAC.
728006
Error Message %ACE-5-728006: Internal communications error (messaging) msg subType variable1 -- type variable2During load balancing, the ACE received an internal message that cannot be identified. This message is discarded without processing.
The variables displayed in this message represent the following:
•
•
Recommended Action If this message occurs frequently, record the syslog message, then contact Cisco TAC.
728007
Error Message %ACE-3-728007: Internal configuration communications error (sticky) type variable1Explanation During load balancing, the ACE received a configuration request for sticky database resources that cannot be honored. The resources might exceed the permitted amounts or the resources cannot be located.
variable1 specifies the unique identifier for the line of code where the error was logged.
Recommended Action Verify that the requested resources are available within the chosen context. If the requested resources are available and are allowed by the configuration, an internal error exists. Contact Cisco TAC.
728008
Error Message %ACE-3-728008: Internal communications error (sticky) /source processor variable1 destination processor variable2 -- type variable3Explanation During load balancing, the ACE detected an error in communication between the two network processors. As a result, sticky load balancing may not occur for some client connections.
The variables displayed in this message represent the following:
•
•
•
Recommended Action Contact Cisco TAC.
728009
Error Message %ACE-3-728009: Context ID variable1 requested variable2 of variable3 sticky entries. No action taken. -- type variable4Explanation This message is reported from the Admin context. A configuration request from the context identified by variable1 cannot be responded to because it exceeds the permitted resources for the sticky entries.
The variables displayed in this message represent the following:
•
•
•
•
Recommended Action Contact Cisco TAC.
728011
Error Message %ACE-4-728011: Context ID variable1 being variable2 should not have variable3 associated sticky groups -- type variable4Explanation This message is reported from the Admin context and appears when adding or removing a context that has associated sticky groups. When this condition exists and the error message is logged, the addition or removal of the context still occurs.
variable1 specifies the context identifier to be added or removed.
variable2 specifies the requested action. Possible values are:
•
•
variable3 specifies the number of associated sticky groups detected.
variable4 specifies the unique identifier for the line of code where the error was logged.
Recommended Action Before adding or removing a context, make sure there are no sticky groups associated with that context.
728012
Error Message %ACE-5-728012: Context ID variable1 failed to receive return data -- type variable2Explanation Data collected in response to a show command at the CLI was not successfully returned from the network processor to the CLI.
The variables displayed in this message represent the following:
•
•
Recommended Action Reissue the show command. If the problem persists, contact Cisco TAC.
728013
Error Message %ACE-4-728013: A cache alignment error variable1 was detected during initialization -- type variable2Explanation A cache alignment error was detected during load-balancing initialization. This may impact performance, but load balancing will still be correctly performed.
The variables displayed in this message represent the following:
•
•
Recommended Action If you see this error message frequently, contact Cisco TAC.
728014
Error Message %ACE-3-728014: Internal cross-processor communications error (sticky) type variable1Explanation During load-balancing, the ACE could not parse a message from the second network processor on the ACE. This can result in the loss of sticky information between the two processors, resulting in a sticky server-connection loss for some clients.
variable1 specifies the unique identifier for the line of code where the error was logged.
Recommended Action Contact Cisco TAC.
728015
Error Message %ACE-3-728015: Internal channel communications error (sticky) type variable1Explanation During load balancing operations, the ACE was unable to open or use an internal communications channel to process a load-balancing configuration or a display directive. The specific directive on which the failure occurred is not be completed (although it may be retried).
variable1 specifies the unique identifier for the line of code where the error was logged.
Recommended Action Contact Cisco TAC.
728016
Error Message %ACE-4-728016: HA data receive failure (type variable1)Explanation This message is logged when an redundancy message received from the redundant peer cannot be understood and is subsequently discarded.
variable1 specifies the unique identifier for the line of code where the error was logged.
Recommended Action Use the show ft stats group_id command to display load-balancing statistics for the FT group:
•
•
728017
Error Message %ACE-3-728017: Internal communications error (ha) -- type variable1Explanation This message is reported from the current context. An attempt to send a redundancy message to the redundant peer was unsuccessful because the message could not be sent.
variable1 specifies the unique identifier for the line of code where the error was logged.
Recommended Action Use the show ft stats group_id command to display load-balancing statistics for the FT group. Monitor the "Number of Send Failures" value. Contact Cisco TAC if the problem persists.
728018
Error Message %ACE-5-728018: Proxy connection variable1 rebalanced to server variable2Explanation The ACE has determined that a the server side of a connection should be rebalanced to another server. This is an informational message issued in the context in which the rebalance occurs.
The variables displayed in this message represent the following:
•
•
Recommended Action None required.
728019
Error Message %ACE-4-728019: Sticky resources were not variable1 for this context -- type variable2Explanation A sticky request (lookup, configure, or delete a sticky entry) was not honored because the sticky group could not locate any configured sticky entries. This is not the result of exceeding the configuration limits, but indicates an unexpected sticky group lookup result.
variable1 specifies the requested sticky action. Possible values are:
•
•
•
variable2 specifies the unique identifier for the line of code where the error was logged.
Recommended Action Contact Cisco TAC.
728020
Error Message %ACE-6-728020: LB is configured to consume variable1 bytes of memory.Explanation The message indicates the amount of physical memory that is mapped by the ACE during load-balancing initialization and indicates that the mapping was successful.
variable1 specifies the bytes of mapped physical memory.
Recommended Action None required.
728021
Error Message %ACE-6-728021: Found inconsistent sticky entry. Terminating variable1.Explanation Various commands processed by the ACE during load balancing require searching the sticky database to find all relevant sticky entries. An unexpected finding of no further sticky entries generates this message. The indicated action is terminated, but further requests of the same type (or of other types) are completed.
variable1 specifies the terminated action. Possible values are:
•
•
•
Recommended Action None required. This message is useful in troubleshooting sticky issues.
728022
Error Message %ACE-6-728022: Invalid hash table index (variable1) used for variable2Explanation The specified action was aborted because of an invalid hash index.
variable1 specifies the value of the invalid hash table index.
variable2 specifies the index table use. Possible values are:
•
•
Recommended Action None required.
728023
Error Message %ACE-6-728023: variable1 variable2 sticky entries from ContextId variable3.Explanation The message indicates that sticky entries have been added or removed from a context as a result of a resource limit change.
variable1 specifies the action. Possible values are:
•
•
variable2 specifies the number of sticky entries moved.
variable3 specifies the context ID from which the entries were added or removed.
Recommended Action None required.
728024
Error Message %ACE-4-728024: Received an unknown variable1 type message (variable2) for Sticky from remote IXP variable3!Explanation A request or reply from the second network processor indicates an unknown operation type. The request or reply is not responded to and is discarded.
variable1 specifies the message class. Possible values are:
•
•
variable2 specifies the numerical value of the operation type that could not be identified.
variable3 specifies the identifier of the IXP (network processor) that sent the message.
Recommended Action No action required. This message is useful when troubleshooting sticky database synchronization problems with the network processors.
728025
Error Message %ACE-6-728025: Dropped variable1 'variable2' messages (variable3 total) from IXP variable4 to IXP variable5!Explanation This message indicates that sticky messages between network processors (sticky insert, sticky lookup, or sticky connection close) were lost.
variable1 specifies the number of lost messages.
variable2 specifies the message type. Possible values are:
•
•
variable3 specifies the total number of messages discarded (includes both lost messages and messages which were discarded because they could not be sent).
variable4 specifies the source network processor identifier.
variable5 specifies the destination network processor identifier.
Recommended Action None required. This information may be useful when troubleshooting problems with sticky functionality.
728026
Error Message %ACE-6-728026: Attempting to use invalid lookup key for variable1 processing.Explanation The message indicates that a connection close notification was not sent to the remote network processor because of an invalid key.
Recommended Action None required. This information may be useful in troubleshooting problems with sticky functionality. Variable1 specifies the type of processing ("connection close").
728027
Error Message %ACE-4-728027: Received unhandled message of type variable1 from CP SrcSAP variable2.Explanation This message is logged when an unrecognized message is received from the control processor (CP) during load-balancing operations. The message is discarded.
The variables displayed in this message represent the following:
•
•
Recommended Action No action required. This message is useful when troubleshooting commands or configuration directives from the control processor that are ignored by the ACE.
728028
Error Message %ACE-5-728028: Sticky mapping failed: variable1 variable2Explanation Information received from an redundant peer cannot be mapped locally. The associated sticky entry information is discarded.
variable1 specifies the reason for the mapping failure. Possible values are:
•
•
•
variable2 specifies the (decimal) identifier of the invalid entity. If the entry is an "invalid real server id," the value of the real server id is displayed. Otherwise, the invalid/inactive sticky group id is displayed.
Recommended Action Use the show ft stats group_id command to display load-balancing statistics for the FT group. Monitor the "Number of Sticky Entries Dropped" value. Contact Cisco TAC if the values continue to increase over time.
728029
Error Message %ACE-6-728029: HA state for FtGroup variable1 changed from variable2 to variable3 State variable4.Explanation This message tracks state changes received from the redundant peer. Events that are not relevant to load balancing are ignored.
variable1 specifies the (decimal) fault tolerant group id.
variable2 and variable3 specify the previous and current (respectively) state change event. Possible values are:
•
•
•
•
•
•
•
•
•
•
•
•
•
variable4 specifies the state change action. It is valid to ignore some state change events. Possible values are:
•
•
Recommended Action No action required. This message is useful when tracking redundancy state changes to troubleshoot redundant peer synchronization problems.
728030
Error Message %ACE-6-728030: Silently discarding HA data: variable1Explanation This is an informational message logged when redundancy data must be discarded during load-balancing operations because the ACE could not process the data. The discarding of the data could affect seamless failover.
variable1 specifies the reason for discarding data from the redundant peer. Possible values are:
•
•
Recommended Action No action required. This message is useful when troubleshooting redundant peer problems.
728031
Error Message %ACE-3-978031: Memory mapping for debug logging failed.Explanation This message is logged when memory mapping fails during initialization for debug logging. Load balancing continues, but no debug logging will occur, even if invoked from the command line.
Recommended Action Reboot the ACE to reinitialize the debug logging component (see the Cisco Application Control Engine Module Administration Guide for details). Rebooting may correct a transient mapping issue. If this error persists, contact Cisco TAC.
728032
Error Message %ACE-LB_General-4-728032: Real Server variable1 in Serverfarm variable2 has reached configured threshold for HTTP retcode variable3Explanation This message is logged if you have configured HTTP return codes on a server farm and a specific real server has reached the configured return code threshold.
The variables displayed in this message represent the following:
•
•
•
Recommended Action Review the types of client HTTP requests which cause these server return code responses. Look for return codes that point to possible problems, for example, missing content or incorrect search paths.