Guest

Cisco AON Software

Release Notes for Cisco Application-Oriented Networking 2.2

 Feedback

Table Of Contents

Release Notes for Cisco Application-Oriented Networking Version 2.2

Contents

New Features in Cisco AON 2.2

AON Application System Requirements

AON Supported Hardware

AON Node Supported Software

AON Appliance Supported Software

Important Notes

Resolved Caveats

Open Caveats

Upgrade Instructions

Backward Compatibility

Related Documentation

Obtaining Documentation

Cisco.com

Documentation DVD

Ordering Documentation

Documentation Feedback

Cisco Product Security Overview

Reporting Security Problems in Cisco Products

Obtaining Technical Assistance

Cisco Technical Support Website

Submitting a Service Request

Definitions of Service Request Severity

Obtaining Additional Publications and Information


Release Notes for Cisco Application-Oriented Networking Version 2.2


August 25, 2006

Cisco Application-Oriented Networking (AON) is the first in a new line of Cisco products that embed intelligence into the network to meet the needs of application deployment. AON enables you to:

Integrate dissimilar applications by routing information to the appropriate destination, in the format required at the destination.

Enforce policies for information access and exchange.

Optimize bandwidth and reduce processing overhead for application traffic.

Increase management of information flow, including monitoring for business and infrastructure.

Enhance business continuity by transparently backing up or rerouting critical business data.

Working at the message rather than packet level, AON provides this support by understanding more about the content and context of information flow.

Contents

These release notes cover Cisco Application-Oriented Networking Version 2.2 and include the following topics:

New Features in Cisco AON 2.2

AON Application System Requirements

AON Supported Hardware

AON Node Supported Software

AON Appliance Supported Software

Important Notes

Resolved Caveats

Open Caveats

Upgrade Instructions

Backward Compatibility

Related Documentation

Obtaining Documentation

Documentation Feedback

Cisco Product Security Overview

Obtaining Technical Assistance

New Features in Cisco AON 2.2

Cisco Application-Oriented Networking Version 2.2 introduces a new hardware platform—Cisco AON Enhanced Network Module—that provides a technology foundation with an intelligent network for deploying applications. The AON enhanced network module complements existing network technologies by providing a greater degree of awareness of what information is flowing within the network and helps customers achieve the following goals:

Integrate disparate applications by routing information to the appropriate destinations, in the form expected by that destination.

Enforce security policies for information access and exchange.

Optimize the flow of application traffic, in both network bandwidth and processing overheads.

Provide increased manageability of information flow, including monitoring and metering of information flow for both business and infrastructure purposes.

AON Application System Requirements

Table 1 lists the minimum requirements for installing AON applications for AON Release 2.2.

Table 1 AON Minimum System Requirements

Application
Operating System
CPU
RAM
Hard Drive
Software Image

AON Management Console (AMC)

Red Hat Enterprise Linux 3.0 or later

Single processor;
Pentium III or Xeon

1 GB

20 GB

AON 2.2.0.145

AON Development Studio (ADS)

Windows 2000 or Windows XP with latest service packs.

Pentium IV

1 GB (required)

2 GB (recommended for large adapters)

40 GB

AON 2.2.0.145


AON Supported Hardware

Table 2 lists the hardware platforms that are supported by AON version 2.2.

Table 2 Supported Hardware

AON Appliance
AON Service Module (AON-SM)
AON Network Module (AON-NM)
AON Enhanced Network Module (AON NME)

Cisco 8340 AON Appliance

APL-AON-8340-K9

Cisco 8342 AON Appliance

APL-AON-8342-K9

WS-6503

WS-C6503-E

WS-C6506

WS-6506-E

WS-C6509

WS-6509-E

WS-C6509-NEB-A

WS-6513

Cisco 2610XM

Cisco 2611XM

Cisco 2620XM

Cisco 2650XM

Cisco 2651XM

Cisco 2691XM

Cisco 2811

Cisco 2821

Cisco 2851

Cisco 3725

Cisco 3745

Cisco 3825

Cisco 3845

Cisco 2811

Cisco 2821 

Cisco 2851

Cisco 3725

Cisco 3745

Cisco 3825

Cisco 3845


AON Node Supported Software

Table 3 lists the software levels for the Cisco platforms that support AON.

Table 3 Supported Software on Nodes

Platform
Minimum Software Release Supported
Latest Software Release Supported

Native AON-SM

Catalyst 6500 Series Switches with Supervisor Engine 720

Cisco IOS Release 12.2(18)SXE1

Cisco IOS Release 12.2(18)SXF2

Hybrid AON-SM

Catalyst 6500 Series Switches with Supervisor Engine 720

Cisco IOS Release 12.2(18)SXF

CatOS Release 8.5(3)

CatOS Release 8.5(3)

Cisco IOS Release 12.2(18)SXF

Native AON-SM 2

Catalyst 6500 Series Switches with Supervisor Engine 2

Cisco IOS Release 12.2(18)SXF2

Cisco IOS Release 12.2(18)SXF2

Hybrid AON-SM 2

Catalyst 6500 Series Switches with Supervisor Engine 2

CatOS Release 8.4(2a)

Cisco IOS Release 12.1(23)E3

CatOS Release 8.5(3)

Cisco IOS Release 12.2(18)SXF2

AON-NM

Cisco 2600, Cisco 2800, Cisco 3700,
and Cisco 3800 Series Routers

Cisco IOS Release 12.3(14)T1

Cisco IOS Release 12.4(3)

AON-NME

Cisco 2800, Cisco 3700, and
Cisco 3800 Series Routers

Cisco IOS Release 12.4(9)T


AON Appliance Supported Software

Table 4 lists the software levels for the AON Appliance that support AON.

Table 4 Supported Software on Appliance

Platform
Minimum Software Release Supported
Latest Software Release Supported

Cisco 8340 AON Appliance

AON version 1.1.0.189

AON version 2.1.2.29 (with firmware upgrade)

AON version 2.2.0.145

Cisco 8342 AON Appliance

AON version 2.1.2.29

AON version 2.2.0.145


Important Notes

The AON Management Console (AMC) supports only Microsoft Internet Explorer 6. AMC pages may not render properly in other Web browsers.

AON is implemented in Java where memory is automatically managed by the Java runtime system. This means that there might be moments in the system where the garbage collection (automatic memory management) is still working at freeing up memory. The graceful handling mechanism checks the free memory to determine if a message should be let into the system. So under high loads it is possible that AON will reject messages because the garbage collection is taking time to free up memory.

The following issues may affect AON Development Studio installation, however, the root causes are beyond the control of Cisco:

Using the ALT key during ADS installation can cause some InstallShield screens to become corrupted. Despite this display problem, the ADS installer continues to function. If the display gets corrupted, minimize the ADS installer and then maximize it again. The display should return to normal. This is a known InstallShield issue when using JVMs with version 1.4.2.x.

In rare situations when initially launching ADS on Windows 2000, an error message may be returned indicating the database is busy or unavailable. The error can occur even though the database is listed as started in the list of Windows Services. This occurs when a database port is chosen in the ADS installer that also appears in the output of the netstat -a command in a loopback situation. The port is shown pointing to another server port which in turn points back to it. This behavior has only been seen with one port, though not always the same port on the system. Reboot the PC to correct this problem.

Resolved Caveats

Table 5 lists the caveats that have been resolved in this AON release.

Table 5 Resolved Caveats for Cisco Application-Oriented Networking Version 2.2 

Defect ID
Description

CSCej61795

Problem editing text field in extension config page.

CSCek25614

SNMP: Coldstart trap not being generated on reload of AON blade.

CSCek25745

UC-1: JVM Crashed on both AON Client Proxy nodes after 75 Hrs of Run.

CSCek25788

SOAP message send using MQAdapter fails Sign.

CSCek31535

amcdb lacks prompt ASA There are still active connections.

CSCek32197

WCCP service group ID is zero in ACL TCAM Adjacency.

CSCek32601

fpserver crashes on appliance with success sv flow.

CSCek33285

303 redirection response failure.

CSCek35365

Crash in SV when deploying xsd.

CSCek35399

MDS: An issue in shutting down AON module.

CSCek36038

AMC Global Deployment fails if 1st node in AMC is Inactive.

CSCek36267

Cannot boot AON module due to hard drive space ran out.

CSCek39791

JMS queue properties changes take effect only after AON restart.

CSCek36891

Enhancements on AMC Package size reduction and AMC DB security.

CSCin98529

Unable to create scar file on Win XP using .so without version.

CSCse03361

Dynamically listen for change in JMS resource file (.bindings).

CSCse03363

Dynamic update to SSL configuration is required.

CSCse01517

Loop Bladelet does not reset counters if called from within another loop.

CSCse60390

When ADS fails to login, user password is logged in the clear in AMC log (amc.log).

CSCse60415

ADS cannot handle more than a 32 character host name when connecting.

CSCse76661

AON fails to find SSL Policy if you use a host name instead of IP address.


Open Caveats

Table 6 lists the caveats for this AON release, including defect identification numbers and symptoms. When applicable, conditions under which the defects occur and workarounds are also included.

Table 6 Open Caveats for Cisco Application-Oriented Networking Version 2.2 

Defect ID
Description

CSCeh84583

Symptom

1. One-Way JMS message processed by "Distribute" bladelet fails to copy the message to JMS queue destinations.

2. Message appears in the deadletter queue defined in the JMS policy setup.

3. The AONS flow containing a Distribute bladelet fails with an error resembling the following appearing in aons.log:

06-May-2005 18:24:14 INFO [ MEC-Q-2] aons.mec.adapter.jms Received JMS soap message with id 
ID:MARTINF-JMS-SERVER.44C2D0672583B:3 06-May-2005 18:24:14 INFO [ MEC-Q-3] aons.mec.core 
Executing flow AON_MDS_TIBCOEMS_ORDERED_1SRCTO2DEST for MEC-ID-1115426485881-3- 0 06-May-2005 
18:24:14 ERROR [ MEC-Q-3] bladelet.Distribute Exception cloning and sending out the message : 
null 06-May-2005 18:24:14 ERROR [ MEC-Q-3] aons.mec.core Exception occured while executing 
MessageDistribution:1, ID : 1 : null 06-May-2005 18:24:14 INFO [ MEC-Q-2] aons.mec.core Handler 
Clean up for MEC-IdMEC-ID-1115426485881-3-0 06-May-2005 18:24:14 WARN [ MEC-Q-3] 
com.cisco.aons.DeliveryFailure %%FAILURE% %%received% 2005-05-06 18:24:14.739 %%source% 
jms://10.18.0.2:7222/ao ns.po.in.queue.6 %%dest% jms://10.18.0.2:7222/aons.po.in.queue.6 
%%corrid % Ch0B CgABA7SZ7q0DAAAAAAAC 06-May-2005 18:24:14 WARN [ MEC-Q-3] aons.mec.core 
%%FAILURE% %%received% 2005-05-06 18:24:14.739 %%source% jms://10.18.0.2:7222/ao 
ns.po.in.queue.6 %%dest% jms://10.18.0.2:7222/aons.po.in.queue.6 %%corrid % Ch0B 
CgABA7SZ7q0DAAAAAAAC 

Conditions

1. Message Delivery Semantics (MDS) reliable message enabled with ordering and reliable; with associated database policy configured and functional.

2. The One-Way AONS flow contains only a "Distribute" bladelet The purpose of the "Distribute" bladelet is to copy the incoming JMS message to two JMS queue destinations:

jms://10.18.0.2:7222/aons.po.out.queue.5

jms://10.18.0.2:7222/aons.po.out.queue.6

Workaround

Replace the "Distribute" bladelet with multiple "Send" bladelets. Only one "Send" bladelet in the resulting flow can inherit reliable/ordered delivery semantics. All other "Send" bladelets must have reliable/ordered delivery disabled.

CSCei10353

Symptom

Changes to WCCP load balancing do not take affect due to conflicting masks.

Conditions

User changes the load balancing for WCCP service group from the default source-IP to other values, including source port and source IP port on the AMC.

Workaround

If possible, keep the default load balancing configuration. If a conflict happens, remove and recreate the WCCP service group.

CSCej53292

Symptom

1. Out of buffers error (in case of http) or Out of memory error (in case of queue based adapters) in the log when a PEP with distribute bladelet is used.

2. The message does not get distributed to all the end points as expected.

Conditions

1. The message size is 80 MB and the platform is AON Service Module.

2. The message size is 10 MB and the platform is AON Network Module.

3. A PEP containing Distribute is executed.

Workaround

None.

CSCej53322

Symptom

Encryption or decryption of large messages gives Java Out-of-memory error in AON log, with the following entry:

WARN    [MEC-Q-4]       aons.mec.core   OutOfMemoryError in PoolWorker.run().Will notify VM 
to run GC java.lang.OutOfMemoryError: Java heap space   Condition  This could happen when 
the original message is a large message (around 80MB) and the structure of the message 
(that is being encrypted or decrypted) falls into the following descriptions (or increases 
the complexity of the message more than what are given)   Type 1: Huge number of nodes 
(around or more than 500,000), each node may have children and namespaces or attributes. In 
the following message template, the 'add' element falls into this category.    <?xml 
version="1.0" encoding="UTF-8"?>    <SOAP-ENV:Envelope 
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:xsd="http://www.w3.org/2001/XMLSchema"        
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">     <SOAP-ENV:Body>      <add 
xmlns="http://calculator.examples.soap.webtool">       <in0 xmlns="">DATA</in0>       <in1 
xmlns=""> DATA </in1>      </add>      <add 
xmlns="http://calculator.examples.soap.webtool">       <in0 xmlns="">DATA</in0>       <in1 
xmlns=""> DATA </in1>      </add>      ......      ......      <add 
xmlns="http://calculator.examples.soap.webtool">       <in0 xmlns="">DATA</in0>       <in1 
xmlns=""> DATA </in1>      </add>     </SOAP-ENV:Body>    </SOAP-ENV:Envelope>           
Type 2: Huge number of nodes (around or more than 500, 000), each node has only CDATA (no 
children), but the nodes may have namespaces or attributes. In the following message 
template, the 'add' element falls into this category.     <?xml version="1.0" 
encoding="UTF-8"?>     <SOAP-ENV:Envelope 
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:xsd="http://www.w3.org/2001/XMLSchema"        
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">       <SOAP-ENV:Body>         <add 
xmlns="http://calculator.examples.soap.webtool">        CDATA         </add>         <add 
xmlns="http://calculator.examples.soap.webtool">        CDATA         </add>        
........        .......         <add xmlns="http://calculator.examples.soap.webtool">        
CDATA         </add>       </SOAP-ENV:Body>     </SOAP-ENV:Envelope>    

Workaround

The following approaches can be used for overcoming this problem:

1. Verify if indeed the entire message needs to be encrypted, if only parts of the message need be encrypted, this problem can be overcome regardless of the total size of the message.

2. If the message can be split, then it can be sent as multiple messages that are smaller in size.

3. If the message structure can be altered, then determine if the number of nodes can be decreased.

CSCej55045

Symptom

There are various symptoms for this problem depending upon configuration, including:

A message is delivered after it has expired.

EMS adapter causes an exception.

"Duplicate message" error appears in the AON log.

Conditions

Messages created in the PEP with CreateMessage and CreateResponse when MDS is configured.

Workaround

Replace the CreateMessage bladelet with CreateContent and UpdateMessage bladelets to achieve the similar effect.

CSCej57944

Symptom

Message processing in a three-node scenario does not work correctly. Request and response paths do not traverse the same set of nodes.

Workaround

Three-node configurations are not supported in AON release 2.1. Use 2-node setup.

CSCej69080

Symptom

The show interfaces command on the AON-NM command-line interface does not show the MAC address of the Ethernet interface. This information is needed in order to configure promiscuous mode.

Workaround

To determine the MAC address of the internal data port on an AON-NM, you can enter the show arp command on the router command-line interface.

CSCek10570

Symptom

When logging object type of PEP variables, the string representation of the Object(toString) should be used and logged.

Condition

Using a PEP with Message Log configured to log content type PEP variable, AON logs the object as it is, which is not readable. Instead, it would be more useful to log the string representation of the object.

Workaround

None.

CSCek18587

Symptom

Message delivered even when TTL expired.

Condition

The associated PEP introduced an artificial delay of 60 seconds. The TTL was set for 30 seconds, yet the message was still delivered to the server proxy. The message should have been dropped.

Workaround

None.

CSCek20178

Symptom

EMS adapter with inbound batch size greater than 1 does not work properly. Success and failure notifications do not occur properly, causing incorrect message delivery notifications to occur.

Workaround

Use batch size = 1.

CSCek25145

Symptom

Errors occur when a MQ client or endpoint uses the same correlation-ID for multiple messages.

Workaround

Configure the MQ client to use a unique correlation-ID for each message.

CSCek25514

Symptom

Optimization does not support wild cards in URI for message type classification. In a message type URI, if a string such as /index* is specified, then Optimization classification does not classify messages with URIs /index.html and /index1.html to that message type.

Workaround

Use complete URI for message type classification. If more than one URI needs to be classified to a single message type, and hence execute the same PEP, define a message type for each URI and map all of these message types to the same PEP.

CSCek26871

Symptom

AON does not recognize messages that use HTTP/1.1 pipelining.

Workaround

None. HTTP/1.1 pipelining is not supported in the AON environment.

CSCek27572

Symptom

Endpoint receives HTTP HEAD request with body but with content length header value of zero.

Workaround

None.

CSCek28411

Symptom

Signature Verification Fails after decrypting a signed encrypted message. The message was created using the same X509 certificate for both signing and encryption operations.

Condition

Created a PEP that has Signature Verification and Decryption bladelets on the response path. After the request path PEP was successfully executed, the endpoint server should have responded back with encrypted signed message. Signature verification fails on response path after decrypting an encrypted message from end server. When only Signature Verification is used without any Decryption bladelet, the verification works fine. This is observed with Weblogic 8.1.

Workaround

None.

CSCek28868

Symptom

"Rollback Only" as a delivery failure policy in the JMS adapter configuration policy is no longer supported.

Workaround

None.

CSCek28910

Symptom

Graceful handling causes the following WARN level log message is generated:

"...WARN...STOP_REQUEST_THRESHOLD hit...". 

This indicates that the available free heap memory is too low to allow new messages into AON. When the amount of free heap memory increases, the following and a WARN level log message is generated:

"...WARN...START_REQUEST_THRESHOLD cleared...".  

Condition

The problem was seen while doing a test involving two-node MQ-to-JMS messages. The two nodes were an AON-NM and AON-SM. Five clients were sending 600-KB messages, maintaining 15 messages in the system at any time. On the AON-NM, graceful handling caused request messages to be denied. Later the system garbage collection freed up heap memory and messages were admitted.

Workaround

The request and response thresholds that govern graceful handling can be enhanced by editing the values of some properties in the aonsFactoryDefaults.properties file. The properties are:

aonsSystem.StopReqThreshold=0.3

aonsSystem.StopRespThreshold=0.2

aonsSystem.StopStartReqThresholdDiff=0.03

aonsSystem.StopStartRespThresholdDiff=0.04

Each of the above decimal values represents the percentage at which the given threshold is reached. These settings can be customized for the customer application to avoid graceful handling. These properties can also be set in the aons startup shell script to override the factory defaults.

CSCek29537

Symptom

Too many files open.

Condition

AON system receives very large messages (~60MB) over a long period of time.

Workaround

None.

CSCek29556

Symptom

Rows in the database for the CN_, SP_, and DY_ tables are not removed.

Condition

Occurred in a multi-node deployment requiring reliable/ordered delivery and using HTTP adapter on inbound and/or outbound.

Workaround

Manually drop old rows in these tables.

CSCek29582

Symptom

Unable to parse XML/SOAP documents. Manifestation of this problem appears in the form of security bladelet exception.

Condition

Under rare condition, contents of a HTTP body are corrupted. The root cause is unidentified.

Workaround

The client is notified of error by sending appropriate HTTP error code and it is expected that the client re-tries the message again.

CSCek29630

Symptom

AON fails to establish connection to Tibco EMS server.

Condition

This happens when Tibco EMS server is re-started multiple times after AON system has bootstrapped. It has been observed mostly in virtual cluster (VC) setup. In this situation, AON occasionally fails to establish connection to Tibco EMS server.

Workaround

Restart AON System. In case of VC setup, restart all AON systems participating in the VC.

CSCek29803

Symptom

Messages are sometimes lost and do not appear in the dead letter queue or the destination queue.

Condition

This occurs when the inbound source batch size is greater than 1 in JMS adapter configuration.

Workaround

Change the batch size of the inbound source (including replyTo source) to 1.

CSCek29828

Symptom

Unable to classify a JMS message on the destination URI even though source and destination policies are statically linked via the JMS adapter configuration.

Condition

This happened when source and destination queues are defined on two different JMS brokers and they are statically linked via the JMS adapter configuration.

Workaround

Define the source and destination on the same brokers.

CSCek29892

Symptom

EMS broker running out of resources when EMS adapter is configured incorrectly.

Condition

EMS queue type is Send and EMS adapter is configured as Receive or vice- versa. In this misconfiguration, adapter continues to try to connect to the broker to register itself, causing resource issues on the broker side.

Workaround

Correct the EMS adapter configuration.

CSCek30721

Symptom

Caching MQ and JMS messages causes buffer leaks which eventually results in buffer exhaustion. System is unable to process any messages after this condition. The following message is logged:

11-Oct-2005 17:51:54 DEBUG [MEC-Q-1] aons.mec.monitor *** Buffer space utilized = 6.686  
Under steady state conditions (when no messages are being processed) Buffer space utilized 
should be zero.   

Workaround

No Workaround. This problem does nor occur when caching HTTP messages.

CSCek30950

Symptom

A MsgType exists in AMC without its associated PEP in AMC. A series of exceptions are thrown because of the nonexistant PEP in the AON MsgType.

Condition

This bug occurs in the following sequence of steps:

1. Create a PEP F1 and MsgType M1 and associate this MsgType to F1.

2. Synchronize the PEP F1 and MsgType M1 to AMC.

3. Create another PEP F2 and associate MsgType M1 to this new PEP F2.

4. Now delete PEP F1 from ADS.

5. Synchronize the newly created PEP F2 and the deleted PEP F1; but do not delete the MsgType M1.

6. In AMC, F1 gets deleted and F2 gets created and the old record of MsgType M1 exists and still remains associated with the PEP F1 (which does not exist anymore).

Workaround

Delete this MsgType on ADS and synchronize it to AMC, so that the MsgType also gets deleted on the AMC and deploy it. Then, AON is able to run successfully without any exception.

CSCek31626

Symptom

URI based classification does not seem to work correctly. Messages are rejected even if there is an entry for that URI. This could happen if there are other message types that are classified based on 5-Tuple. This issue can be reproduced only in the following scenario.

Condition

Classification based on URI /index.html does not work correctly. Message type 't1' based on 5-Tuple 'a' and URI '/index-nomatch.html' and message type 't2' based just on URI '/index.html' Client messages that match 5-Tuple 'a' and URI '/index.html' does not get classified to type 't2'. Then the message is rejected.

Workaround

Add a message type 't3' that is based on 5-Tuple 'a' and URI '/index.html'

Or

If 5-Tuple based classification is not required for 't1' classification, remove the 5-Tuple detail from message type 't1'.

CSCek32772

Symptom

AON Appliance log timestamp is different from the system clock.

Condition

This happens after loading a new AON image. Due to the AON bootstrap, time is out of sync with the system time, certain scheduled AON tasks might not start according to scheduler, this could result with issues. In the worse case, under heavy load, it may cause system to be out of memory.

Workaround

Restart AON one more time from CLI.

CSCek33304

Symptom

AON doesn't forward the request via the proxy if the message is implicitly intercepted between a client and proxy.

Conditions

AON implicitly intercepts traffic between a client and proxy.

Workaround

None.

CSCek34188

Symptom

On a 2 node queue based adapters test, a memory leak was observed for reliable and ordered messages. Eventually this leak causes graceful handling too kick in and all new messages to be rejected.

Condition

Observed in a two-node scenario (AON NM as the client and AON SM as the server proxies) for MQ to JMS translation. The messages were being sent on reliable and ordered queues. The concurrency for 5 and message size for 600 KB. Due to the limited heap size on the AON SM, after few hours graceful handling kicks in and a few messages are rejected. The inbound side slows down considerably as the memory keeps increasing (due to the leak). Due to this leak, almost 1000 messages drop down to less than 100 messages in eight hours.

Workaround

There is no workaround for the two-node case. Switching to a single node setup will eliminate the leak.

Note This problem does not happen for HTTP cases or translation from HTTP to queue based adapters.

CSCek34255

Symptom

1. ama.log shows that the Mbean watchdog restarted AON.

2006-03-04 06:01:06,846 [Thread-38] ERROR ama.watchdog.MAgentWatchdog  - Exception connecting 
to MBean Server. AONS must be dead.
com.cisco.aons.ama.jmx.adaptor.http.HttpException: Connection refused
        at 
com.cisco.aons.ama.jmx.adaptor.http.HttpAdaptor.processAonsProxyRequest(HttpAdaptor.java:639)
        at com.cisco.aons.ama.watchdog.MAgentWatchdog.isAONSHealthy(MAgentWatchdog.java:741)
        at com.cisco.aons.ama.watchdog.MAgentWatchdog.access$100(MAgentWatchdog.java:40)
        at 
com.cisco.aons.ama.watchdog.MAgentWatchdog$HeartbeatRunnable.run(MAgentWatchdog.java:624)
        at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
2006-03-04 06:01:06,852 [Thread-38] INFO  ama.watchdog.MAgentWatchdog  - Triggering recovering 
action RESTART_PROCESS

2. various java (JVM) core dumps appear upon logging into the command line interface on the AON module.

3. show aon health and other commands that make calls to the Mbean server may be unresponsive or display the following:

aon-module> show aon health
Request Timeout

Condition

This occurred due to many events sent to the MBean server in a short period of time. For example, over 16 events per minute for many hours or days. Correcting the root cause of whatever is causing so many events to be sent to the Mbean is one way to prevent this watchdog case from occurring.

Workaround

None.

CSCek35365

Symptom

AON does not work when user tries to load schema with invalid path to imported or included schemas.

Condition

This occurs when the schema package contains schemas that has invalid path to imported or included schemas.

Workaround

Make sure all path to imported or included schemas are valid in the schema package.

CSCek35399

Symptom

AON network module shows as "Shutdown" when using the service-module AONS-Engine interface number status Cisco IOS command. However, when you session to the module, the AON CLI is still fully functional and the blade is not really shutdown.

Condition

This occurs when the service-module AONS-Engine interface number shutdown Cisco IOS command is used on the router hosting your AON network module.

Workarounds

Reload the AON network module and perform the following:

1. On the network module, enter the AON shutdown command and wait for the system to halt. This puts the module into the state the router expects it to be in.

2. On the router, enter the Cisco IOS service-module AONS-Engine interface number reset command to reset the AON network module.

Note When the module and the router come back up, the status is correct.

CSCek35429

Symptom

Request message do not reach its destination in a queue-to-queue based message interaction.

Condition

In a multi-blade virtual cluster (VC) setup, AON replyTo queues do not get equitably distributed among all the blades. It is possible that some of the blades may not acquire any AON replyTo queues.

Workaround

Configure the number of AON replyTo queues equal to or greater than twice the number of blades in the VC setup.

CSCek36038

Symptom

After upgrading AMC to a new Cisco-internal build of release 2.1, user attempted to perform a global deployment. The operation failed with the error: Node not active. In the amc.log file, the following messages were seen:

INFO [http7010-Processor23] AMC.Deploy.GlobalDeploymentRequestLogic Executing global deployment 
request, DR ID: (801) NOTICE [http7010-Processor23] AMC.Deploy.GlobalDeploymentRequestLogic 
Deploying global deployment request to all nodes using transactional, 2-phase deploy ... ERROR 
[http7010-Processor23] AMC.Deploy.GlobalDeploymentRequestLogic Exception while preparing nodes. 
Rolling back nodes. ERROR [http7010-Processor23] AMC.Deploy.GlobalDeploymentRequestLogic 
AMCException deploying global deployment request, DR ID: 801. Error message: node.not.activated 
INFO [http7010-Processor23] AMC.Deploy.GlobalDeploymentRequestLogic Value of Message Key: 
node.not.activated NOTICE [http7010-Processor23] AMC.Deploy.GlobalDeploymentRequestLogic 
Successfully changed deployment request state to 'Error' ERROR [http7010-Processor23] 
AMC.Deploy.DRAction DRAction: unable to deploy global DR

Condition

This occurs when the AMC was managing a network of 29 nodes. All were initially in the Registered state. The global deployment was attempted. It failed (correctly) with the "Node not active" message because it is required that at least one node be in the Active state in order to perform a global deployment. User then activated one of the nodes, and re-tried the deployment. It still failed with the same error.

Workaround

Activate THE FIRST NODE IN THE LIST on the "Activate/Deactivate Network Nodes" page. There is apparently an unexpected dependency within the AMC code on the position within the list.

CSCek36378

Symptom

Cookie headers in 302 direction response are not correctly handled by AON. The cookie headers are not forwarded to the directed URL or send to the client.

Conditions

It only happens for 302 redirection response containing cookie headers.

Workaround

None.

CSCek37187

Symptom

When an image upgrade is done, as designed, the AON optimization log level resets to default behavior. However; the running config shows the previously set non-default level.

Condition

This situation occurs only when an image upgrade is done.

Workaround

Reconfigure the desired log level after each upgrade.

CSCek37408

Symptom

AON stops picking message from JMS IN queue

Conditions

This happens in a two blade single node VC when client uses unlimited dynamic Reply Queues.

Workaround

Use Static Queues instead of dynamic queues.

CSCin98529

Symptom

SCAR file packaging causes error when the NativeVersion for the lib file (.so file) is not given. This has occurred on only one Windows XP PC and cannot be reproduced elsewhere.

Condition

This occurred when the nativeVersion is present in the bladelet-info.xml file, the SCAR file packaging looks for the lib file (.so file) with the appropriate version number and the packaging proceeds. When the nativeVersion is not given, an error occurs, and the SCAR file is not packaged.

Workaround

Try another Windows PC, or try using the nativeVersion tag in bladelet-info.xml.

CSCsd99036

Symptom

Multiple links are getting generated after the branch bladelet that is in a loop bladelet.

Conditions

During creation of a loop bladelet in a PEP having a branch bladelet, if one tries to add any bladelet after the branch, multiple links are getting generated after the branch bladelet. For any subsequent operations, the links keep on increasing. This happens when branch bladelet is the first bladelet added inside a loop bladelet.

Workaround

Save the PEP and open it again. All extra links wont be shown.

CSCsd99156

Symptom

When the flow has multiple SetDestination Bladelets (with XPath rules) or if there are Find Bladelets ahead of SetDestination Bladelet (with XPath Rules), SetDestination's XPath evaluation returns NULL for the first request message.

Workaround

Send another message.

CSCse35371

Symptom

If there is an existing or new sequence in the schema, the message log writing and viewing in AMC fails.

Workaround

For Oracle, do not change the schema after it is created using the scripts in appendix A. This information is found in "Create a Message Log Database" section of Chapter 4, Step 1, in the Cisco AON Installation and Administration Guide.

Note If changes are added, it may cause the message log feature to fail. For example, as reported in this defect, existing or new sequences to the schema can cause older version before 2.1.1 to fail.

CSCse41928

Symptom

Multiple modifications to JMS/SSL Property and Single deployment doesn't work correctly. The changes notification did not happen on the corresponding nodes. Only some of the modifications had done to the JMS/SSL property take effect and sometimes AON stops listening to the modified JMS queues.

Conditions

Make a series of changes to a JMS property and do a single deployment from AMC to nodes. The changes notification did not happen on the corresponding nodes.

Workaround

Either deploy one JMS property change at a time, or restart AON.

CSCse46613

Symptom

The following adapter exception is encountered in a two-node setup: No destination could be found

Conditions

This happens if the same reply queue is being used in the following scenario: JMS message is processed in a request-response flow by a JMS adapter on a AON Client Proxy and a JMS adapter on an AON Server Proxy, this is followed by an MQ message processed in a request-response flow by an MQ adapter on the AON Client Proxy and a JMS adapter on the AON Server Proxy.

Workaround

Use different reply queues for the JMS to JMS flow and the MQ to JMS flow.

CSCse46778

Symptom

In a two node reliable/ordered scenario some of the requests do not reach the end point (as a result the corresponding the responses are not received by the client) On SP, the following warning message appears on the log:

aons.mec.core %%FAILURE% %%received% 2006-06-08 03:25:22.963 %%source% null %%dest% 
http://httpserver.aontest.com:80/echo.php %%corrid % 

Conditions

Reliable/Ordered, two node, JMS/MQ at inbound on CP and http at outbound on SP. The end point should close the connection while AON writes a message to it.

Workaround

None

CSCse47151

Symptom

Execution of PEP is really slow to finish, at the same time when checking node events, will see Log bladelet have error exceptions.

Conditions

If the DB is down or if Msg Log Policy has incorrect information, AON cannot establish connection to the database. The Log bladelet, even in Asynchronous mode, will take a long time, ~30 seconds, to execute. During the execution of the PEP, it will pause in the Log bladelet until it times out before moving on to the next bladelet.

Workaround

Fix the incorrect information in the message log policy and deploy. And/Or bring up the database if DB is down.

CSCse55758

Symptom

When the uri is http:cisco.com/index.html, then instead of sending the data to http://cisco.com/index.html, the request is going to destination specified in "host" header field.

Conditions

This condition happens when uri is incorrect. Though http:www.cisco.com is a correct form of uri, as per Fastpath its incorrect. Hence Fastpath tries to reconstruct the url from the "host" header field and tries to connect to it.

Workaround

Currently there is no work around for it.

CSCse67323

Symptom

O/R message processing with EMS in a two-blade VC is very slow almost at the rate of 1 message every two minutes.

Conditions

EMS adapter at inbound in a two blade VC. A number of messages are deposited in the IN Q at once. First few messages get processed quickly, but later the message processing slows down considerably. Eventually the messages get processed. No related error/warning messages are in the log.

Workaround

None

CSCse76913

Symptom

On a CDP neighbor Cisco device, when the show cdp neighbors detail command is running, the Version field value is missing and appears as the following in the output:

Version :
, Version
, Version

Condition

Running an AON 8300 Series appliance (APL-AON-8340-K9 or APL-AON-8342-K9) with AON 2.1, AON 2.1.1 or AON 2.1.2 release.

Retrieving the AON software version from a neighboring network device or network management application

Note Running the show cdp neighbors detail command on the AON appliance to retrieve neighboring device version operates correctly and is unaffected by this problem.

Workaround

Get the AON version information:

3. Use the show version command to verify that the node is running new software version. The AON software version is the value of the "Global Software Version" field.

4. Use AMC to activate the node, then go to Network Node > Manage > Show to confirm that "AON SW Version" field reflects the AON software version.

CSCsf04907

Symptom

Bootloader configuration is lost and the Ethernet MAC address is reported as 00:0E:0C:6F:0F:E6.

Condition

When the user enters the log trace boot command from the AON CLI.

Workaround

To restore the bootloader configuration, simply reconfigure the bootloader and the configuration will be saved again. There is no workaround to restore the MAC address of the Ethernet interface.


Upgrade Instructions

For detailed instructions on upgrading to Cisco Application-Oriented Networking Version 2.2, see the following document:

Upgrading the Cisco Application-Oriented Networking Environment

Backward Compatibility


Note You must install the latest Cisco AON Management Console (AMC) release and the latest AON Development Studio (ADS) release on your node. Both these software versions must match for the node to work.


Cisco Application-Oriented Networking Version 2.2 is backward compatible with any node that is running Cisco AON Version 2.1, Cisco AON Version 2.1.1, and Cisco AON 2.1.2.

Related Documentation

The AON documentation set includes the following guides:

Upgrading the Cisco Application-Oriented Networking Environment—covers the upgrade procedure for the AON environment.

AON Installation and Administration Guidecovers the installation and administration of the AON Management Console and AON nodes.

AON Development Studio User Guidecovers the AON Development Studio, Bladelets, and PEP creation.

AON Programming Guidecovers the development of custom Bladelets, custom adapters, and other features related to extending AON functionality.

Obtaining Documentation

Cisco documentation and additional literature are available on Cisco.com. Cisco also provides several ways to obtain technical assistance and other technical resources. These sections explain how to obtain technical information from Cisco Systems.

Cisco.com

You can access the most current Cisco documentation at this URL:

http://www.cisco.com/univercd/home/home.htm

You can access the Cisco website at this URL:

http://www.cisco.com

You can access international Cisco websites at this URL:

http://www.cisco.com/public/countries_languages.shtml

Documentation DVD

Cisco documentation and additional literature are available in a Documentation DVD package, which may have shipped with your product. The Documentation DVD is updated regularly and may be more current than printed documentation. The Documentation DVD package is available as a single unit.

Registered Cisco.com users (Cisco direct customers) can order a Cisco Documentation DVD (product number DOC-DOCDVD=) from the Ordering tool or Cisco Marketplace.

Cisco Ordering tool:

http://www.cisco.com/en/US/partner/ordering/

Cisco Marketplace:

http://www.cisco.com/go/marketplace/

Ordering Documentation

You can find instructions for ordering documentation at this URL:

http://www.cisco.com/univercd/cc/td/doc/es_inpck/pdi.htm

You can order Cisco documentation in these ways:

Registered Cisco.com users (Cisco direct customers) can order Cisco product documentation from the Ordering tool:

http://www.cisco.com/en/US/partner/ordering/

Nonregistered Cisco.com users can order documentation through a local account representative by calling Cisco Systems Corporate Headquarters (California, USA) at 408 526-7208 or, elsewhere in North America, by calling 1 800 553-NETS (6387).

Documentation Feedback

You can send comments about technical documentation by using the embedded feedback form next to the document on Cisco.com or by writing to the following address:

Cisco Systems, Inc.
Attn: Customer Document Ordering
170 West Tasman Drive
San Jose, CA 95134-9883

We appreciate your comments.

Cisco Product Security Overview

Cisco provides a free online Security Vulnerability Policy portal at this URL:

http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html

From this site, you can perform these tasks:

Report security vulnerabilities in Cisco products.

Obtain assistance with security incidents that involve Cisco products.

Register to receive security information from Cisco.

A current list of security advisories and notices for Cisco products is available at this URL:

http://www.cisco.com/go/psirt

If you prefer to see advisories and notices as they are updated in real time, you can access a Product Security Incident Response Team Really Simple Syndication (PSIRT RSS) feed from this URL:

http://www.cisco.com/en/US/products/products_psirt_rss_feed.html

Reporting Security Problems in Cisco Products

Cisco is committed to delivering secure products. We test our products internally before we release them, and we strive to correct all vulnerabilities quickly. If you think that you might have identified a vulnerability in a Cisco product, contact PSIRT:

Emergencies —  security-alert@cisco.com

Nonemergencies —  psirt@cisco.com


Tip We encourage you to use Pretty Good Privacy (PGP) or a compatible product to encrypt any sensitive information that you send to Cisco. PSIRT can work from encrypted information that is compatible with PGP versions 2.x through 8.x.

Never use a revoked or an expired encryption key. The correct public key to use in your correspondence with PSIRT is the one that has the most recent creation date in this public key server list:

http://pgp.mit.edu:11371/pks/lookup?search=psirt%40cisco.com&op=index&exact=on


In an emergency, you can also reach PSIRT by telephone:

1 877 228-7302

1 408 525-6532

Obtaining Technical Assistance

For all customers, partners, resellers, and distributors who hold valid Cisco service contracts, Cisco Technical Support provides 24-hour-a-day, award-winning technical assistance. The Cisco Technical Support Website on Cisco.com features extensive online support resources. In addition, Cisco Technical Assistance Center (TAC) engineers provide telephone support. If you do not hold a valid Cisco service contract, contact your reseller.

Cisco Technical Support Website

The Cisco Technical Support Website provides online documents and tools for troubleshooting and resolving technical issues with Cisco products and technologies. The website is available 24 hours a day, 365 days a year, at this URL:

http://www.cisco.com/techsupport

Access to all tools on the Cisco Technical Support Website requires a Cisco.com user ID and password. If you have a valid service contract but do not have a user ID or password, you can register at this URL:

http://tools.cisco.com/RPF/register/register.do


Note Use the Cisco Product Identification (CPI) tool to locate your product serial number before submitting a web or phone request for service. You can access the CPI tool from the Cisco Technical Support Website by clicking the Tools & Resources link under Documentation & Tools. Choose Cisco Product Identification Tool from the Alphabetical Index drop-down list, or click the Cisco Product Identification Tool link under Alerts & RMAs. The CPI tool offers three search options: by product ID or model name; by tree view; or for certain products, by copying and pasting show command output. Search results show an illustration of your product with the serial number label location highlighted. Locate the serial number label on your product and record the information before placing a service call.


Submitting a Service Request

Using the online TAC Service Request Tool is the fastest way to open S3 and S4 service requests. (S3 and S4 service requests are those in which your network is minimally impaired or for which you require product information.) After you describe your situation, the TAC Service Request Tool provides recommended solutions. If your issue is not resolved using the recommended resources, your service request is assigned to a Cisco TAC engineer. The TAC Service Request Tool is located at this URL:

http://www.cisco.com/techsupport/servicerequest

For S1 or S2 service requests or if you do not have Internet access, contact the Cisco TAC by telephone. (S1 or S2 service requests are those in which your production network is down or severely degraded.) Cisco TAC engineers are assigned immediately to S1 and S2 service requests to help keep your business operations running smoothly.

To open a service request by telephone, use one of the following numbers:

Asia-Pacific: +61 2 8446 7411 (Australia: 1 800 805 227)
EMEA: +32 2 704 55 55
USA: 1 800 553-2447

For a complete list of Cisco TAC contacts, go to this URL:

http://www.cisco.com/techsupport/contacts

Definitions of Service Request Severity

To ensure that all service requests are reported in a standard format, Cisco has established severity definitions.

Severity 1 (S1)—Your network is "down," or there is a critical impact to your business operations. You and Cisco will commit all necessary resources around the clock to resolve the situation.

Severity 2 (S2)—Operation of an existing network is severely degraded, or significant aspects of your business operation are negatively affected by inadequate performance of Cisco products. You and Cisco will commit full-time resources during normal business hours to resolve the situation.

Severity 3 (S3)—Operational performance of your network is impaired, but most business operations remain functional. You and Cisco will commit resources during normal business hours to restore service to satisfactory levels.

Severity 4 (S4)—You require information or assistance with Cisco product capabilities, installation, or configuration. There is little or no effect on your business operations.

Obtaining Additional Publications and Information

Information about Cisco products, technologies, and network solutions is available from various online and printed sources.

Cisco Marketplace provides a variety of Cisco books, reference guides, and logo merchandise. Visit Cisco Marketplace, the company store, at this URL:

http://www.cisco.com/go/marketplace/

Cisco Press publishes a wide range of general networking, training and certification titles. Both new and experienced users will benefit from these publications. For current Cisco Press titles and other information, go to Cisco Press at this URL:

http://www.ciscopress.com

Packet magazine is the Cisco Systems technical user magazine for maximizing Internet and networking investments. Each quarter, Packet delivers coverage of the latest industry trends, technology breakthroughs, and Cisco products and solutions, as well as network deployment and troubleshooting tips, configuration examples, customer case studies, certification and training information, and links to scores of in-depth online resources. You can access Packet magazine at this URL:

http://www.cisco.com/packet

iQ Magazine is the quarterly publication from Cisco Systems designed to help growing companies learn how they can use technology to increase revenue, streamline their business, and expand services. The publication identifies the challenges facing these companies and the technologies to help solve them, using real-world case studies and business strategies to help readers make sound technology investment decisions. You can access iQ Magazine at this URL:

http://www.cisco.com/go/iqmagazine

Internet Protocol Journal is a quarterly journal published by Cisco Systems for engineering professionals involved in designing, developing, and operating public and private internets and intranets. You can access the Internet Protocol Journal at this URL:

http://www.cisco.com/ipj

World-class networking training is available from Cisco. You can view current offerings at this URL:

http://www.cisco.com/en/US/learning/index.html