Field Notice: AIM-CUE With 512MB CF Storage Capacity May Crash

February 16, 2005

---

Products Affected

Product	Comments
AIM-CUE	Equipped with 512 Megabyte of memory.

Problem Description

The Cisco Unity Express AIM-CUE module equipped with 512MB of Compact Flash may crash if the voice mail storage capacity is completely used up. This is expected to occur when recording time exceeds 6 hours. Previously the product had been specified to support eight hours of voice mail storage.

Background

The AIM-CUE module began shipping with 512 Megabyte of Compact Flash (CF) in March 2004. In December 2004 the default Compact Flash capacity was increased to 1 Gigabyte and all AIM-CUE shipping after this date have 1 Gigabyte of Compact Flash. It was recently discovered that beginning with Cisco Unity Express release 1.1, total storage requirements for product operation and voice mail message storage greater than six hours exceeds the 512MB CF capacity. Releases that are subject to this caveat are 1.1, 2.0 and all subsequent releases for the AIM-CUE module.

This field notice recommends that the Message Storage Capacity be limited to either 275 or 300 minutes depending on the software release loaded on the AIM-CUE module. However, exceeding these limits does not always result in a crash of the AIM-CUE module. There are other factors that also affect whether the AIM-CUE will crash; these factors include the size of the log files, amount and length of saved voice prompts, and the number of configured users and mail boxes. It is possible to exceed the recommended 275 or 300 minute storage capacity without adverse effects.

Note: This issue does not affect the NM-CUE as it is equipped with a hard drive and storage capacity is not an issue.

Problem Symptoms

The symptom is the AIM-CUE will crash. The console may display the messages shown below. Bold text highlights the text strings that appear as a result of exceeding the Compact Flash capacity. At the time of the crash, on the CUE console may display messages as follows:

Jan 13 21:58:28 localhost java: CRASH dbclient database mgmt **** Postgres database is corrupt. Forcing shutdown. **** 

Or

Jan 27 13:54:00 localhost postgres: shutdown subprocess : CRASH sql sql sql FATAL 2: write of log file 0, segment 13, offset 1695744 failed: Bad file descriptor 

As the system boots up, messages such as the following may be visible:

STARTED: Flash wear monitor 
write_stats_proc_file: init, 4 
awk: cmd. line:1: (FILENAME=- FNR=348) fatal: print to "standard output" 
failed (No space left on device) 
awk: cmd. line:2: (FILENAME=/tmp/timesort FNR=164) warning: error writing 
standard output (No space left on device) Sun Feb 10 00:00:00 EST 1980 
/bin/cat: write error: (No space left on device) 

********** rc.aesop **************** 
/bin/runrecovery.sh: line 58: cannot create temp file for here document: 
(No space left on device)

Workaround/Solution

Solution:

Replace the AIM-CUE via the return materials authorization (RMA) process. The default compact flash on the AIM-CUE was increased in December 2004. All units shipped since December 2004 are equipped with 1 Gigabyte of Compact Flash and are not subject to this limitation.

Cisco recommends that back ups of the configuration and database be completed nightly.

Replacement Procedure:

To recover a system with a corrupt database, the recommended way is to:

1. Replace AIM-CUE with a new AIM-CUE that has the 1 GB Compact Flash card. This can be obtained by contacting the TAC.

2. Access the AIM-CUE via the CLI.

3. Examine the software version and license information

4. Load the CUE system software, if necessary.

5. Load the correct CUE license files, if necessary.

6. Restore from backup.

Note: If no replacement AIM-CUE is available at the time and the database is corrupt, you will have to follow steps 2-6 as outlined above. Once a replacement card is available, a backup of the system should be taken and these steps repeated starting with step 1.

Details:

1. Once a replacement AIM-CUE card is ready, it needs to be installed. To replace the AIM card, the following link should be used: Installing, Replacing, and Upgrading Components in Cisco Modular Access Routers and Integrated Services Routers

2. After the hardware is physically installed, access the AIM-CUE CLI by following these steps in theEntering the Command Environment section.

3. Examine the software version and license information. From the CUE CLI, complete the post-installation interview. The NTP server(s), DNS server(s), time zone, a new administrator user ID and password must be entered.

   Then issue a show software versions command.

   Note: the version of the Core software and the installed language.

   If either of these is not correct, the system software will need to be re-installed. If you intend to restore from backup, you should install the same software version or later that the backup was taken with. In other words, if you are not sure if the previous version was 1.2.1 or 2.0.1, which was released mid-December 2004, load 2.0.1 to ensure a proper restore.

   Finally issue a show software licenses command and examine the application mode, either CCM for Cisco CallManager or CCME for Cisco CallManager Express and number of ports to see if this is correct.

   For example:

   aim-cue> show software versions

   Installed Packages:

   • Core 1.1.2

   • Auto Attendant 1.1.2

   • Voice Mail 1.1.2

   • Boot Loader 1.0.17

   Installed Languages:

   • US English 1.1.2

   aim-cue> show software licenses

   Core:

   • application mode: CCME

   • total usable system ports: 4

   Voicemail/Auto Attendant:

   • max system mailbox capacity time: 840

   • max general delivery mailboxes: 15

   • max personal mailboxes: 50

   Languages:

   • max installed languages: 1

   • max enabled languages: 1

4. Load CUE system software, if necessary, as determined in step 3.

   To install the 2.0.1 system software, follow the instructions in the Installing a Software Image Using Boot Helper section.

   To install the 1.1.2 system software, follow the instructions in the Upgrading to a New Software Image section.

5. Install the proper license, if necessary, as determined in step 3.

   To install a 2.0.1 license file, follow the instructions in the Installing a New Software License File section.

   To install a 1.1.2 license file, follow the instructions in the Installing a New Software License File section.

6. Restore from backup from the CUE CLI. This procedure hasn't changed between 1.1.X and 2.0.1. See the Restoring Files section.

   For more information and a summary of CUE documentation see the Cisco Unity Express Documentation Roadmap.

Prevention:

Whenever possible, the system administrator must limit the total recording time to 300 minutes for releases 1.1(2) and 2.0(1) or 275 mintues for all 2.1 and later releases to prevent a system crash from occurring. This can be accomplished via the GUI or the CLI.

GUI Method:

Using the GUI, follow these steps:

1. Use a browser to http:///

2. Enter a username and password. The user must have administrative rights to the system

   

3. Select Defaults > Voice Mail

   

4. Change the Maximum voice message store from 480 minutes to 300 for 1.1(2) or 2.0 releases or 275 for 2.1 release minutes.

   

   If this step fails, see step 8.

5. Press Apply.

6. The Saved successfully dialog box should appear.

   

7. Press Ok

8. If the "Total message time (seconds)" value is greater than 18000 for 1.1(2) or 2.0 releases or 16500 for the 2.1 releases seconds, divide this value by 60, round up to the nearest minute and add one minute. Use this value to enter for the change in step 4, above. See screen display below. In this instance, the user has saved only 401 seconds of messages and may be at risk to fill up their CF depending on other factors such as size of log files, number of configured users and ports and amount of saved voice prompts.

   

For example, if the "Total message time" indicated is 20011 seconds, instead of 401, as in the above example, perform this calculation:

20011 seconds divided by 60 sec/min = 333.52 minutes

Round up to nearest minute: 334 minutes of message time

Add 1 minute to the the number of minutes: 335 minutes.

Use 335 minutes in step 4, above. Perform this step only if the Total mesage time exceeds 18000 for 1.1(2) or 2.0 releases or 16500 for 2.1 and later releases.

An alternative to performing the above calculation is to have users listen to, process and delete enough messages to reduce the Total message time to under 18000 seconds for 1.1(2) and 2.0 releases or 16500 for the 2.1 releases. Then step 4 may be performed using the recommended value of 300 or 275.

CLI Method:

Using the CLI, use the command voicemail capacity time 300|275 to limit the total message capacity to 5 hours for 1.1(2) or 2.0 releases or to 4.6 hours for 2.1 and later releases.

Follow these steps:

1. Log on to the Cisco IOS router that contains the AIM-CUE via telnet or a console connection.

2. Enter enable mode. Type enable and press Enter

3. Connect to AIM-CUE command line by typing: service-module service-engine 0/0 session and press Enter

4. Press Enter to get prompt. If you get a message that the connection was refused, clear the line. For example:

   Router-Name#service-module service-Engine 0/0 session 
   Trying IP.ADD.RE.SS, 2025 ... 
   % Connection refused by remote host 
   
   Router-Name#clear line 25 (Note: that "25" comes from the line above. See bold text. Substitute "25" with the last 2 digits of the port shown when the session is attempted.) 
   [confirm] 
   [OK] 
   Router-Name#service-module service-Engine 0/0 session
   Trying IP.ADD.RE.SS, 2025 ... Open 
   CUE-Name> 
   

5. Enter configuration mode by typing config t and press Enter.

6. Set voice mail maximum capacity by typing voicemail capacity time 300|275 and press enter. Use 300 for AIM-CUE releases 1.2(2) and all 2.0 releases. Use 275 for all 2.1 and later releases. If this command does not execute, see step 6a. Otherwise, go to step 7.

   The AIM-CUE has already used more than 300 minutes and the "message time used" value is greater than 18000 seconds for releases 1.1(2) and all 2.0 releases or 16500 seconds for all 2.1 releases. Use the command show voice usage to see the total consumed space. Note at the parameter "message time used (seconds)" and divide by 60 to get minutes, round up to the nearest minute and add 1 miunte. Use this new value instead of "300" or "275" in step 6. If "message time used" is "20011" perform the following calculation:

   20011 seconds divided by 60 sec/min = 333.52 minutes

   Round up to nearest minute: 334 minutes of message time

   Add 1 minute to the the number of minutes: 335 minutes.

   Use 335 minutes in step 6, above. Perform step 6a ONLY if the "message time used" exceeds 18000 seconds in 1.1(2) or any 2.0 release or 16500 in any 2.1 release.

7. Exit configuration mode by typing exit and press Enter.

8. Verify configuration by typing show voicemail limits and press Enter.

9. On the returned screen verify that the System Capacity (minutes): value is set to the value you entered in step 6 or 6a.

10. Save the configuration by typing write memory and press Enter. The write-to-memory activity takes several seconds.

11. Exit the AIM-CUE command line by typing exit and press Enter.

12. Exit the router command line by typing exit and press Enter.

See the example session below.

router_name>enable 
Password: 
router_name#service-module service-engine 0/0 session 
Trying IP.ADD.RE.SS, 2026 ... Open 
se-IP-ADD-RE-SS> 
se-IP-ADD-RE-SS> config t 
Enter configuration commands, one per line. End with CNTL/Z. 
se-IP-ADD-RE-SS(config)> voicemail capacity time 300 
se-IP-ADD-RE-SS(config)> exit 
se-IP-ADD-RE-SS> show voicemail limits 
Default Mailbox Size (seconds): 420 
Default Caller Message Size (seconds): 60 
Maximum Recording Size (seconds): 900 
Default Message Age (days): 30 
System Capacity (minutes): 300 <<--- (This should be 300 or value entered from step 6a) 
Default Prompt Language: en_US 
Operator Telephone: 0 
Maximum Broadcast Size (seconds): 300 
Broadcast Message Age (days): 30 
Broadcast Message MWI: disabled 
Play Caller Id: disabled 
se-IP-ADD-RE-SS> write mem 
se-IP-ADD-RE-SS> exit 
[Connection to IP.ADD.RE.SS closed by foreign host] 
router_name#exit

Cisco recommends that back ups of the configuration and database be completed nightly.

Recovery:

If the system storage space has already been filled and the system has crashed, during the bootup sequence, the following option may be presented:

IMPORTANT:: 
IMPORTANT:: **************************************************************** 
IMPORTANT:: The system has detected that the database is corrupted beyond 
IMPORTANT:: repair and will need to be deleted. This means that the system 
IMPORTANT:: will loose the following information: 
IMPORTANT:: a. voicemail messages 
IMPORTANT:: b. user mailbox information 
IMPORTANT:: The system can be recovered completely (including the above 
IMPORTANT:: information) by doing a restore. 
IMPORTANT:: 
IMPORTANT:: **************************************************************** 
IMPORTANT:: 

Do you want to clean the database (y,n)? n

By entering "n", the system will attempt to repair the database. It is possible, though, that the system will still not work properly. If the system reboots again and does not recover, then the voicemail messages and configuration data are lost and will have to be restored from the last successful backup.

If there is a current back up stored, the back up may be restored to the replacement AIM-CUE after it is installed AND the AIM-CUE has the same software loaded as the failed AIM-CUE. Follow the directions in the Cisco Unity Express Installation Guide.

If a back up not available, install the AIM-CUE and rebuild configuration manually using the directions in the Cisco Unity Express Installation Guide.

Once the AIM-CUE with the 1 GB Compact Flash card has been installed and the data and configuration restored, the default maximum voicemail capacity will be 840 minutes. If this capacity was reduced per instructions in the Prevention section of this field notice, the same procedure should be used to change the voicemail capacity to 840 minutes.

DDTS

To follow the bug ID link below and see detailed bug information, you must be a registered user and you must be logged in.

DDTS	Description
CSCeg77690 (registered customers only)	CUE reloads at CRASH dbclient database error.
CSCeh02389 (registered customers only)	Insufficent flash memory error when upgrading to Unity Express 2.0

How To Identify Hardware Levels

To verify the Compact Flash size you must examine the maximum message space parameter from either the GUI or the CLI.

If the maximum message space indicates 480 minutes you have a 512 Megabyte Compact Flash on the AIM-CUE Module.

If the maximum message space indicates 840 minutes you have a 1 Gigabyte Compact Flash on the AIM-CUE.

Using the GUI:

From the CUE GUI, you can verify the "Maximum message space" size from the Help > About screen.

Using the CLI:

Use the command show software license to verify Compact Flash size.

If the display indicates 480 minutes you have a 512 Megabyte Compact Flash on the AIM-CUE Module.

If the display indicates 840 minutes you have a 1 Gigabyte Compact Flash on the AIM-CUE.

You may also use the show diag command to view the 800 assembly number.

Service Engine AIM: 0AIM Module in slot: 0 
Hardware Revision : 1.0 
Top Assy. Part Number : 800-20063-02 
Board Revision : B0

All AIM-CUE modules that show 800-20063-02 are equipped with 1 Gigabyte Compact Flash.

AIM-CUE modules that display 800-20063-01 were originally equipped with 512 Megabyte Compact Flash. These may have been upgraded to 1 Gigabyte Compact Flash at a later date. Use the show software license command to verify the size of the Compact Flash.

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email
• By telephone

Receive Email Notification For New Field Notices

Product Alert Tool - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.

---