Guest

Cisco Unity Connection

Unity Connection Replication Issues After Security Password Reset

  • Viewing Options

  • PDF (12.7 KB)
  • Feedback

Document ID: 117623

Updated: Apr 21, 2014

Contributed by Scott Hills, Cisco TAC Engineer.

   Print

Introduction

This document describes a problem encountered in Unity Connection where replication between Publisher and Subscriber might be interrupted after you reset the Security password and offers a solution to the problem.

There are two defects are related to this document: Cisco bug IDs CSCth87452 and CSCua09290.

Problem

After you reset the Security Password in order to address an issue, replication does not set up properly.

You might see this output if you enter this command from Publisher: utils dbreplication runtimestate.

                              PING
SERVER-NAME  IP ADDRESS     (msec)
-----------   ------------    ------
Cisco1       192.168.2.50     0.038
Cisco2       192.168.2.51     0.273

       CDR Server        REPL.   DBver&    REPL.   REPLICATION SETUP
RPC?  (ID) & STATUS   QUEUE   TABLES    LOOP?   (RTMT) & details
----    --------------    -----    -------   -----    -----------------
Yes    (2)  Connected   0       match     Yes     (2) PUB Setup Completed
Yes    (3)  Connected   0       match    N/A     (3) Not Setup

Notice that the REPLICATION SETUP(RTMT) & details column indicates issues because it does not show both (2). Also if you enter the file view activelog /cm/log/informix/ccm.log, the output might show:

10:04:02  Password Validation for user [dbmon] failed!
10:04:02  Check for password aging/account lock-out.
10:04:02  listener-thread: err = -952: oserr = 0: errstr =
dbmon@Cisco2.na.cisco.corp[Cisco2]: User (dbmon@Cisco2.na.cisco.corp[Cisco2])'s
password is not correct for the database server.

10:04:06  Password Validation for user [dbauditlog] failed!
10:04:06  Check for password aging/account lock-out.
10:04:06  listener-thread: err = -952: oserr = 0: errstr =
dbauditlog@Cisco2.na.cisco.corp[Cisco2]: User (dbauditlog@Cisco2.na.cisco.corp[Cisco2])'s
password is not correct for the database server.

10:04:06  Password Validation for user [dbdbl] failed!
10:04:06  Check for password aging/account lock-out.
10:04:06  listener-thread: err = -952: oserr = 0: errstr =
dbdbl@Cisco2.na.cisco.corp[Cisco2]: User (dbdbl@Cisco2.na.cisco.corp[Cisco2])'s
password is not correct for the database server.

10:04:07  Password Validation for user [dblpm] failed!
10:04:07  Check for password aging/account lock-out.
10:04:07  listener-thread: err = -952: oserr = 0: errstr =
dblpm@Cisco2.na.cisco.corp[Cisco2]: User (dblpm@Cisco2.na.cisco.corp[Cisco2])'s
password is not correct for the database server.

Solution

Initially, you could see that database (DB) Replication iss not established between Publisher and Subscriber, and this behavior matches the Cisco bug IDs CSCth87452 and CSCua09290. Unfortunately, since root access is required as part of the solution, the Cisco Technical Assistance Center (TAC) is needed in order to solve the problem.

Complete these steps in order to fix the problem:

  1. In order to stop A Cisco DB, enter this command:

    utils service stop A Cisco DB

  2. Log in to the server with a remote support account and upload the Python script.

    Note: TAC is needed for the next steps.



    Obtain the SyncInformixPasswordsWithSystem.py and put it on the Secure FTP (SFTP) server.

    In order to copy the file from SFTP to Unity Connection, enter these commands :

    cd /tmp <enter>

    sftp username@IPAdressSFTPServer <enter>

    get SyncInformixPasswordWithSystem.py <enter>

    bye <enter>


  3. In order to change the access permissions of the file, enter this command:  

     chmod 644 /tmp/SyncInformixPasswordsWithSystem.py


  4. In order to run the script, execute this command:

     python /tmp/SyncInformixPasswordsWithSystem.py


    The output should be similar to this:

    [root@nw084b-181 /]# python /tmp/SyncInformixPasswordsWithSystem.py
    Changing password for user informix.
    passwd: all authentication tokens updated successfully.
    Changing password for user database.
    passwd: all authentication tokens updated successfully.
    Changing password for user dbuser.
    passwd: all authentication tokens updated successfully.
    Changing password for user dbccm.


  5. Once Step 4 completes, start the DB from admin CLI:

     utils service start A Cisco DB


  6. In order to see if you still have an issue, enter the utils dbreplication runtimestate command:


    10:04:02  Password Validation for user [dbmon] failed!
    10:04:02  Check for password aging/account lock-out.
    10:04:02  listener-thread: err = -952: oserr = 0: errstr =
    dbmon@Cisco2.na.cisco.corp[Cisco2]: User (dbmon@Cisco2.na.cisco.corp[Cisco2])'s
    password is not correct for the database server.

    10:04:06  Password Validation for user [dbauditlog] failed!
    10:04:06  Check for password aging/account lock-out.
    10:04:06  listener-thread: err = -952: oserr = 0: errstr =
    dbauditlog@Cisco2.na.cisco.corp[Cisco2]: User (dbauditlog@Cisco2.na.cisco.corp[Cisco2])'s
    password is not correct for the database server.

    10:04:06  Password Validation for user [dbdbl] failed!
    10:04:06  Check for password aging/account lock-out.
    10:04:06  listener-thread: err = -952: oserr = 0: errstr =
    dbdbl@Cisco2.na.cisco.corp[Cisco2]: User (dbdbl@Cisco2.na.cisco.corp[Cisco2])'s
    password is not correct for the database server.

    10:04:07  Password Validation for user [dblpm] failed!
    10:04:07  Check for password aging/account lock-out.
    10:04:07  listener-thread: err = -952: oserr = 0: errstr =
    dblpm@Cisco2.na.cisco.corp[Cisco2]: User (dblpm@Cisco2.na.cisco.corp[Cisco2])'s
    password is not correct for the database server.


  7. In order to set a new Security password and reboot both servers, enter the set user password security command on Pubisher and Subscriber. In some situations, this might be the final step to solve your problem. However, in some situations, replication might still be broken. In this case, continue to Step 8.

    Note: Allow enough time for replication to set up, probably a minimum of 30 minutes, before you decide that it has not set up properly. You can also check ccm.log for a status update if you enter this command: file view activelog /cm/log/informix/ccm.log.



  8. In order to check if replication is still broken, enter the utils dbreplication runtimestate command: 

                                  PING
    SERVER-NAME  IP ADDRESS     (msec)
    -----------   ------------    ------
    Cisco1       192.168.2.50     0.038
    Cisco2       192.168.2.51     0.273

           CDR Server        REPL.   DBver&    REPL.   REPLICATION SETUP
    RPC?  (ID) & STATUS   QUEUE   TABLES    LOOP?   (RTMT) & details
    ----    --------------    -----    -------   -----    -----------------
    Yes    (2)  Connected   0       match     Yes     (2) PUB Setup Completed
    Yes    (3)  Connected   0       match    N/A     (3) Not Setup


  9. Enter the file view activelog /cm/log/informix/ccm.log command:

    10:41:24  CDR CDRD_1: transaction aborted (All rows in a transaction defined with
    row scope were rejected) with sql error 0 isam error 0.
    10:41:24  CDR CDRD_1: failed rows spooled to
    file /tmp/ris.g_Cisco1_ccm9_1_1_20000_5.g_Cisco2_ccm9_1_1_20000_5.D_1.140322_10:41:24.3
    10:41:24  CDR CDRD_1: failed transaction spooled to
    file /tmp/ats.g_Cisco1_ccm9_1_1_20000_5.g_Cisco2_ccm9_1_1_20000_5.D_1.140322_10:41:24.4
    10:41:24 CiscoAlarmStart:Cisco1_ccm9_1_1_20000_5:ATTENTION:48:CDR: ATS and/or RIS
    files spooled to disk.
    :/tmp/ris.g_Cisco1_ccm9_1_1_20000_5.g_Cisco2_ccm9_1_1_20000_5.D_1.140322_10:41:24.3|
    /tmp/ats.g_Cisco1_ccm9_1_1_20000_5.g_Cisco2_ccm9_1_1_20000_5.D_1.140322_10:41:24.4:CiscoAlarmEnd


  10. In order to try to reset the Cisco CallManager (CCM) DB replication on the Unified Communications (UC) cluster, complete these steps:

    1. Enter the utils dbreplication stop command on both the Publisher and Subscriber servers.
    2. Enter the utils dbreplication dropadmindb command on both Publisher and Subscriber.
    3. Enter the Utils dbreplication reset all command on Publisher and the rebooted Subscriber. Publisher does not need to be rebooted.


  11. Finally, you can enter the utils dbreplication runtimestate command over a preiod of time in order to monitor the set up of replication status.

    Here is an example of what you might see:

    DB and Replication Services: ALL RUNNING

    DB CLI Status: No other dbreplication CLI is running...

    Cluster Replication State: BATCHING SYNC Requests from nodes at: 2014-03-22-11-27
         Sync Request Progress: Received 1 node requests for DB sync
         Sync Request Errors: NO ERRORS

    DB Version: ccm9_1_1_20000_5
    Repltimeout set to: 300s
    PROCESS option set to: 1

    Cluster Detailed View from Cisco1 (2 Servers):

                                PING
    SERVER-NAME  IP ADDRESS     (msec)
    -----------   ------------    ------
    Cisco1       192.168.2.50     0.028
    Cisco2       192.168.2.51     0.221

           CDR Server        REPL.   DBver&    REPL.   REPLICATION SETUP
    RPC?  (ID) & STATUS   QUEUE   TABLES    LOOP?   (RTMT) & details
    ----    --------------    -----    -------   -----    -----------------
    Yes    (2)  Connected   0       match     Yes     (2) PUB Batching Sub Req's
    Yes    (3)  Connected   0       match    N/A     (0) Setup Requested


    Here is what you might see if you enter the command a short time later:

    Cluster Replication State: BROADCAST SYNC Started on 1 server(s) at: 2014-03-22-11-32
         Processing Table: typeconfiginputdata
         Sync Progress: 275 tables sync'ed out of 603
         Sync Errors: NO ERRORS

    DB Version: ccm9_1_1_20000_5
    Repltimeout set to: 300s
    PROCESS option set to: 1

    Cluster Detailed View from Cisco1 (2 Servers):

                                PING
    SERVER-NAME  IP ADDRESS     (msec)
    -----------   ------------    ------
    Cisco1       192.168.2.50     0.031
    Cisco2       192.168.2.51     0.210

           CDR Server        REPL.   DBver&    REPL.   REPLICATION SETUP
    RPC?  (ID) & STATUS   QUEUE   TABLES    LOOP?   (RTMT) & details
    ----    --------------    -----    -------   -----    -----------------
    Yes    (2)  Connected   0       match     Yes     (2) PUB Setting Subs
    Yes    (3)  Connected   0       match    N/A     (0) Setup in Progress


    Here is what you might see if you enter the command a short time later:

    DB and Replication Services: ALL RUNNING

    DB CLI Status: No other dbreplication CLI is running...

    Cluster Replication State: BROADCAST SYNC Completed on 1 servers at: 2014-03-22-11-33
         Last Sync Result: SYNC COMPLETED  603 tables sync'ed out of 603
         Sync Errors: NO ERRORS

    DB Version: ccm9_1_1_20000_5
    Repltimeout set to: 300s
    PROCESS option set to: 1

    Cluster Detailed View from Cisco1 (2 Servers):

                                PING
    SERVER-NAME  IP ADDRESS     (msec)
    -----------   ------------    ------
    Cisco2       192.168.2.51     0.0298
    Cisco1       192.168.2.50     0.050

           CDR Server        REPL.   DBver&    REPL.   REPLICATION SETUP
    RPC?  (ID) & STATUS   QUEUE   TABLES    LOOP?   (RTMT) & details
    ----    --------------    -----    -------   -----    -----------------
    Yes    (3)  Connected   0       match     Yes     (2) Setup Completed
    Yes    (2)  Connected   0       match    Yes     (2) PUB Setup Completed



    At this point, you can see that replication is established because the Replication table shows 2.

Related Information

Updated: Apr 21, 2014
Document ID: 117623