Mailing List Archive

unityconnection db replication issue
hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html


Regards


Naray
Re: unityconnection db replication issue [ In reply to ]
Hi,

Can you check whether all your NTP servers are reachable and accessible. Use a stratum 1 NTP server on your publisher and try again.

Thanks!
Jinto.

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: cisco-voip <cisco-voip-bounces@puck.nether.net> on behalf of naresh rathore <nareh84@hotmail.com>
Sent: Monday, November 25, 2019 8:11:12 PM
To: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: [cisco-voip] unityconnection db replication issue

hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html


Regards


Naray
Re: unityconnection db replication issue [ In reply to ]
hi Jinto


Current ntp status


ntpd (pid 23480) is running...

remote refid st t when poll reach delay offset jitter
==============================================================================
*172.19.30.18 <*> 4 u 597 1024 377 1.537 -5.158 6.202


synchronised to NTP server (172.19.30.18) at stratum 5
time correct to within 84 ms
polling server every 1024 s

Current time in UTC is : Tue Nov 26 01:18:32 UTC 2019
Current time in Australia/Sydney is : Tue Nov 26 12:18:32 AEDT 2019


Regards


Naray
________________________________
From: Jinto Alakkal Kunjumon <jalakkal@uoguelph.ca>
Sent: Tuesday, November 26, 2019 6:17 AM
To: naresh rathore <nareh84@hotmail.com>; cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: unityconnection db replication issue

Hi,

Can you check whether all your NTP servers are reachable and accessible. Use a stratum 1 NTP server on your publisher and try again.

Thanks!
Jinto.

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: cisco-voip <cisco-voip-bounces@puck.nether.net> on behalf of naresh rathore <nareh84@hotmail.com>
Sent: Monday, November 25, 2019 8:11:12 PM
To: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: [cisco-voip] unityconnection db replication issue

hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html


Regards


Naray
Re: unityconnection db replication issue [ In reply to ]
Hi Naray,

I see the stratum to be 5 in your output which is not good, Cisco recommends stratum 3 or lower for your NTP server, on publisher try a stratum 1 NTP server.

Google have stratum one NTP server, if your unity connection have access to internet.

Let me know how it goes.

Thanks!
Jinto.

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: naresh rathore <nareh84@hotmail.com>
Sent: Monday, November 25, 2019 8:20:30 PM
To: Jinto Alakkal Kunjumon <jalakkal@uoguelph.ca>; cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: unityconnection db replication issue


hi Jinto


Current ntp status


ntpd (pid 23480) is running...

remote refid st t when poll reach delay offset jitter
==============================================================================
*172.19.30.18 <*> 4 u 597 1024 377 1.537 -5.158 6.202


synchronised to NTP server (172.19.30.18) at stratum 5
time correct to within 84 ms
polling server every 1024 s

Current time in UTC is : Tue Nov 26 01:18:32 UTC 2019
Current time in Australia/Sydney is : Tue Nov 26 12:18:32 AEDT 2019


Regards


Naray
________________________________
From: Jinto Alakkal Kunjumon <jalakkal@uoguelph.ca>
Sent: Tuesday, November 26, 2019 6:17 AM
To: naresh rathore <nareh84@hotmail.com>; cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: unityconnection db replication issue

Hi,

Can you check whether all your NTP servers are reachable and accessible. Use a stratum 1 NTP server on your publisher and try again.

Thanks!
Jinto.

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: cisco-voip <cisco-voip-bounces@puck.nether.net> on behalf of naresh rathore <nareh84@hotmail.com>
Sent: Monday, November 25, 2019 8:11:12 PM
To: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: [cisco-voip] unityconnection db replication issue

hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html


Regards


Naray
Re: unityconnection db replication issue [ In reply to ]
Run “ utils diagnose test “ from the primary CLI; everything pass?

Sent from my iPhone

On Nov 25, 2019, at 20:11, naresh rathore <nareh84@hotmail.com> wrote:

?
hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cisco.com%2Fc%2Fen%2Fus%2Fsupport%2Fdocs%2Funified-communications%2Funity-connection%2F116942-technote-uc-00.html&data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962950398&sdata=oxx23Q2W6Iz%2F48d663ONW2S0IDiMnC94%2FfhThlxxjJM%3D&reserved=0>


Regards


Naray


_______________________________________________
cisco-voip mailing list
cisco-voip@puck.nether.net
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Fcisco-voip&amp;data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962970408&amp;sdata=FLTmpeiinFcfYQX4ANnxezVeYeYYCuNfKBPLSawCMFo%3D&amp;reserved=0
Re: unityconnection db replication issue [ In reply to ]
hi Ryan



following is the output


admin:utils diagnose test

Log file: platform/log/diag1.log

Starting diagnostic test(s)
===========================
test - disk_space : Passed (available: 17668 MB, used: 10920 MB)
skip - disk_files : This module must be run directly and off hours
test - service_manager : Passed
test - tomcat : Passed
test - tomcat_deadlocks : Passed
test - tomcat_keystore : Passed
test - tomcat_connectors : Passed
test - tomcat_threads : Passed
test - tomcat_memory : Passed
test - tomcat_sessions : Failed - The following web applications have an unusually large number of active sessions: vmrest. Please collect all of the Tomcat logs for root cause analysis: file get activelog tomcat/logs/*
skip - tomcat_heapdump : This module must be run directly and off hours
test - validate_network : Passed
test - raid : Passed
test - system_info : Passed (Collected system information in diagnostic log)
test - ntp_reachability : Passed
test - ntp_clock_drift : Passed
test - ntp_stratum : Failed
The reference NTP server is a stratum 5 clock.
NTP servers with stratum 5 or worse clocks are deemed unreliable.
Please consider using an NTP server with better stratum level.

Please use OS Admin GUI to add/delete NTP servers.

skip - sdl_fragmentation : This module must be run directly and off hours
skip - sdi_fragmentation : This module must be run directly and off hours

Diagnostics Completed


The final output will be in Log file: platform/log/diag1.log


Please use 'file view activelog platform/log/diag1.log' command to see the output

admin:


Regards


Naray

________________________________
From: Ryan Huff <ryanhuff@outlook.com>
Sent: Tuesday, November 26, 2019 6:27 AM
To: naresh rathore <nareh84@hotmail.com>
Cc: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: [cisco-voip] unityconnection db replication issue

Run “ utils diagnose test “ from the primary CLI; everything pass?

Sent from my iPhone

On Nov 25, 2019, at 20:11, naresh rathore <nareh84@hotmail.com> wrote:

?
hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cisco.com%2Fc%2Fen%2Fus%2Fsupport%2Fdocs%2Funified-communications%2Funity-connection%2F116942-technote-uc-00.html&data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962950398&sdata=oxx23Q2W6Iz%2F48d663ONW2S0IDiMnC94%2FfhThlxxjJM%3D&reserved=0>


Regards


Naray


_______________________________________________
cisco-voip mailing list
cisco-voip@puck.nether.net
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Fcisco-voip&amp;data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962970408&amp;sdata=FLTmpeiinFcfYQX4ANnxezVeYeYYCuNfKBPLSawCMFo%3D&amp;reserved=0
Re: unityconnection db replication issue [ In reply to ]
Hi Naray,

In addition you may also verify below


1. “Cisco DB, A Cisco DB Replicator, Cisco Database layer Monitor” these services should be running.

2. Informix database also rely heavily on DNS, if you have DNS configured both forward and reverse DNS lookup should work.


Thanks!
Jinto.



Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: cisco-voip <cisco-voip-bounces@puck.nether.net> on behalf of naresh rathore <nareh84@hotmail.com>
Sent: Monday, November 25, 2019 8:29:56 PM
To: Ryan Huff <ryanhuff@outlook.com>
Cc: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: [cisco-voip] unityconnection db replication issue

hi Ryan



following is the output


admin:utils diagnose test

Log file: platform/log/diag1.log

Starting diagnostic test(s)
===========================
test - disk_space : Passed (available: 17668 MB, used: 10920 MB)
skip - disk_files : This module must be run directly and off hours
test - service_manager : Passed
test - tomcat : Passed
test - tomcat_deadlocks : Passed
test - tomcat_keystore : Passed
test - tomcat_connectors : Passed
test - tomcat_threads : Passed
test - tomcat_memory : Passed
test - tomcat_sessions : Failed - The following web applications have an unusually large number of active sessions: vmrest. Please collect all of the Tomcat logs for root cause analysis: file get activelog tomcat/logs/*
skip - tomcat_heapdump : This module must be run directly and off hours
test - validate_network : Passed
test - raid : Passed
test - system_info : Passed (Collected system information in diagnostic log)
test - ntp_reachability : Passed
test - ntp_clock_drift : Passed
test - ntp_stratum : Failed
The reference NTP server is a stratum 5 clock.
NTP servers with stratum 5 or worse clocks are deemed unreliable.
Please consider using an NTP server with better stratum level.

Please use OS Admin GUI to add/delete NTP servers.

skip - sdl_fragmentation : This module must be run directly and off hours
skip - sdi_fragmentation : This module must be run directly and off hours

Diagnostics Completed


The final output will be in Log file: platform/log/diag1.log


Please use 'file view activelog platform/log/diag1.log' command to see the output

admin:


Regards


Naray

________________________________
From: Ryan Huff <ryanhuff@outlook.com>
Sent: Tuesday, November 26, 2019 6:27 AM
To: naresh rathore <nareh84@hotmail.com>
Cc: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: [cisco-voip] unityconnection db replication issue

Run “ utils diagnose test “ from the primary CLI; everything pass?

Sent from my iPhone

On Nov 25, 2019, at 20:11, naresh rathore <nareh84@hotmail.com> wrote:

?
hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cisco.com%2Fc%2Fen%2Fus%2Fsupport%2Fdocs%2Funified-communications%2Funity-connection%2F116942-technote-uc-00.html&data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962950398&sdata=oxx23Q2W6Iz%2F48d663ONW2S0IDiMnC94%2FfhThlxxjJM%3D&reserved=0>


Regards


Naray


_______________________________________________
cisco-voip mailing list
cisco-voip@puck.nether.net
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Fcisco-voip&amp;data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962970408&amp;sdata=FLTmpeiinFcfYQX4ANnxezVeYeYYCuNfKBPLSawCMFo%3D&amp;reserved=0
Re: unityconnection db replication issue [ In reply to ]
I would do a few things things;

- utils NTP status (replace any NTP server higher than stratum 3). Cisco says stratum 3 or lower is where you need to be; while stratum 5 isn’t incredibly high, it may be just a little too high.

I’m not a fan of using fqdns for NTP servers, try to use IP addresses and while I’d prefer you use an internal NTP server (to your network) that references external NTP sources, directly specifying external NTP servers on a UC server is better than having an unreliably high stratum. Use “utils NTP server”... list, add and delete to manipulate.

- Next, on both CUC servers, after you get NTP straightened out, from the CLI (primary then HA), issue “utils service restart Cisco Tomcat”. Wait about 20 after the primary, then do the HA.

After about 20 minutes from when you restarted the tomcat service on the HA, do “utils dbreplication runtimestate” from the CLI of the primary. If you still don’t have “(2) setup complete” for both nodes, you’ll probably need to manually reset cluster replication.

To do that, perform the following in order specified (all from the respective CLIs;


1.) “utils dbreplication stop all” (only on the primary server)

2.) “utils dbreplication dropadmindb” (first on the HA server, then the primary server)

3.) “utils dbreplication reset all” (only on the primary server)


This process overall could take an hour or more, so be patient.


Afterwards, try “utils dbreplication runtimestate” from the CLI of the primary server and see what you get, you should see “(2) Setup Complete” on both nodes.


If you do, then run “show CUC cluster status” on the CLI of the primary server, and verify the primary server is the Unity Connection DB primary. Alternatively, go to “Tools > Cluster” under Unity Connection Servicability in the Web Admin GUI and make sure the primary server, is the Unity Connection DB primary.


If you still don’t have “(2) Setup Complete” for both nodes, I’d call Cisco TAC and get a case going.

Thanks,

Ryan

On Nov 25, 2019, at 20:43, Jinto Alakkal Kunjumon <jalakkal@uoguelph.ca> wrote:

?
Hi Naray,

In addition you may also verify below


1. “Cisco DB, A Cisco DB Replicator, Cisco Database layer Monitor” these services should be running.

2. Informix database also rely heavily on DNS, if you have DNS configured both forward and reverse DNS lookup should work.


Thanks!
Jinto.



Get Outlook for iOS<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=02%7C01%7C%7Cc01f4349e05a4018a47c08d772121a47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103294378440292&sdata=ESCQ7t%2Fcten55ioK5H%2Bp5%2BzUSwzovyryfDcH7MKVV7o%3D&reserved=0>
________________________________
From: cisco-voip <cisco-voip-bounces@puck.nether.net> on behalf of naresh rathore <nareh84@hotmail.com>
Sent: Monday, November 25, 2019 8:29:56 PM
To: Ryan Huff <ryanhuff@outlook.com>
Cc: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: [cisco-voip] unityconnection db replication issue

hi Ryan



following is the output


admin:utils diagnose test

Log file: platform/log/diag1.log

Starting diagnostic test(s)
===========================
test - disk_space : Passed (available: 17668 MB, used: 10920 MB)
skip - disk_files : This module must be run directly and off hours
test - service_manager : Passed
test - tomcat : Passed
test - tomcat_deadlocks : Passed
test - tomcat_keystore : Passed
test - tomcat_connectors : Passed
test - tomcat_threads : Passed
test - tomcat_memory : Passed
test - tomcat_sessions : Failed - The following web applications have an unusually large number of active sessions: vmrest. Please collect all of the Tomcat logs for root cause analysis: file get activelog tomcat/logs/*
skip - tomcat_heapdump : This module must be run directly and off hours
test - validate_network : Passed
test - raid : Passed
test - system_info : Passed (Collected system information in diagnostic log)
test - ntp_reachability : Passed
test - ntp_clock_drift : Passed
test - ntp_stratum : Failed
The reference NTP server is a stratum 5 clock.
NTP servers with stratum 5 or worse clocks are deemed unreliable.
Please consider using an NTP server with better stratum level.

Please use OS Admin GUI to add/delete NTP servers.

skip - sdl_fragmentation : This module must be run directly and off hours
skip - sdi_fragmentation : This module must be run directly and off hours

Diagnostics Completed


The final output will be in Log file: platform/log/diag1.log


Please use 'file view activelog platform/log/diag1.log' command to see the output

admin:


Regards


Naray

________________________________
From: Ryan Huff <ryanhuff@outlook.com>
Sent: Tuesday, November 26, 2019 6:27 AM
To: naresh rathore <nareh84@hotmail.com>
Cc: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: [cisco-voip] unityconnection db replication issue

Run “ utils diagnose test “ from the primary CLI; everything pass?

Sent from my iPhone

On Nov 25, 2019, at 20:11, naresh rathore <nareh84@hotmail.com> wrote:

?
hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cisco.com%2Fc%2Fen%2Fus%2Fsupport%2Fdocs%2Funified-communications%2Funity-connection%2F116942-technote-uc-00.html&data=02%7C01%7C%7Cc01f4349e05a4018a47c08d772121a47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103294378450297&sdata=SUlSjUX9yEhQjBOjypIq39jUnMj5kE%2BF5NSN0LTlPAs%3D&reserved=0>


Regards


Naray


_______________________________________________
cisco-voip mailing list
cisco-voip@puck.nether.net
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Fcisco-voip&amp;data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962970408&amp;sdata=FLTmpeiinFcfYQX4ANnxezVeYeYYCuNfKBPLSawCMFo%3D&amp;reserved=0
Re: unityconnection db replication issue [ In reply to ]
Ditto on the NTP stratum. Higher values worked fine in 9.x for us, but caused problems in 11.5 in our upgrade. Going to stratum 3 or lower fixed some very strange issues.

Loren

On Nov 25, 2019, at 8:25 PM, Ryan Huff <ryanhuff@outlook.com> wrote:

?
I would do a few things things;

- utils NTP status (replace any NTP server higher than stratum 3). Cisco says stratum 3 or lower is where you need to be; while stratum 5 isn’t incredibly high, it may be just a little too high.

I’m not a fan of using fqdns for NTP servers, try to use IP addresses and while I’d prefer you use an internal NTP server (to your network) that references external NTP sources, directly specifying external NTP servers on a UC server is better than having an unreliably high stratum. Use “utils NTP server”... list, add and delete to manipulate.

- Next, on both CUC servers, after you get NTP straightened out, from the CLI (primary then HA), issue “utils service restart Cisco Tomcat”. Wait about 20 after the primary, then do the HA.

After about 20 minutes from when you restarted the tomcat service on the HA, do “utils dbreplication runtimestate” from the CLI of the primary. If you still don’t have “(2) setup complete” for both nodes, you’ll probably need to manually reset cluster replication.

To do that, perform the following in order specified (all from the respective CLIs;


1.) “utils dbreplication stop all” (only on the primary server)

2.) “utils dbreplication dropadmindb” (first on the HA server, then the primary server)

3.) “utils dbreplication reset all” (only on the primary server)


This process overall could take an hour or more, so be patient.


Afterwards, try “utils dbreplication runtimestate” from the CLI of the primary server and see what you get, you should see “(2) Setup Complete” on both nodes.


If you do, then run “show CUC cluster status” on the CLI of the primary server, and verify the primary server is the Unity Connection DB primary. Alternatively, go to “Tools > Cluster” under Unity Connection Servicability in the Web Admin GUI and make sure the primary server, is the Unity Connection DB primary.


If you still don’t have “(2) Setup Complete” for both nodes, I’d call Cisco TAC and get a case going.

Thanks,

Ryan

On Nov 25, 2019, at 20:43, Jinto Alakkal Kunjumon <jalakkal@uoguelph.ca> wrote:

?
Hi Naray,

In addition you may also verify below


1. “Cisco DB, A Cisco DB Replicator, Cisco Database layer Monitor” these services should be running.

2. Informix database also rely heavily on DNS, if you have DNS configured both forward and reverse DNS lookup should work.


Thanks!
Jinto.



Get Outlook for iOS<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=02%7C01%7C%7Cc01f4349e05a4018a47c08d772121a47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103294378440292&sdata=ESCQ7t%2Fcten55ioK5H%2Bp5%2BzUSwzovyryfDcH7MKVV7o%3D&reserved=0>
________________________________
From: cisco-voip <cisco-voip-bounces@puck.nether.net> on behalf of naresh rathore <nareh84@hotmail.com>
Sent: Monday, November 25, 2019 8:29:56 PM
To: Ryan Huff <ryanhuff@outlook.com>
Cc: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: [cisco-voip] unityconnection db replication issue

hi Ryan



following is the output


admin:utils diagnose test

Log file: platform/log/diag1.log

Starting diagnostic test(s)
===========================
test - disk_space : Passed (available: 17668 MB, used: 10920 MB)
skip - disk_files : This module must be run directly and off hours
test - service_manager : Passed
test - tomcat : Passed
test - tomcat_deadlocks : Passed
test - tomcat_keystore : Passed
test - tomcat_connectors : Passed
test - tomcat_threads : Passed
test - tomcat_memory : Passed
test - tomcat_sessions : Failed - The following web applications have an unusually large number of active sessions: vmrest. Please collect all of the Tomcat logs for root cause analysis: file get activelog tomcat/logs/*
skip - tomcat_heapdump : This module must be run directly and off hours
test - validate_network : Passed
test - raid : Passed
test - system_info : Passed (Collected system information in diagnostic log)
test - ntp_reachability : Passed
test - ntp_clock_drift : Passed
test - ntp_stratum : Failed
The reference NTP server is a stratum 5 clock.
NTP servers with stratum 5 or worse clocks are deemed unreliable.
Please consider using an NTP server with better stratum level.

Please use OS Admin GUI to add/delete NTP servers.

skip - sdl_fragmentation : This module must be run directly and off hours
skip - sdi_fragmentation : This module must be run directly and off hours

Diagnostics Completed


The final output will be in Log file: platform/log/diag1.log


Please use 'file view activelog platform/log/diag1.log' command to see the output

admin:


Regards


Naray

________________________________
From: Ryan Huff <ryanhuff@outlook.com>
Sent: Tuesday, November 26, 2019 6:27 AM
To: naresh rathore <nareh84@hotmail.com>
Cc: cisco-voip@puck.nether.net <cisco-voip@puck.nether.net>
Subject: Re: [cisco-voip] unityconnection db replication issue

Run “ utils diagnose test “ from the primary CLI; everything pass?

Sent from my iPhone

On Nov 25, 2019, at 20:11, naresh rathore <nareh84@hotmail.com> wrote:

?
hi,


unityconnection version is Version 11.5.1.14900-11 . I am currently facing db replication issue. following is the output of utils dbreplication status and runtimestate.

admin:utils dbreplication status

connect to rapcucon01_ccm11_5_1_14900_11 failed
Enterprise Replication not active (62)
command failed -- unable to connect to server specified (5)

Replication status check is now running in background.
Use command 'utils dbreplication runtimestate' to check its progress

The final output will be in file cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out

Please use "file view activelog cm/trace/dbl/sdi/ReplicationStatus.2019_11_26_11_31_03.out " command to see the output
admin:utils dbreplication runtimestate

Server Time: Tue Nov 26 11:36:38 AEDT 2019

Cluster Replication State: dbmonpreflightcheck file is found locally, please check again in few minutes...

DB Version: ccm11_5_1_14900_11

Repltimeout set to: 300s
PROCESS option set to: 1


Cluster Detailed View from RAPCUCON01 (2 Servers):

PING DB/RPC/ REPL. Replication REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) DbMon? QUEUE Group ID (RTMT) & Details
----------- ---------- ------ ------- ----- ----------- ------------------
RAPCUCON02 10.163.72.111 0.133 Y/Y/Y -- (-) (-) Replication Not Setup
RAPCUCON01 10.163.72.110 0.021 Y/Y/Y -- (-) (-) Replication Not Setup



on RTMT, i can see following message
On Tue Nov 26 11:41:19 AEDT 2019; alert DBReplicationFailure has occured. Counter Replicate_State of Number of Replicates Created and State of Replication(ReplicateCount) on node RAPCUCON01 has state value of 4. ReasonCode: Replication Setup did not succeed.



I am able to ping between the two, both unity connection are on the same subnet and there is no firewall in between.


I also used the following link to restart dbreplication


https://www.cisco.com/c/en/us/support/docs/unified-communications/unity-connection/116942-technote-uc-00.html<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cisco.com%2Fc%2Fen%2Fus%2Fsupport%2Fdocs%2Funified-communications%2Funity-connection%2F116942-technote-uc-00.html&data=02%7C01%7C%7Cc01f4349e05a4018a47c08d772121a47%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103294378450297&sdata=SUlSjUX9yEhQjBOjypIq39jUnMj5kE%2BF5NSN0LTlPAs%3D&reserved=0>


Regards


Naray


_______________________________________________
cisco-voip mailing list
cisco-voip@puck.nether.net
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Fcisco-voip&amp;data=02%7C01%7C%7C148294bcabd64250932008d7720d94f0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637103274962970408&amp;sdata=FLTmpeiinFcfYQX4ANnxezVeYeYYCuNfKBPLSawCMFo%3D&amp;reserved=0
_______________________________________________
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip