Mailing List Archive: DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã

DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦

Feb 16, 2015, 6:38 PM

Post #1 of 7 (4970 views)

ãŠä¸–è©±ã«ãªã‚Šã¾ã™ã€‚ã‚ãã‚„ã¾ã¨ç”³ã—ã¾ã™ã€‚

ä¸‹è¨˜ã®ç’°å¢ƒã§é‹ç”¨ã‚’ã—ã¦ã„ã¾ã™ã€‚

ç’°å¢ƒï¼š
Red Hat Enterprise Linux Server 6.1 (for x86)
kernel-2.6.32-431.23.3.el6.i686
corosync-1.4.1-17.el6_5.1.i686
pacemaker-1.1.10-14.el6_5.3.i686
drbd-8.3.16-1.el6.i386

è³ªå•å†…å®¹ï¼š
é‹ç”¨ä¸ã«çªç„¶DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ãŒç™ºç”Ÿã—ã€æ‰‹å‹•ã§å¾©æ—§ã•ã›ã¾ã—ãŸãŒã€
ç™ºç”ŸåŽŸå› ãŒä¸æ˜Žã§å›°ã£ã¦ã„ã¾ã™ã€‚
ç™ºç”ŸåŽŸå› ã¨å›žé¿ç–ãŒåˆ†ã‹ã‚Œã°ã€ã”æ•™æŽˆé¡˜ãˆã¾ã™ã§ã—ã‚‡ã†ã‹ã€‚

ç¾åœ¨ã¾ã§ã«åˆ¤æ˜Žã—ã¦ã„ã‚‹äº‹ï¼š
ãƒ»DRBDãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã®pingå¿œç”ã¯å•é¡Œãªã—
ãƒ»messagesãƒã‚°ã«ãƒªãƒ³ã‚¯ãƒ€ã‚¦ãƒ³ã®å‡ºåŠ›ãªã—

ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®messagesãƒã‚°ã€‘
Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown )
conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on
sock: r=-110
Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
Unconnected )
Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
WFConnection )
Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
from=::ffff:192.171.1.102
Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo
$(date +%Y/%m/%d\ %H:%M:%S)'
Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
duration=0(sec)
Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed
network protocol version 97
Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
WFReportParams )
Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from
drbd0_receiver [3117])
Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg: <not-used>
Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by
peer.
Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
NetworkFailure )
Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
Unconnected )
Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
WFConnection )

ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®corosyncãƒã‚°ã€‘
Feb 16 18:00:27 server1 corosync[1756]: [MAIN ] main.c:1341 Corosync
main process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms).
Consider token timeout increase.
Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0,
lost=1
Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:773 info:
pcmk_peer_update: memb: server1 1694608064
Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:779 info:
pcmk_peer_update: lost: server2 1711385280
Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:791 info:
pcmk_peer_update: NEW: server2 1711385280
Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server1 1694608064
Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server2 1711385280
Feb 16 18:00:27 server1 corosync[1756]: [TOTEM ] totemsrp.c:1892 A
processor joined or left the membership and a new membership was formed.
Feb 16 18:00:27 server1 corosync[1756]: [CPG ] cpg.c:783 chosen
downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
Feb 16 18:00:28 server1 corosync[1756]: [MAIN ] main.c:1341 Corosync
main process was not scheduled for 826.4143 ms (threshold is 800.0000 ms).
Consider token timeout increase.
Feb 16 18:00:28 server1 corosync[1756]: [MAIN ] main.c:278 Completed
service synchronization, ready to provide service.
Feb 16 18:00:28 server1 crmd[1766]: warning: crmd_ha_msg_filter: Another
DC detected: server2 (op=noop)
Feb 16 18:00:28 server1 crmd[1766]: notice: do_state_transition: State
transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
origin=crmd_ha_msg_filter ]
Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff: Not
requesting full refresh in R/W mode
Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff: Not
requesting full refresh in R/W mode
Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff: Not
requesting full refresh in R/W mode
Feb 16 18:00:30 server1 corosync[1756]: [MAIN ] main.c:1341 Corosync
main process was not scheduled for 2248.0801 ms (threshold is 800.0000 ms).
Consider token timeout increase.
Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0,
lost=1
Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:773 info:
pcmk_peer_update: memb: server1 1694608064
Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:779 info:
pcmk_peer_update: lost: server2 1711385280
Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:791 info:
pcmk_peer_update: NEW: server2 1711385280
Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server1 1694608064
Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server2 1711385280
Feb 16 18:00:31 server1 corosync[1756]: [TOTEM ] totemsrp.c:1892 A
processor joined or left the membership and a new membership was formed.
Feb 16 18:00:32 server1 corosync[1756]: [CPG ] cpg.c:783 chosen
downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
Feb 16 18:00:32 server1 corosync[1756]: [MAIN ] main.c:278 Completed
service synchronization, ready to provide service.
Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff: Not
requesting full refresh in R/W mode
Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff: Not
requesting full refresh in R/W mode
Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff: Not
requesting full refresh in R/W mode
Feb 16 18:02:28 server1 crmd[1766]: error: crm_timer_popped: Election
Timeout (I_ELECTION_DC) just popped in state S_ELECTION! (120000ms)
Feb 16 18:02:28 server1 crmd[1766]: notice: do_state_transition: State
transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
cause=C_TIMER_POPPED origin=crm_timer_popped ]
Feb 16 18:02:28 server1 crmd[1766]: notice: do_state_transition: State
transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
cause=C_FSA_INTERNAL origin=do_election_check ]
Feb 16 18:02:28 server1 crmd[1766]: notice: do_dc_join_finalize: join-4:
Syncing the CIB from server2 to the rest of the cluster
Feb 16 18:02:28 server1 crmd[1766]: notice: do_dc_join_finalize:
Requested version <generation_tuple epoch="7" num_updates="87"
admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Wed Jan 21
00:17:56 2015" update-origin="server1" update-client="crmd"
crm_feature_set="3.0.7" have-quorum="1" dc-uuid="server2"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: Diff: --- 0.7.69
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: Diff: +++ 0.7.87
1afd9af32d3a574358543e1d4751aaf1
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: -- <node_state
crm-debug-origin="do_state_transition" join="member" id="server1"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
<lrm_rsc_op operation_key="res_drbd_r0_start_0" operation="start"
transition-key="46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
transition-magic="0:0;46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
call-id="44" last-run="1421767134" last-rc-change="1421767134"
exec-time="430" id="res_drbd_r0_last_0"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
<lrm_rsc_op id="res_drbd_r0_monitor_30000"
operation_key="res_drbd_r0_monitor_30000" operation="monitor"
crm-debug-origin="do_update_resource" crm_feature_set="3.0.7"
transition-key="47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
transition-magic="0:0;47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
call-id="50" rc-code="0" op-status="0" interval="30000"
last-rc-change="1421767135" exec-time="60" queue-time="0"
op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: -- <nvpair
value="1000" id="status-server2-master-res_drbd_r0"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++ <node_state
id="server1" uname="server1" in_ccm="true" crmd="online"
crm-debug-origin="post_cache_update" join="down" expected="member"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++ <nvpair
id="status-server2-master-res_drbd_r0" name="master-res_drbd_r0"
value="10000"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
<lrm_rsc_op id="res_drbd_r0_last_0" operation_key="res_drbd_r0_promote_0"
operation="promote" crm-debug-origin="do_update_resource"
crm_feature_set="3.0.7"
transition-key="14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
transition-magic="0:0;14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
call-id="62" rc-code="0" op-status="0" interval="0" last-run="1424077226"
last-rc-change="1424077226" exec-time="120" queue-time="0"
op-digest="c0e018b73fdf522b6cdd355e125af15e"/>
Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
<lrm_rsc_op id="res_drbd_r0_monitor_10000"
operation_key="res_drbd_r0_monitor_10000" operation="monitor"
crm-debug-origin="do_update_resource" crm_feature_set="3.0.7"
transition-key="15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
transition-magic="0:8;15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
call-id="68" rc-code="8" op-status="0" interval="10000"
last-rc-change="1424077226" exec-time="53" queue-time="0"
op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
Feb 16 18:02:28 server1 stonith-ng[1762]: warning: cib_process_diff: Diff
0.7.69 -> 0.7.87 from local not applied to 0.7.69: Failed application of an
update diff
Feb 16 18:02:28 server1 stonith-ng[1762]: notice: update_cib_cache_cb:
[cib_diff_notify] Patch aborted: Application of an update diff failed (-206)
Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_local_callback:
Sending full refresh (origin=crmd)
Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (10000)
Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_trigger_update:
Sending flush op to all hosts for: probe_complete (true)
Feb 16 18:02:28 server1 pengine[1765]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Feb 16 18:02:28 server1 pengine[1765]: notice: LogActions: Demote
res_drbd_r0:1#011(Master -> Slave server2)
Feb 16 18:02:28 server1 pengine[1765]: notice: process_pe_message:
Calculated Transition 2582: /var/lib/pacemaker/pengine/pe-input-24.bz2
Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 8: cancel res_drbd_r0_cancel_10000 on server2
Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 81: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 83: notify res_drbd_r0_pre_notify_demote_0 on server2
Feb 16 18:02:28 server1 crmd[1766]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=113, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:02:28 server1 crmd[1766]: notice: run_graph: Transition 2582
(Complete=4, Pending=0, Fired=0, Skipped=4, Incomplete=5,
Source=/var/lib/pacemaker/pengine/pe-input-24.bz2): Stopped
Feb 16 18:02:28 server1 pengine[1765]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Feb 16 18:02:28 server1 pengine[1765]: notice: LogActions: Demote
res_drbd_r0:1#011(Master -> Slave server2)
Feb 16 18:02:28 server1 pengine[1765]: notice: process_pe_message:
Calculated Transition 2583: /var/lib/pacemaker/pengine/pe-input-25.bz2
Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 80: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 82: notify res_drbd_r0_pre_notify_demote_0 on server2
Feb 16 18:02:29 server1 crmd[1766]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=116, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 39: demote res_drbd_r0_demote_0 on server2
Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 81: notify res_drbd_r0_post_notify_demote_0 on server1 (local)
Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 83: notify res_drbd_r0_post_notify_demote_0 on server2
Feb 16 18:02:29 server1 crmd[1766]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=119, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:02:29 server1 crmd[1766]: notice: run_graph: Transition 2583
(Complete=11, Pending=0, Fired=0, Skipped=1, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-25.bz2): Stopped
Feb 16 18:02:29 server1 pengine[1765]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Feb 16 18:02:29 server1 pengine[1765]: notice: process_pe_message:
Calculated Transition 2584: /var/lib/pacemaker/pengine/pe-input-26.bz2
Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
action 40: monitor res_drbd_r0_monitor_30000 on server2
Feb 16 18:02:29 server1 crmd[1766]: notice: run_graph: Transition 2584
(Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-26.bz2): Complete
Feb 16 18:02:29 server1 crmd[1766]: notice: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
cause=C_FSA_INTERNAL origin=notify_crmd ]
Feb 16 18:17:29 server1 crmd[1766]: notice: do_state_transition: State
transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED
origin=crm_timer_popped ]
Feb 16 18:17:29 server1 pengine[1765]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Feb 16 18:17:29 server1 pengine[1765]: notice: process_pe_message:
Calculated Transition 2585: /var/lib/pacemaker/pengine/pe-input-27.bz2
Feb 16 18:17:29 server1 crmd[1766]: notice: run_graph: Transition 2585
(Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-27.bz2): Complete
Feb 16 18:17:29 server1 crmd[1766]: notice: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
cause=C_FSA_INTERNAL origin=notify_crmd ]

ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®messagesãƒã‚°ã€‘
Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown )
conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on
sock: r=-110
Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
Unconnected )
Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
WFConnection )
Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
from=::ffff:192.171.1.102
Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo
$(date +%Y/%m/%d\ %H:%M:%S)'
Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
duration=0(sec)
Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed
network protocol version 97
Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
WFReportParams )
Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from
drbd0_receiver [3117])
Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg: <not-used>
Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by
peer.
Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
NetworkFailure )
Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
Unconnected )
Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
WFConnection )

ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®corosyncãƒã‚°ã€‘
Feb 16 18:00:23 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (1000)
Feb 16 18:00:23 server2 attrd[1557]: notice: attrd_perform_update: Sent
update 45: master-res_drbd_r0=1000
Feb 16 18:00:24 server2 corosync[1549]: [TOTEM ] totemsrp.c:1595 A
processor failed, forming new configuration.
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Transitional membership event on ring 12: memb=1, new=0,
lost=1
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
pcmk_peer_update: memb: server2 1711385280
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:779 info:
pcmk_peer_update: lost: server1 1694608064
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Stable membership event on ring 12: memb=1, new=0, lost=0
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server2 1711385280
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:731 info:
ais_mark_unseen_peer_dead: Node server1 was not seen in the previous
transition
Feb 16 18:00:25 server2 cib[1554]: notice: plugin_handle_membership:
Membership 12: quorum lost
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] utils.c:358 info:
update_member: Node 1694608064/server1 is now: lost
Feb 16 18:00:25 server2 crmd[1559]: notice: plugin_handle_membership:
Membership 12: quorum lost
Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
send_member_notification: Sending membership update 12 to 2 children
Feb 16 18:00:25 server2 cib[1554]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now lost (was
member)
Feb 16 18:00:25 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
processor joined or left the membership and a new membership was formed.
Feb 16 18:00:25 server2 crmd[1559]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now lost (was
member)
Feb 16 18:00:25 server2 crmd[1559]: warning: reap_dead_nodes: Our DC node
(server1) left the cluster
Feb 16 18:00:25 server2 crmd[1559]: notice: do_state_transition: State
transition S_NOT_DC -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
origin=reap_dead_nodes ]
Feb 16 18:00:25 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
Feb 16 18:00:25 server2 corosync[1549]: [MAIN ] main.c:278 Completed
service synchronization, ready to provide service.
Feb 16 18:00:25 server2 crmd[1559]: notice: do_state_transition: State
transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
cause=C_FSA_INTERNAL origin=do_election_check ]
Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_local_callback:
Sending full refresh (origin=crmd)
Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (1000)
Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_perform_update: Sent
update 47: master-res_drbd_r0=1000
Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: probe_complete (true)
Feb 16 18:00:26 server2 pengine[1558]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Feb 16 18:00:26 server2 pengine[1558]: notice: LogActions: Promote
res_drbd_r0:0#011(Slave -> Master server2)
Feb 16 18:00:26 server2 pengine[1558]: notice: process_pe_message:
Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-0.bz2
Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
action 1: cancel res_drbd_r0_cancel_30000 on server2 (local)
Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
action 56: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=56, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:00:26 server2 crmd[1559]: notice: run_graph: Transition 0
(Complete=3, Pending=0, Fired=0, Skipped=5, Incomplete=4,
Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Stopped
Feb 16 18:00:26 server2 pengine[1558]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Feb 16 18:00:26 server2 pengine[1558]: notice: LogActions: Promote
res_drbd_r0:0#011(Slave -> Master server2)
Feb 16 18:00:26 server2 pengine[1558]: notice: process_pe_message:
Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-1.bz2
Feb 16 18:00:26 server2 crmd[1559]: warning: destroy_action: Cancelling
timer for action 1 (src=45)
Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
action 55: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0,
lost=0
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
pcmk_peer_update: memb: server2 1711385280
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] utils.c:358 info:
update_member: Node 1694608064/server1 is now: member
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:791 info:
pcmk_peer_update: NEW: server1 1694608064
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server1 1694608064
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server2 1711385280
Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
send_member_notification: Sending membership update 16 to 2 children
Feb 16 18:00:26 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
processor joined or left the membership and a new membership was formed.
Feb 16 18:00:26 server2 crmd[1559]: notice: plugin_handle_membership:
Membership 16: quorum acquired
Feb 16 18:00:26 server2 crmd[1559]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now member
(was lost)
Feb 16 18:00:26 server2 cib[1554]: notice: plugin_handle_membership:
Membership 16: quorum acquired
Feb 16 18:00:26 server2 cib[1554]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now member
(was lost)
Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=59, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
action 14: promote res_drbd_r0_promote_0 on server2 (local)
Feb 16 18:00:26 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_promote_0 (call=62, rc=0, cib-update=49,
confirmed=true) ok
Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
action 56: notify res_drbd_r0_post_notify_promote_0 on server2 (local)
Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (10000)
Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_perform_update: Sent
update 51: master-res_drbd_r0=10000
Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=65, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
action 15: monitor res_drbd_r0_monitor_10000 on server2 (local)
Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (10000)
Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_perform_update: Sent
update 53: master-res_drbd_r0=10000
Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_monitor_10000 (call=68, rc=8, cib-update=50,
confirmed=false) master
Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event:
server2-res_drbd_r0_monitor_10000:68 [ \n ]
Feb 16 18:00:26 server2 crmd[1559]: notice: run_graph: Transition 1
(Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
Feb 16 18:00:27 server2 corosync[1549]: [MAIN ] main.c:278 Completed
service synchronization, ready to provide service.
Feb 16 18:00:28 server2 corosync[1549]: [TOTEM ] totemsrp.c:1595 A
processor failed, forming new configuration.
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Transitional membership event on ring 20: memb=1, new=0,
lost=1
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
pcmk_peer_update: memb: server2 1711385280
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:779 info:
pcmk_peer_update: lost: server1 1694608064
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Stable membership event on ring 20: memb=1, new=0, lost=0
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server2 1711385280
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:731 info:
ais_mark_unseen_peer_dead: Node server1 was not seen in the previous
transition
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] utils.c:358 info:
update_member: Node 1694608064/server1 is now: lost
Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
send_member_notification: Sending membership update 20 to 2 children
Feb 16 18:00:29 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
processor joined or left the membership and a new membership was formed.
Feb 16 18:00:29 server2 cib[1554]: notice: plugin_handle_membership:
Membership 20: quorum lost
Feb 16 18:00:29 server2 crmd[1559]: notice: plugin_handle_membership:
Membership 20: quorum lost
Feb 16 18:00:29 server2 cib[1554]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now lost (was
member)
Feb 16 18:00:29 server2 crmd[1559]: warning: match_down_event: No match
for shutdown action on server1
Feb 16 18:00:29 server2 crmd[1559]: notice: peer_update_callback:
Stonith/shutdown of server1 not matched
Feb 16 18:00:29 server2 crmd[1559]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now lost (was
member)
Feb 16 18:00:29 server2 crmd[1559]: warning: match_down_event: No match
for shutdown action on server1
Feb 16 18:00:29 server2 crmd[1559]: notice: peer_update_callback:
Stonith/shutdown of server1 not matched
Feb 16 18:00:29 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
Feb 16 18:00:29 server2 corosync[1549]: [MAIN ] main.c:278 Completed
service synchronization, ready to provide service.
Feb 16 18:00:29 server2 crmd[1559]: notice: do_election_count_vote:
Election 4 (current: 2, owner: server1): Processed vote from server1 (Peer
is not part of our cluster)
Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_local_callback:
Sending full refresh (origin=crmd)
Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (10000)
Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: probe_complete (true)
Feb 16 18:00:29 server2 pengine[1558]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Feb 16 18:00:29 server2 pengine[1558]: notice: process_pe_message:
Calculated Transition 2: /var/lib/pacemaker/pengine/pe-input-2.bz2
Feb 16 18:00:29 server2 crmd[1559]: notice: run_graph: Transition 2
(Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-2.bz2): Complete
Feb 16 18:00:29 server2 crmd[1559]: notice: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
cause=C_FSA_INTERNAL origin=notify_crmd ]
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0,
lost=0
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
pcmk_peer_update: memb: server2 1711385280
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] utils.c:358 info:
update_member: Node 1694608064/server1 is now: member
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:791 info:
pcmk_peer_update: NEW: server1 1694608064
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server1 1694608064
Feb 16 18:00:30 server2 crmd[1559]: notice: plugin_handle_membership:
Membership 24: quorum acquired
Feb 16 18:00:30 server2 cib[1554]: notice: plugin_handle_membership:
Membership 24: quorum acquired
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
pcmk_peer_update: MEMB: server2 1711385280
Feb 16 18:00:30 server2 cib[1554]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now member
(was lost)
Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
send_member_notification: Sending membership update 24 to 2 children
Feb 16 18:00:30 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
processor joined or left the membership and a new membership was formed.
Feb 16 18:00:30 server2 crmd[1559]: notice: crm_update_peer_state:
plugin_handle_membership: Node server1[1694608064] - state is now member
(was lost)
Feb 16 18:00:30 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
Feb 16 18:00:31 server2 corosync[1549]: [MAIN ] main.c:278 Completed
service synchronization, ready to provide service.
Feb 16 18:00:31 server2 attrd[1557]: warning: attrd_cib_callback: Update
45 for master-res_drbd_r0=1000 failed: Application of an update diff failed
Feb 16 18:00:56 server2 crmd[1559]: warning: cib_rsc_callback: Resource
update 49 failed: (rc=-62) Timer expired
Feb 16 18:00:56 server2 crmd[1559]: warning: cib_rsc_callback: Resource
update 50 failed: (rc=-62) Timer expired
Feb 16 18:02:26 server2 attrd[1557]: warning: attrd_cib_callback: Update
51 for master-res_drbd_r0=10000 failed: Timer expired
Feb 16 18:02:27 server2 crmd[1559]: warning: crmd_ha_msg_filter: Another
DC detected: server1 (op=join_offer)
Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition: State
transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
origin=crmd_ha_msg_filter ]
Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition: State
transition S_ELECTION -> S_RELEASE_DC [ input=I_RELEASE_DC
cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition: State
transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE
origin=do_cl_join_finalize_respond ]
Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_local_callback:
Sending full refresh (origin=crmd)
Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (10000)
Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: probe_complete (true)
Feb 16 18:02:27 server2 crmd[1559]: notice: process_lrm_event:
server2-res_drbd_r0_monitor_10000:68 [ \n ]
Feb 16 18:02:27 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=74, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=77, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_demote_0 (call=80, rc=0, cib-update=74,
confirmed=true) ok
Feb 16 18:02:28 server2 attrd[1557]: notice: attrd_trigger_update:
Sending flush op to all hosts for: master-res_drbd_r0 (1000)
Feb 16 18:02:28 server2 attrd[1557]: notice: attrd_perform_update: Sent
update 67: master-res_drbd_r0=1000
Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_notify_0 (call=83, rc=0, cib-update=0,
confirmed=true) ok
Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
operation res_drbd_r0_monitor_30000 (call=86, rc=0, cib-update=75,
confirmed=false) ok

ã€global_common.confã€‘
global {
usage-count no;
}

common {
protocol C;
handlers {
}
startup {
}
disk {
no-disk-barrier;
no-disk-flushes;
}
net {
max-buffers 8000;
max-epoch-size 8000;
after-sb-0pri discard-younger-primary;
after-sb-1pri discard-secondary;
after-sb-2pri disconnect;
}
syncer {
rate 30M;
verify-alg sha1;
}
}

Re: DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦ [ In reply to ]

renayama19661014 at ybb

Feb 16, 2015, 7:31 PM

Post #2 of 7 (5338 views)

Permalink

ã‚ãã‚„ã¾ã•ã‚“

ã“ã‚“ã«ã¡ã¯ã€å±±å†…ã§ã™ã€‚

corosyncã®ãƒ—ãƒ©ã‚¤ãƒžãƒªãƒã‚°ã‚’è¦‹ã‚‹é™ã‚Šã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã¨ã®corosyncå±¤ã®é€šä¿¡ãŒåˆ‡ã‚Œã¦ã€ä¸€æ—¦ã€å†æ§‹æˆãŒã‹ã‹ã£ãŸã‚ˆã†ã§ã™ã€‚

corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ

>Feb 16 18:00:27 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms). Consider token timeout increase.

ä¸Šè¨˜ã®ãƒã‚°ãŒå‡ºã¦ã„ã¾ã™ã®ã§ã€æœ¬æ¥ã®corosyncã®ã‚¹ã‚±ã‚¸ãƒ¥ãƒ¼ãƒªãƒ³ã‚°ãŒï¼’ç§’å¼·ã€é…ã‚Œã¦ã„ã‚‹çŠ¶æ…‹ãŒé€šçŸ¥ã•ã‚Œã¦ã¾ã™ã€‚

ä»¥ä¸Šã€å®œã—ããŠé¡˜ã„ã„ãŸã—ã¾ã™ã€‚

----- Original Message -----
>From: ã‚ãã‚„ã¾ã˜ã‚ã† <mayama0130@gmail.com>
>To: linux-ha-japan@lists.sourceforge.jp
>Date: 2015/2/17, Tue 11:38
>Subject: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
>
>
>ãŠä¸–è©±ã«ãªã‚Šã¾ã™ã€‚ã‚ãã‚„ã¾ã¨ç”³ã—ã¾ã™ã€‚
>
>ä¸‹è¨˜ã®ç’°å¢ƒã§é‹ç”¨ã‚’ã—ã¦ã„ã¾ã™ã€‚
>
>ç’°å¢ƒï¼š
>Red Hat Enterprise Linux Server 6.1 (for x86)
>kernel-2.6.32-431.23.3.el6.i686
>corosync-1.4.1-17.el6_5.1.i686
>pacemaker-1.1.10-14.el6_5.3.i686
>drbd-8.3.16-1.el6.i386
>
>è³ªå•å†…å®¹ï¼š
>é‹ç”¨ä¸ã«çªç„¶DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ãŒç™ºç”Ÿã—ã€æ‰‹å‹•ã§å¾©æ—§ã•ã›ã¾ã—ãŸãŒã€
>ç™ºç”ŸåŽŸå› ãŒä¸æ˜Žã§å›°ã£ã¦ã„ã¾ã™ã€‚
>ç™ºç”ŸåŽŸå› ã¨å›žé¿ç–ãŒåˆ†ã‹ã‚Œã°ã€ã”æ•™æŽˆé¡˜ãˆã¾ã™ã§ã—ã‚‡ã†ã‹ã€‚
>
>ç¾åœ¨ã¾ã§ã«åˆ¤æ˜Žã—ã¦ã„ã‚‹äº‹ï¼š
>ãƒ»DRBDãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã®pingå¿œç”ã¯å•é¡Œãªã—
>ãƒ»messagesãƒã‚°ã«ãƒªãƒ³ã‚¯ãƒ€ã‚¦ãƒ³ã®å‡ºåŠ›ãªã—
>
>ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®messagesãƒã‚°ã€‘
>Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
>Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown ) conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
>Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on sock: r=-110
>Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
>Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
>Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
>Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
>Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe -> Unconnected )
>Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
>Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
>Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
>Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected -> WFConnection )
>Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056 from=::ffff:192.171.1.102
>Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo $(date +%Y/%m/%d\ %H:%M:%S)'
>Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056 duration=0(sec)
>Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed network protocol version 97
>Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection -> WFReportParams )
>Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from drbd0_receiver [3117])
>Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg: <not-used>
>Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by peer.
>Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams -> NetworkFailure )
>Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
>Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
>Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
>Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure -> Unconnected )
>Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
>Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
>Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
>Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected -> WFConnection )
>
>ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®corosyncãƒã‚°ã€‘
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms). Consider token timeout increase.
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0, lost=1
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:773 info: pcmk_peer_update: memb: server1 1694608064
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:779 info: pcmk_peer_update: lost: server2 1711385280
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:791 info: pcmk_peer_update: NEW:Â server2 1711385280
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server1 1694608064
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server2 1711385280
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [TOTEM ] totemsrp.c:1892 A processor joined or left the membership and a new membership was formed.
>Feb 16 18:00:27 server1 corosync[1756]:Â Â [CPGÂ Â ] cpg.c:783 chosen downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
>Feb 16 18:00:28 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main process was not scheduled for 826.4143 ms (threshold is 800.0000 ms). Consider token timeout increase.
>Feb 16 18:00:28 server1 corosync[1756]:Â Â [MAINÂ ] main.c:278 Completed service synchronization, ready to provide service.
>Feb 16 18:00:28 server1 crmd[1766]:Â warning: crmd_ha_msg_filter: Another DC detected: server2 (op=noop)
>Feb 16 18:00:28 server1 crmd[1766]:Â Â notice: do_state_transition: State transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=crmd_ha_msg_filter ]
>Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not requesting full refresh in R/W mode
>Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not requesting full refresh in R/W mode
>Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not requesting full refresh in R/W mode
>Feb 16 18:00:30 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main process was not scheduled for 2248.0801 ms (threshold is 800.0000 ms). Consider token timeout increase.
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0, lost=1
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:773 info: pcmk_peer_update: memb: server1 1694608064
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:779 info: pcmk_peer_update: lost: server2 1711385280
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:791 info: pcmk_peer_update: NEW:Â server2 1711385280
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server1 1694608064
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server2 1711385280
>Feb 16 18:00:31 server1 corosync[1756]:Â Â [TOTEM ] totemsrp.c:1892 A processor joined or left the membership and a new membership was formed.
>Feb 16 18:00:32 server1 corosync[1756]:Â Â [CPGÂ Â ] cpg.c:783 chosen downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
>Feb 16 18:00:32 server1 corosync[1756]:Â Â [MAINÂ ] main.c:278 Completed service synchronization, ready to provide service.
>Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not requesting full refresh in R/W mode
>Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not requesting full refresh in R/W mode
>Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not requesting full refresh in R/W mode
>Feb 16 18:02:28 server1 crmd[1766]:Â Â Â error: crm_timer_popped: Election Timeout (I_ELECTION_DC) just popped in state S_ELECTION! (120000ms)
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_TIMER_POPPED origin=crm_timer_popped ]
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_dc_join_finalize: join-4: Syncing the CIB from server2 to the rest of the cluster
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_dc_join_finalize: Requested versionÂ Â <generation_tuple epoch="7" num_updates="87" admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Wed Jan 21 00:17:56 2015" update-origin="server1" update-client="crmd" crm_feature_set="3.0.7" have-quorum="1" dc-uuid="server2"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: Diff: --- 0.7.69
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: Diff: +++ 0.7.87 1afd9af32d3a574358543e1d4751aaf1
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â <node_state crm-debug-origin="do_state_transition" join="member" id="server1"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â Â Â <lrm_rsc_op operation_key="res_drbd_r0_start_0" operation="start" transition-key="46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df" transition-magic="0:0;46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df" call-id="44" last-run="1421767134" last-rc-change="1421767134" exec-time="430" id="res_drbd_r0_last_0"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â Â Â <lrm_rsc_op id="res_drbd_r0_monitor_30000" operation_key="res_drbd_r0_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.7" transition-key="47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df" transition-magic="0:0;47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df" call-id="50" rc-code="0" op-status="0" interval="30000" last-rc-change="1421767135" exec-time="60" queue-time="0" op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â <nvpair value="1000" id="status-server2-master-res_drbd_r0"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â <node_state id="server1" uname="server1" in_ccm="true" crmd="online" crm-debug-origin="post_cache_update" join="down" expected="member"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â <nvpair id="status-server2-master-res_drbd_r0" name="master-res_drbd_r0" value="10000"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â Â Â <lrm_rsc_op id="res_drbd_r0_last_0" operation_key="res_drbd_r0_promote_0" operation="promote" crm-debug-origin="do_update_resource" crm_feature_set="3.0.7" transition-key="14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb" transition-magic="0:0;14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb" call-id="62" rc-code="0" op-status="0" interval="0" last-run="1424077226" last-rc-change="1424077226" exec-time="120" queue-time="0" op-digest="c0e018b73fdf522b6cdd355e125af15e"/>
>Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â Â Â <lrm_rsc_op id="res_drbd_r0_monitor_10000" operation_key="res_drbd_r0_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.7" transition-key="15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb" transition-magic="0:8;15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb" call-id="68" rc-code="8" op-status="0" interval="10000" last-rc-change="1424077226" exec-time="53" queue-time="0" op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
>Feb 16 18:02:28 server1 stonith-ng[1762]:Â warning: cib_process_diff: Diff 0.7.69 -> 0.7.87 from local not applied to 0.7.69: Failed application of an update diff
>Feb 16 18:02:28 server1 stonith-ng[1762]:Â Â notice: update_cib_cache_cb: [cib_diff_notify] Patch aborted: Application of an update diff failed (-206)
>Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_local_callback: Sending full refresh (origin=crmd)
>Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (10000)
>Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
>Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: unpack_config: On loss of CCM Quorum: Ignore
>Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: LogActions: DemoteÂ res_drbd_r0:1#011(Master -> Slave server2)
>Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: process_pe_message: Calculated Transition 2582: /var/lib/pacemaker/pengine/pe-input-24.bz2
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 8: cancel res_drbd_r0_cancel_10000 on server2
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 81: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 83: notify res_drbd_r0_pre_notify_demote_0 on server2
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=113, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: run_graph: Transition 2582 (Complete=4, Pending=0, Fired=0, Skipped=4, Incomplete=5, Source=/var/lib/pacemaker/pengine/pe-input-24.bz2): Stopped
>Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: unpack_config: On loss of CCM Quorum: Ignore
>Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: LogActions: DemoteÂ res_drbd_r0:1#011(Master -> Slave server2)
>Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: process_pe_message: Calculated Transition 2583: /var/lib/pacemaker/pengine/pe-input-25.bz2
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 80: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
>Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 82: notify res_drbd_r0_pre_notify_demote_0 on server2
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=116, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 39: demote res_drbd_r0_demote_0 on server2
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 81: notify res_drbd_r0_post_notify_demote_0 on server1 (local)
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 83: notify res_drbd_r0_post_notify_demote_0 on server2
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=119, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2583 (Complete=11, Pending=0, Fired=0, Skipped=1, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-25.bz2): Stopped
>Feb 16 18:02:29 server1 pengine[1765]:Â Â notice: unpack_config: On loss of CCM Quorum: Ignore
>Feb 16 18:02:29 server1 pengine[1765]:Â Â notice: process_pe_message: Calculated Transition 2584: /var/lib/pacemaker/pengine/pe-input-26.bz2
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating action 40: monitor res_drbd_r0_monitor_30000 on server2
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2584 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-26.bz2): Complete
>Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
>Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ]
>Feb 16 18:17:29 server1 pengine[1765]:Â Â notice: unpack_config: On loss of CCM Quorum: Ignore
>Feb 16 18:17:29 server1 pengine[1765]:Â Â notice: process_pe_message: Calculated Transition 2585: /var/lib/pacemaker/pengine/pe-input-27.bz2
>Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2585 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-27.bz2): Complete
>Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
>
>ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®messagesãƒã‚°ã€‘
>Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
>Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown ) conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
>Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on sock: r=-110
>Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
>Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
>Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
>Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
>Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe -> Unconnected )
>Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
>Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
>Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
>Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected -> WFConnection )
>Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056 from=::ffff:192.171.1.102
>Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo $(date +%Y/%m/%d\ %H:%M:%S)'
>Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056 duration=0(sec)
>Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed network protocol version 97
>Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection -> WFReportParams )
>Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from drbd0_receiver [3117])
>Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg: <not-used>
>Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by peer.
>Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams -> NetworkFailure )
>Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
>Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
>Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
>Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure -> Unconnected )
>Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
>Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
>Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
>Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected -> WFConnection )
>
>ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®corosyncãƒã‚°ã€‘
>Feb 16 18:00:23 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (1000)
>Feb 16 18:00:23 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent update 45: master-res_drbd_r0=1000
>Feb 16 18:00:24 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1595 A processor failed, forming new configuration.
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Transitional membership event on ring 12: memb=1, new=0, lost=1
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info: pcmk_peer_update: memb: server2 1711385280
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:779 info: pcmk_peer_update: lost: server1 1694608064
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Stable membership event on ring 12: memb=1, new=0, lost=0
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server2 1711385280
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:731 info: ais_mark_unseen_peer_dead: Node server1 was not seen in the previous transition
>Feb 16 18:00:25 server2 cib[1554]:Â Â notice: plugin_handle_membership: Membership 12: quorum lost
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info: update_member: Node 1694608064/server1 is now: lost
>Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: plugin_handle_membership: Membership 12: quorum lost
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info: send_member_notification: Sending membership update 12 to 2 children
>Feb 16 18:00:25 server2 cib[1554]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now lost (was member)
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A processor joined or left the membership and a new membership was formed.
>Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now lost (was member)
>Feb 16 18:00:25 server2 crmd[1559]:Â warning: reap_dead_nodes: Our DC node (server1) left the cluster
>Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: do_state_transition: State transition S_NOT_DC -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=reap_dead_nodes ]
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
>Feb 16 18:00:25 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed service synchronization, ready to provide service.
>Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
>Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending full refresh (origin=crmd)
>Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (1000)
>Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent update 47: master-res_drbd_r0=1000
>Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
>Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: unpack_config: On loss of CCM Quorum: Ignore
>Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: LogActions: Promote res_drbd_r0:0#011(Slave -> Master server2)
>Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: process_pe_message: Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-0.bz2
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating action 1: cancel res_drbd_r0_cancel_30000 on server2 (local)
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating action 56: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=56, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: run_graph: Transition 0 (Complete=3, Pending=0, Fired=0, Skipped=5, Incomplete=4, Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Stopped
>Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: unpack_config: On loss of CCM Quorum: Ignore
>Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: LogActions: Promote res_drbd_r0:0#011(Slave -> Master server2)
>Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: process_pe_message: Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-1.bz2
>Feb 16 18:00:26 server2 crmd[1559]:Â warning: destroy_action: Cancelling timer for action 1 (src=45)
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating action 55: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0, lost=0
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info: pcmk_peer_update: memb: server2 1711385280
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info: update_member: Node 1694608064/server1 is now: member
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:791 info: pcmk_peer_update: NEW:Â server1 1694608064
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server1 1694608064
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server2 1711385280
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info: send_member_notification: Sending membership update 16 to 2 children
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A processor joined or left the membership and a new membership was formed.
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: plugin_handle_membership: Membership 16: quorum acquired
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now member (was lost)
>Feb 16 18:00:26 server2 cib[1554]:Â Â notice: plugin_handle_membership: Membership 16: quorum acquired
>Feb 16 18:00:26 server2 cib[1554]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now member (was lost)
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=59, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating action 14: promote res_drbd_r0_promote_0 on server2 (local)
>Feb 16 18:00:26 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_promote_0 (call=62, rc=0, cib-update=49, confirmed=true) ok
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating action 56: notify res_drbd_r0_post_notify_promote_0 on server2 (local)
>Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (10000)
>Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent update 51: master-res_drbd_r0=10000
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=65, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating action 15: monitor res_drbd_r0_monitor_10000 on server2 (local)
>Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (10000)
>Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent update 53: master-res_drbd_r0=10000
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_monitor_10000 (call=68, rc=8, cib-update=50, confirmed=false) master
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: server2-res_drbd_r0_monitor_10000:68 [ \n ]
>Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: run_graph: Transition 1 (Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
>Feb 16 18:00:27 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed service synchronization, ready to provide service.
>Feb 16 18:00:28 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1595 A processor failed, forming new configuration.
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Transitional membership event on ring 20: memb=1, new=0, lost=1
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info: pcmk_peer_update: memb: server2 1711385280
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:779 info: pcmk_peer_update: lost: server1 1694608064
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Stable membership event on ring 20: memb=1, new=0, lost=0
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server2 1711385280
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:731 info: ais_mark_unseen_peer_dead: Node server1 was not seen in the previous transition
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info: update_member: Node 1694608064/server1 is now: lost
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info: send_member_notification: Sending membership update 20 to 2 children
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A processor joined or left the membership and a new membership was formed.
>Feb 16 18:00:29 server2 cib[1554]:Â Â notice: plugin_handle_membership: Membership 20: quorum lost
>Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: plugin_handle_membership: Membership 20: quorum lost
>Feb 16 18:00:29 server2 cib[1554]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now lost (was member)
>Feb 16 18:00:29 server2 crmd[1559]:Â warning: match_down_event: No match for shutdown action on server1
>Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: peer_update_callback: Stonith/shutdown of server1 not matched
>Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now lost (was member)
>Feb 16 18:00:29 server2 crmd[1559]:Â warning: match_down_event: No match for shutdown action on server1
>Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: peer_update_callback: Stonith/shutdown of server1 not matched
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
>Feb 16 18:00:29 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed service synchronization, ready to provide service.
>Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: do_election_count_vote: Election 4 (current: 2, owner: server1): Processed vote from server1 (Peer is not part of our cluster)
>Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending full refresh (origin=crmd)
>Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (10000)
>Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
>Feb 16 18:00:29 server2 pengine[1558]:Â Â notice: unpack_config: On loss of CCM Quorum: Ignore
>Feb 16 18:00:29 server2 pengine[1558]:Â Â notice: process_pe_message: Calculated Transition 2: /var/lib/pacemaker/pengine/pe-input-2.bz2
>Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: run_graph: Transition 2 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-2.bz2): Complete
>Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0, lost=0
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info: pcmk_peer_update: memb: server2 1711385280
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice: pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info: update_member: Node 1694608064/server1 is now: member
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:791 info: pcmk_peer_update: NEW:Â server1 1694608064
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server1 1694608064
>Feb 16 18:00:30 server2 crmd[1559]:Â Â notice: plugin_handle_membership: Membership 24: quorum acquired
>Feb 16 18:00:30 server2 cib[1554]:Â Â notice: plugin_handle_membership: Membership 24: quorum acquired
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info: pcmk_peer_update: MEMB: server2 1711385280
>Feb 16 18:00:30 server2 cib[1554]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now member (was lost)
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info: send_member_notification: Sending membership update 24 to 2 children
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A processor joined or left the membership and a new membership was formed.
>Feb 16 18:00:30 server2 crmd[1559]:Â Â notice: crm_update_peer_state: plugin_handle_membership: Node server1[1694608064] - state is now member (was lost)
>Feb 16 18:00:30 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
>Feb 16 18:00:31 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed service synchronization, ready to provide service.
>Feb 16 18:00:31 server2 attrd[1557]:Â warning: attrd_cib_callback: Update 45 for master-res_drbd_r0=1000 failed: Application of an update diff failed
>Feb 16 18:00:56 server2 crmd[1559]:Â warning: cib_rsc_callback: Resource update 49 failed: (rc=-62) Timer expired
>Feb 16 18:00:56 server2 crmd[1559]:Â warning: cib_rsc_callback: Resource update 50 failed: (rc=-62) Timer expired
>Feb 16 18:02:26 server2 attrd[1557]:Â warning: attrd_cib_callback: Update 51 for master-res_drbd_r0=10000 failed: Timer expired
>Feb 16 18:02:27 server2 crmd[1559]:Â warning: crmd_ha_msg_filter: Another DC detected: server1 (op=join_offer)
>Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=crmd_ha_msg_filter ]
>Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State transition S_ELECTION -> S_RELEASE_DC [ input=I_RELEASE_DC cause=C_FSA_INTERNAL origin=do_election_count_vote ]
>Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
>Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending full refresh (origin=crmd)
>Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (10000)
>Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
>Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: process_lrm_event: server2-res_drbd_r0_monitor_10000:68 [ \n ]
>Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=74, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=77, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_demote_0 (call=80, rc=0, cib-update=74, confirmed=true) ok
>Feb 16 18:02:28 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending flush op to all hosts for: master-res_drbd_r0 (1000)
>Feb 16 18:02:28 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent update 67: master-res_drbd_r0=1000
>Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_notify_0 (call=83, rc=0, cib-update=0, confirmed=true) ok
>Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM operation res_drbd_r0_monitor_30000 (call=86, rc=0, cib-update=75, confirmed=false) ok
>
>ã€global_common.confã€‘
>global {
>Â Â Â Â Â Â Â usage-count no;
>}
>
>common {
>Â Â Â Â Â Â Â protocol C;
>Â Â Â Â Â Â Â handlers {
>Â Â Â Â Â Â Â }
>Â Â Â Â Â Â Â startup {
>Â Â Â Â Â Â Â }
>Â Â Â Â Â Â Â disk {
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â no-disk-barrier;
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â no-disk-flushes;
>Â Â Â Â Â Â Â }
>Â Â Â Â Â Â Â net {
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â max-buffers 8000;
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â max-epoch-size 8000;
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-0pri discard-younger-primary;
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-1pri discard-secondary;
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-2pri disconnect;
>Â Â Â Â Â Â Â }
>Â Â Â Â Â Â Â syncer {
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â rate 30M;
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â verify-alg sha1;
>Â Â Â Â Â Â Â }
>}
>
>
>
>
>_______________________________________________
>Linux-ha-japan mailing list
>Linux-ha-japan@lists.sourceforge.jp
>http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>
>
>

_______________________________________________
Linux-ha-japan mailing list
Linux-ha-japan@lists.sourceforge.jp
http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan

Re: DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦ [ In reply to ]

renayama19661014 at ybb

Feb 16, 2015, 7:58 PM

Post #3 of 7 (4908 views)

Permalink

ã‚ãã‚„ã¾ã•ã‚“

å±±å†…ã§ã™ã€‚

ã¡ãªã¿ã«ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã®è² è·ãŒé«˜ã„å¯èƒ½æ€§ã‚‚ã‚ã‚Šã¾ã™ã®ã§ã€
sarã‚„vmstatã§cpuã®çŠ¶æ…‹ã‚‚è¦‹ã¦ã¿ãŸæ–¹ãŒã‚ˆã„ã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚

ï¼ƒã‚ã‚‹æ™‚ã«ã€ä½•ã‹ã®cronã‚¸ãƒ§ãƒ–ãªã©ã§è² è·ãŒã‚ãŒã£ãŸã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚

ä»¥ä¸Šã§ã™ã€‚

----- Original Message -----
> From: "renayama19661014@ybb.ne.jp" <renayama19661014@ybb.ne.jp>
> To: "linux-ha-japan@lists.sourceforge.jp" <linux-ha-japan@lists.sourceforge.jp>
> Cc:
> Date: 2015/2/17, Tue 12:31
> Subject: Re: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
>
> ã‚ãã‚„ã¾ã•ã‚“
>
> ã“ã‚“ã«ã¡ã¯ã€å±±å†…ã§ã™ã€‚
>
> corosyncã®ãƒ—ãƒ©ã‚¤ãƒžãƒªãƒã‚°ã‚’è¦‹ã‚‹é™ã‚Šã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã¨ã®corosyncå±¤ã®é€šä¿¡ãŒåˆ‡ã‚Œã¦ã€ä¸€æ—¦ã€å†æ§‹æˆãŒã‹ã‹ã£ãŸã‚ˆã†ã§ã™ã€‚
>
> corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ
>
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
> process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms). Consider
> token timeout increase.
>
>
> ä¸Šè¨˜ã®ãƒã‚°ãŒå‡ºã¦ã„ã¾ã™ã®ã§ã€æœ¬æ¥ã®corosyncã®ã‚¹ã‚±ã‚¸ãƒ¥ãƒ¼ãƒªãƒ³ã‚°ãŒï¼’ç§’å¼·ã€é…ã‚Œã¦ã„ã‚‹çŠ¶æ…‹ãŒé€šçŸ¥ã•ã‚Œã¦ã¾ã™ã€‚
>
> ä»¥ä¸Šã€å®œã—ããŠé¡˜ã„ã„ãŸã—ã¾ã™ã€‚
>
>
> ----- Original Message -----
>> From: ã‚ãã‚„ã¾ã˜ã‚ã† <mayama0130@gmail.com>
>> To: linux-ha-japan@lists.sourceforge.jp
>> Date: 2015/2/17, Tue 11:38
>> Subject: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
>>
>>
>> ãŠä¸–è©±ã«ãªã‚Šã¾ã™ã€‚ã‚ãã‚„ã¾ã¨ç”³ã—ã¾ã™ã€‚
>>
>> ä¸‹è¨˜ã®ç’°å¢ƒã§é‹ç”¨ã‚’ã—ã¦ã„ã¾ã™ã€‚
>>
>> ç’°å¢ƒï¼š
>> Red Hat Enterprise Linux Server 6.1 (for x86)
>> kernel-2.6.32-431.23.3.el6.i686
>> corosync-1.4.1-17.el6_5.1.i686
>> pacemaker-1.1.10-14.el6_5.3.i686
>> drbd-8.3.16-1.el6.i386
>>
>> è³ªå•å†…å®¹ï¼š
>> é‹ç”¨ä¸ã«çªç„¶DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ãŒç™ºç”Ÿã—ã€æ‰‹å‹•ã§å¾©æ—§ã•ã›ã¾ã—ãŸãŒã€
>> ç™ºç”ŸåŽŸå› ãŒä¸æ˜Žã§å›°ã£ã¦ã„ã¾ã™ã€‚
>> ç™ºç”ŸåŽŸå› ã¨å›žé¿ç–ãŒåˆ†ã‹ã‚Œã°ã€ã”æ•™æŽˆé¡˜ãˆã¾ã™ã§ã—ã‚‡ã†ã‹ã€‚
>>
>> ç¾åœ¨ã¾ã§ã«åˆ¤æ˜Žã—ã¦ã„ã‚‹äº‹ï¼š
>> ãƒ»DRBDãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã®pingå¿œç”ã¯å•é¡Œãªã—
>> ãƒ»messagesãƒã‚°ã«ãƒªãƒ³ã‚¯ãƒ€ã‚¦ãƒ³ã®å‡ºåŠ›ãªã—
>>
>> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®messagesãƒã‚°ã€‘
>> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
>> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown )
> conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
>> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on
> sock: r=-110
>> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
> 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
>> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
>> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
>> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
> Unconnected )
>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
>> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
> WFConnection )
>> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
> from=::ffff:192.171.1.102
>> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo
> $(date +%Y/%m/%d\ %H:%M:%S)'
>> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
> duration=0(sec)
>> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed
> network protocol version 97
>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
> WFReportParams )
>> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from
> drbd0_receiver [3117])
>> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
> <not-used>
>> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by
> peer.
>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
> NetworkFailure )
>> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
>> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
>> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
> Unconnected )
>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
>> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
> WFConnection )
>>
>> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®corosyncãƒã‚°ã€‘
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
> process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms). Consider
> token timeout increase.
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0,
> lost=1
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:773 info:
> pcmk_peer_update: memb: server1 1694608064
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:779 info:
> pcmk_peer_update: lost: server2 1711385280
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:791 info:
> pcmk_peer_update: NEW:Â server2 1711385280
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server1 1694608064
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server2 1711385280
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [TOTEM ] totemsrp.c:1892 A
> processor joined or left the membership and a new membership was formed.
>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [CPGÂ Â ] cpg.c:783 chosen
> downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
>> Feb 16 18:00:28 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
> process was not scheduled for 826.4143 ms (threshold is 800.0000 ms). Consider
> token timeout increase.
>> Feb 16 18:00:28 server1 corosync[1756]:Â Â [MAINÂ ] main.c:278 Completed
> service synchronization, ready to provide service.
>> Feb 16 18:00:28 server1 crmd[1766]:Â warning: crmd_ha_msg_filter: Another DC
> detected: server2 (op=noop)
>> Feb 16 18:00:28 server1 crmd[1766]:Â Â notice: do_state_transition: State
> transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> origin=crmd_ha_msg_filter ]
>> Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not
> requesting full refresh in R/W mode
>> Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not
> requesting full refresh in R/W mode
>> Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not
> requesting full refresh in R/W mode
>> Feb 16 18:00:30 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
> process was not scheduled for 2248.0801 ms (threshold is 800.0000 ms). Consider
> token timeout increase.
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0,
> lost=1
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:773 info:
> pcmk_peer_update: memb: server1 1694608064
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:779 info:
> pcmk_peer_update: lost: server2 1711385280
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:791 info:
> pcmk_peer_update: NEW:Â server2 1711385280
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server1 1694608064
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server2 1711385280
>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [TOTEM ] totemsrp.c:1892 A
> processor joined or left the membership and a new membership was formed.
>> Feb 16 18:00:32 server1 corosync[1756]:Â Â [CPGÂ Â ] cpg.c:783 chosen
> downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
>> Feb 16 18:00:32 server1 corosync[1756]:Â Â [MAINÂ ] main.c:278 Completed
> service synchronization, ready to provide service.
>> Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not
> requesting full refresh in R/W mode
>> Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not
> requesting full refresh in R/W mode
>> Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not
> requesting full refresh in R/W mode
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â Â error: crm_timer_popped: Election
> Timeout (I_ELECTION_DC) just popped in state S_ELECTION! (120000ms)
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_state_transition: State
> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> cause=C_TIMER_POPPED origin=crm_timer_popped ]
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_state_transition: State
> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> cause=C_FSA_INTERNAL origin=do_election_check ]
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_dc_join_finalize: join-4:
> Syncing the CIB from server2 to the rest of the cluster
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_dc_join_finalize: Requested
> versionÂ Â <generation_tuple epoch="7" num_updates="87"
> admin_epoch="0" validate-with="pacemaker-1.2"
> cib-last-written="Wed Jan 21 00:17:56 2015"
> update-origin="server1" update-client="crmd"
> crm_feature_set="3.0.7" have-quorum="1"
> dc-uuid="server2"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: Diff: --- 0.7.69
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: Diff: +++ 0.7.87
> 1afd9af32d3a574358543e1d4751aaf1
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â <node_state
> crm-debug-origin="do_state_transition" join="member"
> id="server1"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â Â Â
> <lrm_rsc_op operation_key="res_drbd_r0_start_0"
> operation="start"
> transition-key="46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> transition-magic="0:0;46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> call-id="44" last-run="1421767134"
> last-rc-change="1421767134" exec-time="430"
> id="res_drbd_r0_last_0"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â Â Â
> <lrm_rsc_op id="res_drbd_r0_monitor_30000"
> operation_key="res_drbd_r0_monitor_30000"
> operation="monitor" crm-debug-origin="do_update_resource"
> crm_feature_set="3.0.7"
> transition-key="47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> transition-magic="0:0;47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> call-id="50" rc-code="0" op-status="0"
> interval="30000" last-rc-change="1421767135"
> exec-time="60" queue-time="0"
> op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â
> <nvpair value="1000"
> id="status-server2-master-res_drbd_r0"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â <node_state
> id="server1" uname="server1" in_ccm="true"
> crmd="online" crm-debug-origin="post_cache_update"
> join="down" expected="member"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â
> <nvpair id="status-server2-master-res_drbd_r0"
> name="master-res_drbd_r0" value="10000"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â Â Â
> <lrm_rsc_op id="res_drbd_r0_last_0"
> operation_key="res_drbd_r0_promote_0" operation="promote"
> crm-debug-origin="do_update_resource"
> crm_feature_set="3.0.7"
> transition-key="14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> transition-magic="0:0;14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> call-id="62" rc-code="0" op-status="0"
> interval="0" last-run="1424077226"
> last-rc-change="1424077226" exec-time="120"
> queue-time="0"
> op-digest="c0e018b73fdf522b6cdd355e125af15e"/>
>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â Â Â
> <lrm_rsc_op id="res_drbd_r0_monitor_10000"
> operation_key="res_drbd_r0_monitor_10000"
> operation="monitor" crm-debug-origin="do_update_resource"
> crm_feature_set="3.0.7"
> transition-key="15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> transition-magic="0:8;15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> call-id="68" rc-code="8" op-status="0"
> interval="10000" last-rc-change="1424077226"
> exec-time="53" queue-time="0"
> op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
>> Feb 16 18:02:28 server1 stonith-ng[1762]:Â warning: cib_process_diff: Diff
> 0.7.69 -> 0.7.87 from local not applied to 0.7.69: Failed application of an
> update diff
>> Feb 16 18:02:28 server1 stonith-ng[1762]:Â Â notice: update_cib_cache_cb:
> [cib_diff_notify] Patch aborted: Application of an update diff failed (-206)
>> Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_local_callback: Sending
> full refresh (origin=crmd)
>> Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (10000)
>> Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
> CCM Quorum: Ignore
>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: LogActions: DemoteÂ
> res_drbd_r0:1#011(Master -> Slave server2)
>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: process_pe_message:
> Calculated Transition 2582: /var/lib/pacemaker/pengine/pe-input-24.bz2
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 8: cancel res_drbd_r0_cancel_10000 on server2
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 81: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 83: notify res_drbd_r0_pre_notify_demote_0 on server2
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=113, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: run_graph: Transition 2582
> (Complete=4, Pending=0, Fired=0, Skipped=4, Incomplete=5,
> Source=/var/lib/pacemaker/pengine/pe-input-24.bz2): Stopped
>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
> CCM Quorum: Ignore
>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: LogActions: DemoteÂ
> res_drbd_r0:1#011(Master -> Slave server2)
>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: process_pe_message:
> Calculated Transition 2583: /var/lib/pacemaker/pengine/pe-input-25.bz2
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 80: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 82: notify res_drbd_r0_pre_notify_demote_0 on server2
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=116, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 39: demote res_drbd_r0_demote_0 on server2
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 81: notify res_drbd_r0_post_notify_demote_0 on server1 (local)
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 83: notify res_drbd_r0_post_notify_demote_0 on server2
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=119, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2583
> (Complete=11, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> Source=/var/lib/pacemaker/pengine/pe-input-25.bz2): Stopped
>> Feb 16 18:02:29 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
> CCM Quorum: Ignore
>> Feb 16 18:02:29 server1 pengine[1765]:Â Â notice: process_pe_message:
> Calculated Transition 2584: /var/lib/pacemaker/pengine/pe-input-26.bz2
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
> action 40: monitor res_drbd_r0_monitor_30000 on server2
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2584
> (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pacemaker/pengine/pe-input-26.bz2): Complete
>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> cause=C_FSA_INTERNAL origin=notify_crmd ]
>> Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: do_state_transition: State
> transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED
> origin=crm_timer_popped ]
>> Feb 16 18:17:29 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
> CCM Quorum: Ignore
>> Feb 16 18:17:29 server1 pengine[1765]:Â Â notice: process_pe_message:
> Calculated Transition 2585: /var/lib/pacemaker/pengine/pe-input-27.bz2
>> Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2585
> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pacemaker/pengine/pe-input-27.bz2): Complete
>> Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> cause=C_FSA_INTERNAL origin=notify_crmd ]
>>
>> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®messagesãƒã‚°ã€‘
>> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
>> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown )
> conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
>> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on
> sock: r=-110
>> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
> 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
>> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
>> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
>> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
> Unconnected )
>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
>> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
> WFConnection )
>> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
> from=::ffff:192.171.1.102
>> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo
> $(date +%Y/%m/%d\ %H:%M:%S)'
>> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
> duration=0(sec)
>> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed
> network protocol version 97
>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
> WFReportParams )
>> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from
> drbd0_receiver [3117])
>> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
> <not-used>
>> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by
> peer.
>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
> NetworkFailure )
>> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
>> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
>> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
> Unconnected )
>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
>> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
> WFConnection )
>>
>> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®corosyncãƒã‚°ã€‘
>> Feb 16 18:00:23 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (1000)
>> Feb 16 18:00:23 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
> update 45: master-res_drbd_r0=1000
>> Feb 16 18:00:24 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1595 A
> processor failed, forming new configuration.
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Transitional membership event on ring 12: memb=1, new=0,
> lost=1
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
> pcmk_peer_update: memb: server2 1711385280
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:779 info:
> pcmk_peer_update: lost: server1 1694608064
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Stable membership event on ring 12: memb=1, new=0, lost=0
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server2 1711385280
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:731 info:
> ais_mark_unseen_peer_dead: Node server1 was not seen in the previous transition
>> Feb 16 18:00:25 server2 cib[1554]:Â Â notice: plugin_handle_membership:
> Membership 12: quorum lost
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
> update_member: Node 1694608064/server1 is now: lost
>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
> Membership 12: quorum lost
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
> send_member_notification: Sending membership update 12 to 2 children
>> Feb 16 18:00:25 server2 cib[1554]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
> member)
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
> processor joined or left the membership and a new membership was formed.
>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
> member)
>> Feb 16 18:00:25 server2 crmd[1559]:Â warning: reap_dead_nodes: Our DC node
> (server1) left the cluster
>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: do_state_transition: State
> transition S_NOT_DC -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> origin=reap_dead_nodes ]
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
> downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
> service synchronization, ready to provide service.
>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: do_state_transition: State
> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> cause=C_FSA_INTERNAL origin=do_election_check ]
>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending
> full refresh (origin=crmd)
>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (1000)
>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
> update 47: master-res_drbd_r0=1000
>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: unpack_config: On loss of
> CCM Quorum: Ignore
>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: LogActions: Promote
> res_drbd_r0:0#011(Slave -> Master server2)
>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: process_pe_message:
> Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-0.bz2
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
> action 1: cancel res_drbd_r0_cancel_30000 on server2 (local)
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
> action 56: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=56, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: run_graph: Transition 0
> (Complete=3, Pending=0, Fired=0, Skipped=5, Incomplete=4,
> Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Stopped
>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: unpack_config: On loss of
> CCM Quorum: Ignore
>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: LogActions: Promote
> res_drbd_r0:0#011(Slave -> Master server2)
>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: process_pe_message:
> Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-1.bz2
>> Feb 16 18:00:26 server2 crmd[1559]:Â warning: destroy_action: Cancelling
> timer for action 1 (src=45)
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
> action 55: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0,
> lost=0
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
> pcmk_peer_update: memb: server2 1711385280
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
> update_member: Node 1694608064/server1 is now: member
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:791 info:
> pcmk_peer_update: NEW:Â server1 1694608064
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server1 1694608064
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server2 1711385280
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
> send_member_notification: Sending membership update 16 to 2 children
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
> processor joined or left the membership and a new membership was formed.
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
> Membership 16: quorum acquired
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now member (was
> lost)
>> Feb 16 18:00:26 server2 cib[1554]:Â Â notice: plugin_handle_membership:
> Membership 16: quorum acquired
>> Feb 16 18:00:26 server2 cib[1554]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now member (was
> lost)
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=59, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
> action 14: promote res_drbd_r0_promote_0 on server2 (local)
>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
> downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_promote_0 (call=62, rc=0, cib-update=49, confirmed=true)
> ok
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
> action 56: notify res_drbd_r0_post_notify_promote_0 on server2 (local)
>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (10000)
>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
> update 51: master-res_drbd_r0=10000
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=65, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
> action 15: monitor res_drbd_r0_monitor_10000 on server2 (local)
>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (10000)
>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
> update 53: master-res_drbd_r0=10000
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_monitor_10000 (call=68, rc=8, cib-update=50,
> confirmed=false) master
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event:
> server2-res_drbd_r0_monitor_10000:68 [ \n ]
>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: run_graph: Transition 1
> (Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
>> Feb 16 18:00:27 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
> service synchronization, ready to provide service.
>> Feb 16 18:00:28 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1595 A
> processor failed, forming new configuration.
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Transitional membership event on ring 20: memb=1, new=0,
> lost=1
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
> pcmk_peer_update: memb: server2 1711385280
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:779 info:
> pcmk_peer_update: lost: server1 1694608064
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Stable membership event on ring 20: memb=1, new=0, lost=0
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server2 1711385280
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:731 info:
> ais_mark_unseen_peer_dead: Node server1 was not seen in the previous transition
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
> update_member: Node 1694608064/server1 is now: lost
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
> send_member_notification: Sending membership update 20 to 2 children
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
> processor joined or left the membership and a new membership was formed.
>> Feb 16 18:00:29 server2 cib[1554]:Â Â notice: plugin_handle_membership:
> Membership 20: quorum lost
>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
> Membership 20: quorum lost
>> Feb 16 18:00:29 server2 cib[1554]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
> member)
>> Feb 16 18:00:29 server2 crmd[1559]:Â warning: match_down_event: No match for
> shutdown action on server1
>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: peer_update_callback:
> Stonith/shutdown of server1 not matched
>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
> member)
>> Feb 16 18:00:29 server2 crmd[1559]:Â warning: match_down_event: No match for
> shutdown action on server1
>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: peer_update_callback:
> Stonith/shutdown of server1 not matched
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
> downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
> service synchronization, ready to provide service.
>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: do_election_count_vote:
> Election 4 (current: 2, owner: server1): Processed vote from server1 (Peer is
> not part of our cluster)
>> Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending
> full refresh (origin=crmd)
>> Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (10000)
>> Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
>> Feb 16 18:00:29 server2 pengine[1558]:Â Â notice: unpack_config: On loss of
> CCM Quorum: Ignore
>> Feb 16 18:00:29 server2 pengine[1558]:Â Â notice: process_pe_message:
> Calculated Transition 2: /var/lib/pacemaker/pengine/pe-input-2.bz2
>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: run_graph: Transition 2
> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pacemaker/pengine/pe-input-2.bz2): Complete
>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> cause=C_FSA_INTERNAL origin=notify_crmd ]
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0,
> lost=0
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
> pcmk_peer_update: memb: server2 1711385280
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
> pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
> update_member: Node 1694608064/server1 is now: member
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:791 info:
> pcmk_peer_update: NEW:Â server1 1694608064
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server1 1694608064
>> Feb 16 18:00:30 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
> Membership 24: quorum acquired
>> Feb 16 18:00:30 server2 cib[1554]:Â Â notice: plugin_handle_membership:
> Membership 24: quorum acquired
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
> pcmk_peer_update: MEMB: server2 1711385280
>> Feb 16 18:00:30 server2 cib[1554]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now member (was
> lost)
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
> send_member_notification: Sending membership update 24 to 2 children
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
> processor joined or left the membership and a new membership was formed.
>> Feb 16 18:00:30 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
> plugin_handle_membership: Node server1[1694608064] - state is now member (was
> lost)
>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
> downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
>> Feb 16 18:00:31 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
> service synchronization, ready to provide service.
>> Feb 16 18:00:31 server2 attrd[1557]:Â warning: attrd_cib_callback: Update 45
> for master-res_drbd_r0=1000 failed: Application of an update diff failed
>> Feb 16 18:00:56 server2 crmd[1559]:Â warning: cib_rsc_callback: Resource
> update 49 failed: (rc=-62) Timer expired
>> Feb 16 18:00:56 server2 crmd[1559]:Â warning: cib_rsc_callback: Resource
> update 50 failed: (rc=-62) Timer expired
>> Feb 16 18:02:26 server2 attrd[1557]:Â warning: attrd_cib_callback: Update 51
> for master-res_drbd_r0=10000 failed: Timer expired
>> Feb 16 18:02:27 server2 crmd[1559]:Â warning: crmd_ha_msg_filter: Another DC
> detected: server1 (op=join_offer)
>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State
> transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> origin=crmd_ha_msg_filter ]
>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State
> transition S_ELECTION -> S_RELEASE_DC [ input=I_RELEASE_DC
> cause=C_FSA_INTERNAL origin=do_election_count_vote ]
>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State
> transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE
> origin=do_cl_join_finalize_respond ]
>> Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending
> full refresh (origin=crmd)
>> Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (10000)
>> Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: process_lrm_event:
> server2-res_drbd_r0_monitor_10000:68 [ \n ]
>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=74, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=77, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_demote_0 (call=80, rc=0, cib-update=74, confirmed=true) ok
>> Feb 16 18:02:28 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-res_drbd_r0 (1000)
>> Feb 16 18:02:28 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
> update 67: master-res_drbd_r0=1000
>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_notify_0 (call=83, rc=0, cib-update=0, confirmed=true) ok
>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
> operation res_drbd_r0_monitor_30000 (call=86, rc=0, cib-update=75,
> confirmed=false) ok
>>
>> ã€global_common.confã€‘
>> global {
>> Â Â Â Â Â Â Â usage-count no;
>> }
>>
>> common {
>> Â Â Â Â Â Â Â protocol C;
>> Â Â Â Â Â Â Â handlers {
>> Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â startup {
>> Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â disk {
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â no-disk-barrier;
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â no-disk-flushes;
>> Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â net {
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â max-buffers 8000;
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â max-epoch-size 8000;
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-0pri discard-younger-primary;
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-1pri discard-secondary;
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-2pri disconnect;
>> Â Â Â Â Â Â Â }
>> Â Â Â Â Â Â Â syncer {
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â rate 30M;
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â verify-alg sha1;
>> Â Â Â Â Â Â Â }
>> }
>>
>>
>>
>>
>> _______________________________________________
>> Linux-ha-japan mailing list
>> Linux-ha-japan@lists.sourceforge.jp
>> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>>
>>
>>
>
> _______________________________________________
> Linux-ha-japan mailing list
> Linux-ha-japan@lists.sourceforge.jp
> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>

_______________________________________________
Linux-ha-japan mailing list
Linux-ha-japan@lists.sourceforge.jp
http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan

Re: DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦ [ In reply to ]

renayama19661014 at ybb

Feb 16, 2015, 7:58 PM

Post #4 of 7 (4929 views)

Permalink

Re: DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦ [ In reply to ]

mayama0130 at gmail

Feb 16, 2015, 9:08 PM

Post #5 of 7 (4903 views)

Permalink

å±±å†…æ§˜

ã‚ãã‚„ã¾ã§ã™ã€‚

ã”è¿”ä¿¡ã‚ã‚ŠãŒã¨ã†ã”ã–ã„ã¾ã™ã€‚

> ã¡ãªã¿ã«ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã®è² è·ãŒé«˜ã„å¯èƒ½æ€§ã‚‚ã‚ã‚Šã¾ã™ã®ã§ã€
> sarã‚„vmstatã§cpuã®çŠ¶æ…‹ã‚‚è¦‹ã¦ã¿ãŸæ–¹ãŒã‚ˆã„ã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
> ï¼ƒã‚ã‚‹æ™‚ã«ã€ä½•ã‹ã®cronã‚¸ãƒ§ãƒ–ãªã©ã§è² è·ãŒã‚ãŒã£ãŸã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
å½“è©²æ™‚é–“ä»–ã®ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®å‡¦ç†ãŒä»–ã®æ™‚é–“å¸¯ã«æ¯”ã¹ã¦è² è·ãŒé«˜ã‹ã£ãŸã®ã§ã€
ãã®å¯èƒ½æ€§ã‚‚å«ã‚ã€çŠ¶æ…‹ã®ç¢ºèªã‚’ã—ã¦ã¿ãŸã„ã¨æ€ã„ã¾ã™ã€‚

> corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ
tokenå€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆå€¤ï¼ˆ1000msï¼‰ã®ã¾ã¾ã§ã—ãŸã€‚
å¤‰æ›´ã™ã‚‹å ´åˆã§ã™ãŒã€æŽ¨å¥¨å€¤ã¯ã‚ã‚‹ã®ã§ã—ã‚‡ã†ã‹ï¼Ÿ
ã‚‚ã—ãã¯ç®—å‡ºæ–¹æ³•ç‰ã‚ã‚Œã°ã€ã”æ•™æŽˆé ‚ã‘ã‚‹ã¨åŠ©ã‹ã‚Šã¾ã™ã€‚

2015-02-17 12:58 GMT+09:00 <renayama19661014@ybb.ne.jp>:

> ã‚ãã‚„ã¾ã•ã‚“
>
> å±±å†…ã§ã™ã€‚
>
> ã¡ãªã¿ã«ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã®è² è·ãŒé«˜ã„å¯èƒ½æ€§ã‚‚ã‚ã‚Šã¾ã™ã®ã§ã€
> sarã‚„vmstatã§cpuã®çŠ¶æ…‹ã‚‚è¦‹ã¦ã¿ãŸæ–¹ãŒã‚ˆã„ã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
>
> ï¼ƒã‚ã‚‹æ™‚ã«ã€ä½•ã‹ã®cronã‚¸ãƒ§ãƒ–ãªã©ã§è² è·ãŒã‚ãŒã£ãŸã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
>
> ä»¥ä¸Šã§ã™ã€‚
>
>
> ----- Original Message -----
> > From: "renayama19661014@ybb.ne.jp" <renayama19661014@ybb.ne.jp>
> > To: "linux-ha-japan@lists.sourceforge.jp" <
> linux-ha-japan@lists.sourceforge.jp>
> > Cc:
> > Date: 2015/2/17, Tue 12:31
> > Subject: Re: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
> >
> > ã‚ãã‚„ã¾ã•ã‚“
> >
> > ã“ã‚“ã«ã¡ã¯ã€å±±å†…ã§ã™ã€‚
> >
> > corosyncã®ãƒ—ãƒ©ã‚¤ãƒžãƒªãƒã‚°ã‚’è¦‹ã‚‹é™ã‚Šã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã¨ã®corosyncå±¤ã®é€šä¿¡ãŒåˆ‡ã‚Œã¦ã€ä¸€æ—¦ã€å†æ§‹æˆãŒã‹ã‹ã£ãŸã‚ˆã†ã§ã™ã€‚
> >
> > corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ
> >
> >> Feb 16 18:00:27 server1 corosync[1756]: [MAIN ] main.c:1341 Corosync
> main
> > process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms).
> Consider
> > token timeout increase.
> >
> >
> > ä¸Šè¨˜ã®ãƒã‚°ãŒå‡ºã¦ã„ã¾ã™ã®ã§ã€æœ¬æ¥ã®corosyncã®ã‚¹ã‚±ã‚¸ãƒ¥ãƒ¼ãƒªãƒ³ã‚°ãŒï¼’ç§’å¼·ã€é…ã‚Œã¦ã„ã‚‹çŠ¶æ…‹ãŒé€šçŸ¥ã•ã‚Œã¦ã¾ã™ã€‚
> >
> > ä»¥ä¸Šã€å®œã—ããŠé¡˜ã„ã„ãŸã—ã¾ã™ã€‚
> >
> >
> > ----- Original Message -----
> >> From: ã‚ãã‚„ã¾ã˜ã‚ã† <mayama0130@gmail.com>
> >> To: linux-ha-japan@lists.sourceforge.jp
> >> Date: 2015/2/17, Tue 11:38
> >> Subject: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
> >>
> >>
> >> ãŠä¸–è©±ã«ãªã‚Šã¾ã™ã€‚ã‚ãã‚„ã¾ã¨ç”³ã—ã¾ã™ã€‚
> >>
> >> ä¸‹è¨˜ã®ç’°å¢ƒã§é‹ç”¨ã‚’ã—ã¦ã„ã¾ã™ã€‚
> >>
> >> ç’°å¢ƒï¼š
> >> Red Hat Enterprise Linux Server 6.1 (for x86)
> >> kernel-2.6.32-431.23.3.el6.i686
> >> corosync-1.4.1-17.el6_5.1.i686
> >> pacemaker-1.1.10-14.el6_5.3.i686
> >> drbd-8.3.16-1.el6.i386
> >>
> >> è³ªå•å†…å®¹ï¼š
> >> é‹ç”¨ä¸ã«çªç„¶DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ãŒç™ºç”Ÿã—ã€æ‰‹å‹•ã§å¾©æ—§ã•ã›ã¾ã—ãŸãŒã€
> >> ç™ºç”ŸåŽŸå› ãŒä¸æ˜Žã§å›°ã£ã¦ã„ã¾ã™ã€‚
> >> ç™ºç”ŸåŽŸå› ã¨å›žé¿ç–ãŒåˆ†ã‹ã‚Œã°ã€ã”æ•™æŽˆé¡˜ãˆã¾ã™ã§ã—ã‚‡ã†ã‹ã€‚
> >>
> >> ç¾åœ¨ã¾ã§ã«åˆ¤æ˜Žã—ã¦ã„ã‚‹äº‹ï¼š
> >> ãƒ»DRBDãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã®pingå¿œç”ã¯å•é¡Œãªã—
> >> ãƒ»messagesãƒã‚°ã«ãƒªãƒ³ã‚¯ãƒ€ã‚¦ãƒ³ã®å‡ºåŠ›ãªã—
> >>
> >> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®messagesãƒã‚°ã€‘
> >> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
> >> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown
> )
> > conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
> >> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting
> header on
> > sock: r=-110
> >> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
> > 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
> >> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
> >> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
> >> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
> >> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
> > Unconnected )
> >> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
> >> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
> >> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
> >> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
> > WFConnection )
> >> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
> > from=::ffff:192.171.1.102
> >> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root:
> cmd='echo
> > $(date +%Y/%m/%d\ %H:%M:%S)'
> >> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
> > duration=0(sec)
> >> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful:
> Agreed
> > network protocol version 97
> >> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
> > WFReportParams )
> >> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread
> (from
> > drbd0_receiver [3117])
> >> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
> > <not-used>
> >> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down
> by
> > peer.
> >> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
> > NetworkFailure )
> >> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
> >> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
> >> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
> >> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
> > Unconnected )
> >> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
> >> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
> >> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
> >> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
> > WFConnection )
> >>
> >> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®corosyncãƒã‚°ã€‘
> >> Feb 16 18:00:27 server1 corosync[1756]: [MAIN ] main.c:1341 Corosync
> main
> > process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms).
> Consider
> > token timeout increase.
> >> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Transitional membership event on ring 16: memb=1,
> new=0,
> > lost=1
> >> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:773 info:
> > pcmk_peer_update: memb: server1 1694608064
> >> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:779 info:
> > pcmk_peer_update: lost: server2 1711385280
> >> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1,
> lost=0
> >> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:791 info:
> > pcmk_peer_update: NEW: server2 1711385280
> >> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server1 1694608064
> >> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server2 1711385280
> >> Feb 16 18:00:27 server1 corosync[1756]: [TOTEM ] totemsrp.c:1892 A
> > processor joined or left the membership and a new membership was formed.
> >> Feb 16 18:00:27 server1 corosync[1756]: [CPG ] cpg.c:783 chosen
> > downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
> >> Feb 16 18:00:28 server1 corosync[1756]: [MAIN ] main.c:1341 Corosync
> main
> > process was not scheduled for 826.4143 ms (threshold is 800.0000 ms).
> Consider
> > token timeout increase.
> >> Feb 16 18:00:28 server1 corosync[1756]: [MAIN ] main.c:278 Completed
> > service synchronization, ready to provide service.
> >> Feb 16 18:00:28 server1 crmd[1766]: warning: crmd_ha_msg_filter:
> Another DC
> > detected: server2 (op=noop)
> >> Feb 16 18:00:28 server1 crmd[1766]: notice: do_state_transition: State
> > transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> > origin=crmd_ha_msg_filter ]
> >> Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> > requesting full refresh in R/W mode
> >> Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> > requesting full refresh in R/W mode
> >> Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> > requesting full refresh in R/W mode
> >> Feb 16 18:00:30 server1 corosync[1756]: [MAIN ] main.c:1341 Corosync
> main
> > process was not scheduled for 2248.0801 ms (threshold is 800.0000 ms).
> Consider
> > token timeout increase.
> >> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Transitional membership event on ring 24: memb=1,
> new=0,
> > lost=1
> >> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:773 info:
> > pcmk_peer_update: memb: server1 1694608064
> >> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:779 info:
> > pcmk_peer_update: lost: server2 1711385280
> >> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1,
> lost=0
> >> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:791 info:
> > pcmk_peer_update: NEW: server2 1711385280
> >> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server1 1694608064
> >> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server2 1711385280
> >> Feb 16 18:00:31 server1 corosync[1756]: [TOTEM ] totemsrp.c:1892 A
> > processor joined or left the membership and a new membership was formed.
> >> Feb 16 18:00:32 server1 corosync[1756]: [CPG ] cpg.c:783 chosen
> > downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
> >> Feb 16 18:00:32 server1 corosync[1756]: [MAIN ] main.c:278 Completed
> > service synchronization, ready to provide service.
> >> Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> > requesting full refresh in R/W mode
> >> Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> > requesting full refresh in R/W mode
> >> Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> > requesting full refresh in R/W mode
> >> Feb 16 18:02:28 server1 crmd[1766]: error: crm_timer_popped: Election
> > Timeout (I_ELECTION_DC) just popped in state S_ELECTION! (120000ms)
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: do_state_transition: State
> > transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> > cause=C_TIMER_POPPED origin=crm_timer_popped ]
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: do_state_transition: State
> > transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> > cause=C_FSA_INTERNAL origin=do_election_check ]
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: do_dc_join_finalize:
> join-4:
> > Syncing the CIB from server2 to the rest of the cluster
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: do_dc_join_finalize:
> Requested
> > version <generation_tuple epoch="7" num_updates="87"
> > admin_epoch="0" validate-with="pacemaker-1.2"
> > cib-last-written="Wed Jan 21 00:17:56 2015"
> > update-origin="server1" update-client="crmd"
> > crm_feature_set="3.0.7" have-quorum="1"
> > dc-uuid="server2"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: Diff: --- 0.7.69
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: Diff: +++ 0.7.87
> > 1afd9af32d3a574358543e1d4751aaf1
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> <node_state
> > crm-debug-origin="do_state_transition" join="member"
> > id="server1"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> > <lrm_rsc_op operation_key="res_drbd_r0_start_0"
> > operation="start"
> > transition-key="46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> > transition-magic="0:0;46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> > call-id="44" last-run="1421767134"
> > last-rc-change="1421767134" exec-time="430"
> > id="res_drbd_r0_last_0"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> > <lrm_rsc_op id="res_drbd_r0_monitor_30000"
> > operation_key="res_drbd_r0_monitor_30000"
> > operation="monitor" crm-debug-origin="do_update_resource"
> > crm_feature_set="3.0.7"
> > transition-key="47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> > transition-magic="0:0;47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> > call-id="50" rc-code="0" op-status="0"
> > interval="30000" last-rc-change="1421767135"
> > exec-time="60" queue-time="0"
> > op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> > <nvpair value="1000"
> > id="status-server2-master-res_drbd_r0"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> <node_state
> > id="server1" uname="server1" in_ccm="true"
> > crmd="online" crm-debug-origin="post_cache_update"
> > join="down" expected="member"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> > <nvpair id="status-server2-master-res_drbd_r0"
> > name="master-res_drbd_r0" value="10000"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> > <lrm_rsc_op id="res_drbd_r0_last_0"
> > operation_key="res_drbd_r0_promote_0" operation="promote"
> > crm-debug-origin="do_update_resource"
> > crm_feature_set="3.0.7"
> > transition-key="14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> > transition-magic="0:0;14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> > call-id="62" rc-code="0" op-status="0"
> > interval="0" last-run="1424077226"
> > last-rc-change="1424077226" exec-time="120"
> > queue-time="0"
> > op-digest="c0e018b73fdf522b6cdd355e125af15e"/>
> >> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> > <lrm_rsc_op id="res_drbd_r0_monitor_10000"
> > operation_key="res_drbd_r0_monitor_10000"
> > operation="monitor" crm-debug-origin="do_update_resource"
> > crm_feature_set="3.0.7"
> > transition-key="15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> > transition-magic="0:8;15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> > call-id="68" rc-code="8" op-status="0"
> > interval="10000" last-rc-change="1424077226"
> > exec-time="53" queue-time="0"
> > op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
> >> Feb 16 18:02:28 server1 stonith-ng[1762]: warning: cib_process_diff:
> Diff
> > 0.7.69 -> 0.7.87 from local not applied to 0.7.69: Failed application of
> an
> > update diff
> >> Feb 16 18:02:28 server1 stonith-ng[1762]: notice: update_cib_cache_cb:
> > [cib_diff_notify] Patch aborted: Application of an update diff failed
> (-206)
> >> Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_local_callback:
> Sending
> > full refresh (origin=crmd)
> >> Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (10000)
> >> Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: probe_complete (true)
> >> Feb 16 18:02:28 server1 pengine[1765]: notice: unpack_config: On loss
> of
> > CCM Quorum: Ignore
> >> Feb 16 18:02:28 server1 pengine[1765]: notice: LogActions: Demote
> > res_drbd_r0:1#011(Master -> Slave server2)
> >> Feb 16 18:02:28 server1 pengine[1765]: notice: process_pe_message:
> > Calculated Transition 2582: /var/lib/pacemaker/pengine/pe-input-24.bz2
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 8: cancel res_drbd_r0_cancel_10000 on server2
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 81: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 83: notify res_drbd_r0_pre_notify_demote_0 on server2
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=113, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: run_graph: Transition 2582
> > (Complete=4, Pending=0, Fired=0, Skipped=4, Incomplete=5,
> > Source=/var/lib/pacemaker/pengine/pe-input-24.bz2): Stopped
> >> Feb 16 18:02:28 server1 pengine[1765]: notice: unpack_config: On loss
> of
> > CCM Quorum: Ignore
> >> Feb 16 18:02:28 server1 pengine[1765]: notice: LogActions: Demote
> > res_drbd_r0:1#011(Master -> Slave server2)
> >> Feb 16 18:02:28 server1 pengine[1765]: notice: process_pe_message:
> > Calculated Transition 2583: /var/lib/pacemaker/pengine/pe-input-25.bz2
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 80: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
> >> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 82: notify res_drbd_r0_pre_notify_demote_0 on server2
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=116, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 39: demote res_drbd_r0_demote_0 on server2
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 81: notify res_drbd_r0_post_notify_demote_0 on server1 (local)
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 83: notify res_drbd_r0_post_notify_demote_0 on server2
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=119, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: run_graph: Transition 2583
> > (Complete=11, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> > Source=/var/lib/pacemaker/pengine/pe-input-25.bz2): Stopped
> >> Feb 16 18:02:29 server1 pengine[1765]: notice: unpack_config: On loss
> of
> > CCM Quorum: Ignore
> >> Feb 16 18:02:29 server1 pengine[1765]: notice: process_pe_message:
> > Calculated Transition 2584: /var/lib/pacemaker/pengine/pe-input-26.bz2
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command: Initiating
> > action 40: monitor res_drbd_r0_monitor_30000 on server2
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: run_graph: Transition 2584
> > (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> > Source=/var/lib/pacemaker/pengine/pe-input-26.bz2): Complete
> >> Feb 16 18:02:29 server1 crmd[1766]: notice: do_state_transition: State
> > transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> > cause=C_FSA_INTERNAL origin=notify_crmd ]
> >> Feb 16 18:17:29 server1 crmd[1766]: notice: do_state_transition: State
> > transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_TIMER_POPPED
> > origin=crm_timer_popped ]
> >> Feb 16 18:17:29 server1 pengine[1765]: notice: unpack_config: On loss
> of
> > CCM Quorum: Ignore
> >> Feb 16 18:17:29 server1 pengine[1765]: notice: process_pe_message:
> > Calculated Transition 2585: /var/lib/pacemaker/pengine/pe-input-27.bz2
> >> Feb 16 18:17:29 server1 crmd[1766]: notice: run_graph: Transition 2585
> > (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> > Source=/var/lib/pacemaker/pengine/pe-input-27.bz2): Complete
> >> Feb 16 18:17:29 server1 crmd[1766]: notice: do_state_transition: State
> > transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> > cause=C_FSA_INTERNAL origin=notify_crmd ]
> >>
> >> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®messagesãƒã‚°ã€‘
> >> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
> >> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown
> )
> > conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
> >> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting
> header on
> > sock: r=-110
> >> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
> > 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
> >> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
> >> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
> >> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
> >> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
> > Unconnected )
> >> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
> >> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
> >> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
> >> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
> > WFConnection )
> >> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
> > from=::ffff:192.171.1.102
> >> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root:
> cmd='echo
> > $(date +%Y/%m/%d\ %H:%M:%S)'
> >> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
> > duration=0(sec)
> >> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful:
> Agreed
> > network protocol version 97
> >> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
> > WFReportParams )
> >> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread
> (from
> > drbd0_receiver [3117])
> >> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
> > <not-used>
> >> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down
> by
> > peer.
> >> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
> > NetworkFailure )
> >> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
> >> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
> >> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
> >> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
> > Unconnected )
> >> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
> >> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
> >> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
> >> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
> > WFConnection )
> >>
> >> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®corosyncãƒã‚°ã€‘
> >> Feb 16 18:00:23 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (1000)
> >> Feb 16 18:00:23 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> > update 45: master-res_drbd_r0=1000
> >> Feb 16 18:00:24 server2 corosync[1549]: [TOTEM ] totemsrp.c:1595 A
> > processor failed, forming new configuration.
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Transitional membership event on ring 12: memb=1,
> new=0,
> > lost=1
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> > pcmk_peer_update: memb: server2 1711385280
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:779 info:
> > pcmk_peer_update: lost: server1 1694608064
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Stable membership event on ring 12: memb=1, new=0,
> lost=0
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server2 1711385280
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:731 info:
> > ais_mark_unseen_peer_dead: Node server1 was not seen in the previous
> transition
> >> Feb 16 18:00:25 server2 cib[1554]: notice: plugin_handle_membership:
> > Membership 12: quorum lost
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> > update_member: Node 1694608064/server1 is now: lost
> >> Feb 16 18:00:25 server2 crmd[1559]: notice: plugin_handle_membership:
> > Membership 12: quorum lost
> >> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> > send_member_notification: Sending membership update 12 to 2 children
> >> Feb 16 18:00:25 server2 cib[1554]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> > member)
> >> Feb 16 18:00:25 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> > processor joined or left the membership and a new membership was formed.
> >> Feb 16 18:00:25 server2 crmd[1559]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> > member)
> >> Feb 16 18:00:25 server2 crmd[1559]: warning: reap_dead_nodes: Our DC
> node
> > (server1) left the cluster
> >> Feb 16 18:00:25 server2 crmd[1559]: notice: do_state_transition: State
> > transition S_NOT_DC -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> > origin=reap_dead_nodes ]
> >> Feb 16 18:00:25 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> > downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
> >> Feb 16 18:00:25 server2 corosync[1549]: [MAIN ] main.c:278 Completed
> > service synchronization, ready to provide service.
> >> Feb 16 18:00:25 server2 crmd[1559]: notice: do_state_transition: State
> > transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> > cause=C_FSA_INTERNAL origin=do_election_check ]
> >> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_local_callback:
> Sending
> > full refresh (origin=crmd)
> >> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (1000)
> >> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> > update 47: master-res_drbd_r0=1000
> >> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: probe_complete (true)
> >> Feb 16 18:00:26 server2 pengine[1558]: notice: unpack_config: On loss
> of
> > CCM Quorum: Ignore
> >> Feb 16 18:00:26 server2 pengine[1558]: notice: LogActions: Promote
> > res_drbd_r0:0#011(Slave -> Master server2)
> >> Feb 16 18:00:26 server2 pengine[1558]: notice: process_pe_message:
> > Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-0.bz2
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
> > action 1: cancel res_drbd_r0_cancel_30000 on server2 (local)
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
> > action 56: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=56, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: run_graph: Transition 0
> > (Complete=3, Pending=0, Fired=0, Skipped=5, Incomplete=4,
> > Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Stopped
> >> Feb 16 18:00:26 server2 pengine[1558]: notice: unpack_config: On loss
> of
> > CCM Quorum: Ignore
> >> Feb 16 18:00:26 server2 pengine[1558]: notice: LogActions: Promote
> > res_drbd_r0:0#011(Slave -> Master server2)
> >> Feb 16 18:00:26 server2 pengine[1558]: notice: process_pe_message:
> > Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-1.bz2
> >> Feb 16 18:00:26 server2 crmd[1559]: warning: destroy_action: Cancelling
> > timer for action 1 (src=45)
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
> > action 55: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Transitional membership event on ring 16: memb=1,
> new=0,
> > lost=0
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> > pcmk_peer_update: memb: server2 1711385280
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1,
> lost=0
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> > update_member: Node 1694608064/server1 is now: member
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:791 info:
> > pcmk_peer_update: NEW: server1 1694608064
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server1 1694608064
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server2 1711385280
> >> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> > send_member_notification: Sending membership update 16 to 2 children
> >> Feb 16 18:00:26 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> > processor joined or left the membership and a new membership was formed.
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: plugin_handle_membership:
> > Membership 16: quorum acquired
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now member
> (was
> > lost)
> >> Feb 16 18:00:26 server2 cib[1554]: notice: plugin_handle_membership:
> > Membership 16: quorum acquired
> >> Feb 16 18:00:26 server2 cib[1554]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now member
> (was
> > lost)
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=59, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
> > action 14: promote res_drbd_r0_promote_0 on server2 (local)
> >> Feb 16 18:00:26 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> > downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_promote_0 (call=62, rc=0, cib-update=49,
> confirmed=true)
> > ok
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
> > action 56: notify res_drbd_r0_post_notify_promote_0 on server2 (local)
> >> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (10000)
> >> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> > update 51: master-res_drbd_r0=10000
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=65, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command: Initiating
> > action 15: monitor res_drbd_r0_monitor_10000 on server2 (local)
> >> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (10000)
> >> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> > update 53: master-res_drbd_r0=10000
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_monitor_10000 (call=68, rc=8, cib-update=50,
> > confirmed=false) master
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event:
> > server2-res_drbd_r0_monitor_10000:68 [ \n ]
> >> Feb 16 18:00:26 server2 crmd[1559]: notice: run_graph: Transition 1
> > (Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> > Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
> >> Feb 16 18:00:27 server2 corosync[1549]: [MAIN ] main.c:278 Completed
> > service synchronization, ready to provide service.
> >> Feb 16 18:00:28 server2 corosync[1549]: [TOTEM ] totemsrp.c:1595 A
> > processor failed, forming new configuration.
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Transitional membership event on ring 20: memb=1,
> new=0,
> > lost=1
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> > pcmk_peer_update: memb: server2 1711385280
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:779 info:
> > pcmk_peer_update: lost: server1 1694608064
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Stable membership event on ring 20: memb=1, new=0,
> lost=0
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server2 1711385280
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:731 info:
> > ais_mark_unseen_peer_dead: Node server1 was not seen in the previous
> transition
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> > update_member: Node 1694608064/server1 is now: lost
> >> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> > send_member_notification: Sending membership update 20 to 2 children
> >> Feb 16 18:00:29 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> > processor joined or left the membership and a new membership was formed.
> >> Feb 16 18:00:29 server2 cib[1554]: notice: plugin_handle_membership:
> > Membership 20: quorum lost
> >> Feb 16 18:00:29 server2 crmd[1559]: notice: plugin_handle_membership:
> > Membership 20: quorum lost
> >> Feb 16 18:00:29 server2 cib[1554]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> > member)
> >> Feb 16 18:00:29 server2 crmd[1559]: warning: match_down_event: No
> match for
> > shutdown action on server1
> >> Feb 16 18:00:29 server2 crmd[1559]: notice: peer_update_callback:
> > Stonith/shutdown of server1 not matched
> >> Feb 16 18:00:29 server2 crmd[1559]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> > member)
> >> Feb 16 18:00:29 server2 crmd[1559]: warning: match_down_event: No
> match for
> > shutdown action on server1
> >> Feb 16 18:00:29 server2 crmd[1559]: notice: peer_update_callback:
> > Stonith/shutdown of server1 not matched
> >> Feb 16 18:00:29 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> > downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
> >> Feb 16 18:00:29 server2 corosync[1549]: [MAIN ] main.c:278 Completed
> > service synchronization, ready to provide service.
> >> Feb 16 18:00:29 server2 crmd[1559]: notice: do_election_count_vote:
> > Election 4 (current: 2, owner: server1): Processed vote from server1
> (Peer is
> > not part of our cluster)
> >> Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_local_callback:
> Sending
> > full refresh (origin=crmd)
> >> Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (10000)
> >> Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: probe_complete (true)
> >> Feb 16 18:00:29 server2 pengine[1558]: notice: unpack_config: On loss
> of
> > CCM Quorum: Ignore
> >> Feb 16 18:00:29 server2 pengine[1558]: notice: process_pe_message:
> > Calculated Transition 2: /var/lib/pacemaker/pengine/pe-input-2.bz2
> >> Feb 16 18:00:29 server2 crmd[1559]: notice: run_graph: Transition 2
> > (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> > Source=/var/lib/pacemaker/pengine/pe-input-2.bz2): Complete
> >> Feb 16 18:00:29 server2 crmd[1559]: notice: do_state_transition: State
> > transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> > cause=C_FSA_INTERNAL origin=notify_crmd ]
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Transitional membership event on ring 24: memb=1,
> new=0,
> > lost=0
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> > pcmk_peer_update: memb: server2 1711385280
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:760 notice:
> > pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1,
> lost=0
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> > update_member: Node 1694608064/server1 is now: member
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:791 info:
> > pcmk_peer_update: NEW: server1 1694608064
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server1 1694608064
> >> Feb 16 18:00:30 server2 crmd[1559]: notice: plugin_handle_membership:
> > Membership 24: quorum acquired
> >> Feb 16 18:00:30 server2 cib[1554]: notice: plugin_handle_membership:
> > Membership 24: quorum acquired
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> > pcmk_peer_update: MEMB: server2 1711385280
> >> Feb 16 18:00:30 server2 cib[1554]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now member
> (was
> > lost)
> >> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> > send_member_notification: Sending membership update 24 to 2 children
> >> Feb 16 18:00:30 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> > processor joined or left the membership and a new membership was formed.
> >> Feb 16 18:00:30 server2 crmd[1559]: notice: crm_update_peer_state:
> > plugin_handle_membership: Node server1[1694608064] - state is now member
> (was
> > lost)
> >> Feb 16 18:00:30 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> > downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
> >> Feb 16 18:00:31 server2 corosync[1549]: [MAIN ] main.c:278 Completed
> > service synchronization, ready to provide service.
> >> Feb 16 18:00:31 server2 attrd[1557]: warning: attrd_cib_callback:
> Update 45
> > for master-res_drbd_r0=1000 failed: Application of an update diff failed
> >> Feb 16 18:00:56 server2 crmd[1559]: warning: cib_rsc_callback: Resource
> > update 49 failed: (rc=-62) Timer expired
> >> Feb 16 18:00:56 server2 crmd[1559]: warning: cib_rsc_callback: Resource
> > update 50 failed: (rc=-62) Timer expired
> >> Feb 16 18:02:26 server2 attrd[1557]: warning: attrd_cib_callback:
> Update 51
> > for master-res_drbd_r0=10000 failed: Timer expired
> >> Feb 16 18:02:27 server2 crmd[1559]: warning: crmd_ha_msg_filter:
> Another DC
> > detected: server1 (op=join_offer)
> >> Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition: State
> > transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> > origin=crmd_ha_msg_filter ]
> >> Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition: State
> > transition S_ELECTION -> S_RELEASE_DC [ input=I_RELEASE_DC
> > cause=C_FSA_INTERNAL origin=do_election_count_vote ]
> >> Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition: State
> > transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE
> > origin=do_cl_join_finalize_respond ]
> >> Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_local_callback:
> Sending
> > full refresh (origin=crmd)
> >> Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (10000)
> >> Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: probe_complete (true)
> >> Feb 16 18:02:27 server2 crmd[1559]: notice: process_lrm_event:
> > server2-res_drbd_r0_monitor_10000:68 [ \n ]
> >> Feb 16 18:02:27 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=74, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=77, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_demote_0 (call=80, rc=0, cib-update=74,
> confirmed=true) ok
> >> Feb 16 18:02:28 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> > flush op to all hosts for: master-res_drbd_r0 (1000)
> >> Feb 16 18:02:28 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> > update 67: master-res_drbd_r0=1000
> >> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_notify_0 (call=83, rc=0, cib-update=0,
> confirmed=true) ok
> >> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> > operation res_drbd_r0_monitor_30000 (call=86, rc=0, cib-update=75,
> > confirmed=false) ok
> >>
> >> ã€global_common.confã€‘
> >> global {
> >> usage-count no;
> >> }
> >>
> >> common {
> >> protocol C;
> >> handlers {
> >> }
> >> startup {
> >> }
> >> disk {
> >> no-disk-barrier;
> >> no-disk-flushes;
> >> }
> >> net {
> >> max-buffers 8000;
> >> max-epoch-size 8000;
> >> after-sb-0pri discard-younger-primary;
> >> after-sb-1pri discard-secondary;
> >> after-sb-2pri disconnect;
> >> }
> >> syncer {
> >> rate 30M;
> >> verify-alg sha1;
> >> }
> >> }
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Linux-ha-japan mailing list
> >> Linux-ha-japan@lists.sourceforge.jp
> >> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
> >>
> >>
> >>
> >
> > _______________________________________________
> > Linux-ha-japan mailing list
> > Linux-ha-japan@lists.sourceforge.jp
> > http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
> >
>
> _______________________________________________
> Linux-ha-japan mailing list
> Linux-ha-japan@lists.sourceforge.jp
> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>

Re: DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦ [ In reply to ]

renayama19661014 at ybb

Feb 16, 2015, 9:35 PM

Post #6 of 7 (5064 views)

Permalink

ã‚ãã‚„ã¾ã•ã‚“

ã“ã‚“ã«ã¡ã¯ã€å±±å†…ã§ã™ã€‚

>tokenå€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆå€¤ï¼ˆ1000msï¼‰ã®ã¾ã¾ã§ã—ãŸã€‚
>å¤‰æ›´ã™ã‚‹å ´åˆã§ã™ãŒã€æŽ¨å¥¨å€¤ã¯ã‚ã‚‹ã®ã§ã—ã‚‡ã†ã‹ï¼Ÿ
>ã‚‚ã—ãã¯ç®—å‡ºæ–¹æ³•ç‰ã‚ã‚Œã°ã€ã”æ•™æŽˆé ‚ã‘ã‚‹ã¨åŠ©ã‹ã‚Šã¾ã™ã€‚

corosync+PMã®æŽ¨å¥¨å€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã®1000msã ã£ãŸã¨è¨˜æ†¶ã—ã¦ã„ã¾ã™ã€‚
å¤‰æ›´ã™ã‚‹å ´åˆã®å€¤ã¯ã€æ§‹æˆï¼ˆè² è·çŠ¶æ³ãªã©ï¼‰ã«ã‚ˆã‚Šã¾ã™ã®ã§ã€å€ç¨‹åº¦ã‹ã‚‰ã‚ã’ã¦ã¿ã¦ã¯å¦‚ä½•ã§ã—ã‚‡ã†ã‹ï¼Ÿ

tokenã‚’ä¼¸ã°ã™å ´åˆã®ãƒ‡ãƒ¡ãƒªãƒƒãƒˆã¯ã€æ•…éšœæ¤œçŸ¥ï¼ˆç›¸æ‰‹ã®ãƒ€ã‚¦ãƒ³ï¼‰æ™‚é–“ãŒé•·ããªã‚‹ã“ã¨ã«ãªã‚Šã¾ã™ã€‚

ä»¥ä¸Šã€å®œã—ããŠé¡˜ã„ã„ãŸã—ã¾ã™ã€‚

----- Original Message -----
>From: ã‚ãã‚„ã¾ã˜ã‚ã† <mayama0130@gmail.com>
>To: linux-ha-japan@lists.sourceforge.jp
>Date: 2015/2/17, Tue 14:08
>Subject: Re: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
>
>
>å±±å†…æ§˜
>
>ã‚ãã‚„ã¾ã§ã™ã€‚
>
>ã”è¿”ä¿¡ã‚ã‚ŠãŒã¨ã†ã”ã–ã„ã¾ã™ã€‚
>
>> ã¡ãªã¿ã«ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã®è² è·ãŒé«˜ã„å¯èƒ½æ€§ã‚‚ã‚ã‚Šã¾ã™ã®ã§ã€
>> sarã‚„vmstatã§cpuã®çŠ¶æ…‹ã‚‚è¦‹ã¦ã¿ãŸæ–¹ãŒã‚ˆã„ã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
>> ï¼ƒã‚ã‚‹æ™‚ã«ã€ä½•ã‹ã®cronã‚¸ãƒ§ãƒ–ãªã©ã§è² è·ãŒã‚ãŒã£ãŸã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
>å½“è©²æ™‚é–“ä»–ã®ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®å‡¦ç†ãŒä»–ã®æ™‚é–“å¸¯ã«æ¯”ã¹ã¦è² è·ãŒé«˜ã‹ã£ãŸã®ã§ã€
>ãã®å¯èƒ½æ€§ã‚‚å«ã‚ã€çŠ¶æ…‹ã®ç¢ºèªã‚’ã—ã¦ã¿ãŸã„ã¨æ€ã„ã¾ã™ã€‚
>
>> corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ
>tokenå€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆå€¤ï¼ˆ1000msï¼‰ã®ã¾ã¾ã§ã—ãŸã€‚
>å¤‰æ›´ã™ã‚‹å ´åˆã§ã™ãŒã€æŽ¨å¥¨å€¤ã¯ã‚ã‚‹ã®ã§ã—ã‚‡ã†ã‹ï¼Ÿ
>ã‚‚ã—ãã¯ç®—å‡ºæ–¹æ³•ç‰ã‚ã‚Œã°ã€ã”æ•™æŽˆé ‚ã‘ã‚‹ã¨åŠ©ã‹ã‚Šã¾ã™ã€‚
>
>
>
>2015-02-17 12:58 GMT+09:00 <renayama19661014@ybb.ne.jp>:
>
>ã‚ãã‚„ã¾ã•ã‚“
>>
>>å±±å†…ã§ã™ã€‚
>>
>>ã¡ãªã¿ã«ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã®è² è·ãŒé«˜ã„å¯èƒ½æ€§ã‚‚ã‚ã‚Šã¾ã™ã®ã§ã€
>>sarã‚„vmstatã§cpuã®çŠ¶æ…‹ã‚‚è¦‹ã¦ã¿ãŸæ–¹ãŒã‚ˆã„ã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
>>
>>ï¼ƒã‚ã‚‹æ™‚ã«ã€ä½•ã‹ã®cronã‚¸ãƒ§ãƒ–ãªã©ã§è² è·ãŒã‚ãŒã£ãŸã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
>>
>>ä»¥ä¸Šã§ã™ã€‚
>>
>>
>>
>>----- Original Message -----
>>> From: "renayama19661014@ybb.ne.jp" <renayama19661014@ybb.ne.jp>
>>> To: "linux-ha-japan@lists.sourceforge.jp" <linux-ha-japan@lists.sourceforge.jp>
>>> Cc:
>>> Date: 2015/2/17, Tue 12:31
>>> Subject: Re: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
>>>
>>> ã‚ãã‚„ã¾ã•ã‚“
>>>
>>> ã“ã‚“ã«ã¡ã¯ã€å±±å†…ã§ã™ã€‚
>>>
>>> corosyncã®ãƒ—ãƒ©ã‚¤ãƒžãƒªãƒã‚°ã‚’è¦‹ã‚‹é™ã‚Šã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã¨ã®corosyncå±¤ã®é€šä¿¡ãŒåˆ‡ã‚Œã¦ã€ä¸€æ—¦ã€å†æ§‹æˆãŒã‹ã‹ã£ãŸã‚ˆã†ã§ã™ã€‚
>>>
>>> corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ
>>>
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
>>> process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms). Consider
>>> token timeout increase.
>>>
>>>
>>> ä¸Šè¨˜ã®ãƒã‚°ãŒå‡ºã¦ã„ã¾ã™ã®ã§ã€æœ¬æ¥ã®corosyncã®ã‚¹ã‚±ã‚¸ãƒ¥ãƒ¼ãƒªãƒ³ã‚°ãŒï¼’ç§’å¼·ã€é…ã‚Œã¦ã„ã‚‹çŠ¶æ…‹ãŒé€šçŸ¥ã•ã‚Œã¦ã¾ã™ã€‚
>>>
>>> ä»¥ä¸Šã€å®œã—ããŠé¡˜ã„ã„ãŸã—ã¾ã™ã€‚
>>>
>>>
>>> ----- Original Message -----
>>>> From: ã‚ãã‚„ã¾ã˜ã‚ã† <mayama0130@gmail.com>
>>>> To: linux-ha-japan@lists.sourceforge.jp
>>>> Date: 2015/2/17, Tue 11:38
>>>> Subject: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
>>>>
>>>>
>>>> ãŠä¸–è©±ã«ãªã‚Šã¾ã™ã€‚ã‚ãã‚„ã¾ã¨ç”³ã—ã¾ã™ã€‚
>>>>
>>>> ä¸‹è¨˜ã®ç’°å¢ƒã§é‹ç”¨ã‚’ã—ã¦ã„ã¾ã™ã€‚
>>>>
>>>> ç’°å¢ƒï¼š
>>>> Red Hat Enterprise Linux Server 6.1 (for x86)
>>>> kernel-2.6.32-431.23.3.el6.i686
>>>> corosync-1.4.1-17.el6_5.1.i686
>>>> pacemaker-1.1.10-14.el6_5.3.i686
>>>> drbd-8.3.16-1.el6.i386
>>>>
>>>> è³ªå•å†…å®¹ï¼š
>>>> é‹ç”¨ä¸ã«çªç„¶DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ãŒç™ºç”Ÿã—ã€æ‰‹å‹•ã§å¾©æ—§ã•ã›ã¾ã—ãŸãŒã€
>>>> ç™ºç”ŸåŽŸå› ãŒä¸æ˜Žã§å›°ã£ã¦ã„ã¾ã™ã€‚
>>>> ç™ºç”ŸåŽŸå› ã¨å›žé¿ç–ãŒåˆ†ã‹ã‚Œã°ã€ã”æ•™æŽˆé¡˜ãˆã¾ã™ã§ã—ã‚‡ã†ã‹ã€‚
>>>>
>>>> ç¾åœ¨ã¾ã§ã«åˆ¤æ˜Žã—ã¦ã„ã‚‹äº‹ï¼š
>>>> ãƒ»DRBDãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã®pingå¿œç”ã¯å•é¡Œãªã—
>>>> ãƒ»messagesãƒã‚°ã«ãƒªãƒ³ã‚¯ãƒ€ã‚¦ãƒ³ã®å‡ºåŠ›ãªã—
>>>>
>>>> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®messagesãƒã‚°ã€‘
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown )
>>> conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on
>>> sock: r=-110
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
>>> 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
>>> Unconnected )
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
>>> WFConnection )
>>>> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
>>> from=::ffff:192.171.1.102
>>>> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo
>>> $(date +%Y/%m/%d\ %H:%M:%S)'
>>>> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
>>> duration=0(sec)
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed
>>> network protocol version 97
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
>>> WFReportParams )
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from
>>> drbd0_receiver [3117])
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
>>> <not-used>
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by
>>> peer.
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
>>> NetworkFailure )
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
>>> Unconnected )
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
>>> WFConnection )
>>>>
>>>> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®corosyncãƒã‚°ã€‘
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
>>> process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms). Consider
>>> token timeout increase.
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0,
>>> lost=1
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:773 info:
>>> pcmk_peer_update: memb: server1 1694608064
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:779 info:
>>> pcmk_peer_update: lost: server2 1711385280
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:791 info:
>>> pcmk_peer_update: NEW:Â server2 1711385280
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server1 1694608064
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server2 1711385280
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [TOTEM ] totemsrp.c:1892 A
>>> processor joined or left the membership and a new membership was formed.
>>>> Feb 16 18:00:27 server1 corosync[1756]:Â Â [CPGÂ Â ] cpg.c:783 chosen
>>> downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
>>>> Feb 16 18:00:28 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
>>> process was not scheduled for 826.4143 ms (threshold is 800.0000 ms). Consider
>>> token timeout increase.
>>>> Feb 16 18:00:28 server1 corosync[1756]:Â Â [MAINÂ ] main.c:278 Completed
>>> service synchronization, ready to provide service.
>>>> Feb 16 18:00:28 server1 crmd[1766]:Â warning: crmd_ha_msg_filter: Another DC
>>> detected: server2 (op=noop)
>>>> Feb 16 18:00:28 server1 crmd[1766]:Â Â notice: do_state_transition: State
>>> transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
>>> origin=crmd_ha_msg_filter ]
>>>> Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not
>>> requesting full refresh in R/W mode
>>>> Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not
>>> requesting full refresh in R/W mode
>>>> Feb 16 18:00:30 server1 cib[1761]:Â warning: cib_server_process_diff: Not
>>> requesting full refresh in R/W mode
>>>> Feb 16 18:00:30 server1 corosync[1756]:Â Â [MAINÂ ] main.c:1341 Corosync main
>>> process was not scheduled for 2248.0801 ms (threshold is 800.0000 ms). Consider
>>> token timeout increase.
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0,
>>> lost=1
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:773 info:
>>> pcmk_peer_update: memb: server1 1694608064
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:779 info:
>>> pcmk_peer_update: lost: server2 1711385280
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:791 info:
>>> pcmk_peer_update: NEW:Â server2 1711385280
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server1 1694608064
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server2 1711385280
>>>> Feb 16 18:00:31 server1 corosync[1756]:Â Â [TOTEM ] totemsrp.c:1892 A
>>> processor joined or left the membership and a new membership was formed.
>>>> Feb 16 18:00:32 server1 corosync[1756]:Â Â [CPGÂ Â ] cpg.c:783 chosen
>>> downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
>>>> Feb 16 18:00:32 server1 corosync[1756]:Â Â [MAINÂ ] main.c:278 Completed
>>> service synchronization, ready to provide service.
>>>> Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not
>>> requesting full refresh in R/W mode
>>>> Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not
>>> requesting full refresh in R/W mode
>>>> Feb 16 18:00:32 server1 cib[1761]:Â warning: cib_server_process_diff: Not
>>> requesting full refresh in R/W mode
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â Â error: crm_timer_popped: Election
>>> Timeout (I_ELECTION_DC) just popped in state S_ELECTION! (120000ms)
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_state_transition: State
>>> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
>>> cause=C_TIMER_POPPED origin=crm_timer_popped ]
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_state_transition: State
>>> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
>>> cause=C_FSA_INTERNAL origin=do_election_check ]
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_dc_join_finalize: join-4:
>>> Syncing the CIB from server2 to the rest of the cluster
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: do_dc_join_finalize: Requested
>>> versionÂ Â <generation_tuple epoch="7" num_updates="87"
>>> admin_epoch="0" validate-with="pacemaker-1.2"
>>> cib-last-written="Wed Jan 21 00:17:56 2015"
>>> update-origin="server1" update-client="crmd"
>>> crm_feature_set="3.0.7" have-quorum="1"
>>> dc-uuid="server2"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: Diff: --- 0.7.69
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: Diff: +++ 0.7.87
>>> 1afd9af32d3a574358543e1d4751aaf1
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â <node_state
>>> crm-debug-origin="do_state_transition" join="member"
>>> id="server1"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â Â Â
>>> <lrm_rsc_op operation_key="res_drbd_r0_start_0"
>>> operation="start"
>>> transition-key="46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
>>> transition-magic="0:0;46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
>>> call-id="44" last-run="1421767134"
>>> last-rc-change="1421767134" exec-time="430"
>>> id="res_drbd_r0_last_0"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â Â Â
>>> <lrm_rsc_op id="res_drbd_r0_monitor_30000"
>>> operation_key="res_drbd_r0_monitor_30000"
>>> operation="monitor" crm-debug-origin="do_update_resource"
>>> crm_feature_set="3.0.7"
>>> transition-key="47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
>>> transition-magic="0:0;47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
>>> call-id="50" rc-code="0" op-status="0"
>>> interval="30000" last-rc-change="1421767135"
>>> exec-time="60" queue-time="0"
>>> op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: --Â Â Â Â Â Â Â Â Â Â
>>> <nvpair value="1000"
>>> id="status-server2-master-res_drbd_r0"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â <node_state
>>> id="server1" uname="server1" in_ccm="true"
>>> crmd="online" crm-debug-origin="post_cache_update"
>>> join="down" expected="member"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â
>>> <nvpair id="status-server2-master-res_drbd_r0"
>>> name="master-res_drbd_r0" value="10000"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â Â Â
>>> <lrm_rsc_op id="res_drbd_r0_last_0"
>>> operation_key="res_drbd_r0_promote_0" operation="promote"
>>> crm-debug-origin="do_update_resource"
>>> crm_feature_set="3.0.7"
>>> transition-key="14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
>>> transition-magic="0:0;14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
>>> call-id="62" rc-code="0" op-status="0"
>>> interval="0" last-run="1424077226"
>>> last-rc-change="1424077226" exec-time="120"
>>> queue-time="0"
>>> op-digest="c0e018b73fdf522b6cdd355e125af15e"/>
>>>> Feb 16 18:02:28 server1 cib[1761]:Â Â notice: cib:diff: ++Â Â Â Â Â Â Â Â Â Â Â Â
>>> <lrm_rsc_op id="res_drbd_r0_monitor_10000"
>>> operation_key="res_drbd_r0_monitor_10000"
>>> operation="monitor" crm-debug-origin="do_update_resource"
>>> crm_feature_set="3.0.7"
>>> transition-key="15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
>>> transition-magic="0:8;15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
>>> call-id="68" rc-code="8" op-status="0"
>>> interval="10000" last-rc-change="1424077226"
>>> exec-time="53" queue-time="0"
>>> op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
>>>> Feb 16 18:02:28 server1 stonith-ng[1762]:Â warning: cib_process_diff: Diff
>>> 0.7.69 -> 0.7.87 from local not applied to 0.7.69: Failed application of an
>>> update diff
>>>> Feb 16 18:02:28 server1 stonith-ng[1762]:Â Â notice: update_cib_cache_cb:
>>> [cib_diff_notify] Patch aborted: Application of an update diff failed (-206)
>>>> Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_local_callback: Sending
>>> full refresh (origin=crmd)
>>>> Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (10000)
>>>> Feb 16 18:02:28 server1 attrd[1764]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: probe_complete (true)
>>>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
>>> CCM Quorum: Ignore
>>>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: LogActions: DemoteÂ
>>> res_drbd_r0:1#011(Master -> Slave server2)
>>>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: process_pe_message:
>>> Calculated Transition 2582: /var/lib/pacemaker/pengine/pe-input-24.bz2
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 8: cancel res_drbd_r0_cancel_10000 on server2
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 81: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 83: notify res_drbd_r0_pre_notify_demote_0 on server2
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=113, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: run_graph: Transition 2582
>>> (Complete=4, Pending=0, Fired=0, Skipped=4, Incomplete=5,
>>> Source=/var/lib/pacemaker/pengine/pe-input-24.bz2): Stopped
>>>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
>>> CCM Quorum: Ignore
>>>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: LogActions: DemoteÂ
>>> res_drbd_r0:1#011(Master -> Slave server2)
>>>> Feb 16 18:02:28 server1 pengine[1765]:Â Â notice: process_pe_message:
>>> Calculated Transition 2583: /var/lib/pacemaker/pengine/pe-input-25.bz2
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 80: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
>>>> Feb 16 18:02:28 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 82: notify res_drbd_r0_pre_notify_demote_0 on server2
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=116, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 39: demote res_drbd_r0_demote_0 on server2
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 81: notify res_drbd_r0_post_notify_demote_0 on server1 (local)
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 83: notify res_drbd_r0_post_notify_demote_0 on server2
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=119, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2583
>>> (Complete=11, Pending=0, Fired=0, Skipped=1, Incomplete=0,
>>> Source=/var/lib/pacemaker/pengine/pe-input-25.bz2): Stopped
>>>> Feb 16 18:02:29 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
>>> CCM Quorum: Ignore
>>>> Feb 16 18:02:29 server1 pengine[1765]:Â Â notice: process_pe_message:
>>> Calculated Transition 2584: /var/lib/pacemaker/pengine/pe-input-26.bz2
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: te_rsc_command: Initiating
>>> action 40: monitor res_drbd_r0_monitor_30000 on server2
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2584
>>> (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
>>> Source=/var/lib/pacemaker/pengine/pe-input-26.bz2): Complete
>>>> Feb 16 18:02:29 server1 crmd[1766]:Â Â notice: do_state_transition: State
>>> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
>>> cause=C_FSA_INTERNAL origin=notify_crmd ]
>>>> Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: do_state_transition: State
>>> transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED
>>> origin=crm_timer_popped ]
>>>> Feb 16 18:17:29 server1 pengine[1765]:Â Â notice: unpack_config: On loss of
>>> CCM Quorum: Ignore
>>>> Feb 16 18:17:29 server1 pengine[1765]:Â Â notice: process_pe_message:
>>> Calculated Transition 2585: /var/lib/pacemaker/pengine/pe-input-27.bz2
>>>> Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: run_graph: Transition 2585
>>> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
>>> Source=/var/lib/pacemaker/pengine/pe-input-27.bz2): Complete
>>>> Feb 16 18:17:29 server1 crmd[1766]:Â Â notice: do_state_transition: State
>>> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
>>> cause=C_FSA_INTERNAL origin=notify_crmd ]
>>>>
>>>> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®messagesãƒã‚°ã€‘
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned -110
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary -> Unknown )
>>> conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting header on
>>> sock: r=-110
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
>>> 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
>>> Unconnected )
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
>>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
>>> WFConnection )
>>>> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
>>> from=::ffff:192.171.1.102
>>>> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root: cmd='echo
>>> $(date +%Y/%m/%d\ %H:%M:%S)'
>>>> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
>>> duration=0(sec)
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful: Agreed
>>> network protocol version 97
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
>>> WFReportParams )
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread (from
>>> drbd0_receiver [3117])
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
>>> <not-used>
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut down by
>>> peer.
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
>>> NetworkFailure )
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
>>>> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
>>> Unconnected )
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
>>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
>>> WFConnection )
>>>>
>>>> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®corosyncãƒã‚°ã€‘
>>>> Feb 16 18:00:23 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (1000)
>>>> Feb 16 18:00:23 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
>>> update 45: master-res_drbd_r0=1000
>>>> Feb 16 18:00:24 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1595 A
>>> processor failed, forming new configuration.
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Transitional membership event on ring 12: memb=1, new=0,
>>> lost=1
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
>>> pcmk_peer_update: memb: server2 1711385280
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:779 info:
>>> pcmk_peer_update: lost: server1 1694608064
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Stable membership event on ring 12: memb=1, new=0, lost=0
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server2 1711385280
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:731 info:
>>> ais_mark_unseen_peer_dead: Node server1 was not seen in the previous transition
>>>> Feb 16 18:00:25 server2 cib[1554]:Â Â notice: plugin_handle_membership:
>>> Membership 12: quorum lost
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
>>> update_member: Node 1694608064/server1 is now: lost
>>>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
>>> Membership 12: quorum lost
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
>>> send_member_notification: Sending membership update 12 to 2 children
>>>> Feb 16 18:00:25 server2 cib[1554]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
>>> member)
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
>>> processor joined or left the membership and a new membership was formed.
>>>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
>>> member)
>>>> Feb 16 18:00:25 server2 crmd[1559]:Â warning: reap_dead_nodes: Our DC node
>>> (server1) left the cluster
>>>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: do_state_transition: State
>>> transition S_NOT_DC -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
>>> origin=reap_dead_nodes ]
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
>>> downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
>>>> Feb 16 18:00:25 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
>>> service synchronization, ready to provide service.
>>>> Feb 16 18:00:25 server2 crmd[1559]:Â Â notice: do_state_transition: State
>>> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
>>> cause=C_FSA_INTERNAL origin=do_election_check ]
>>>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending
>>> full refresh (origin=crmd)
>>>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (1000)
>>>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
>>> update 47: master-res_drbd_r0=1000
>>>> Feb 16 18:00:25 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: probe_complete (true)
>>>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: unpack_config: On loss of
>>> CCM Quorum: Ignore
>>>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: LogActions: Promote
>>> res_drbd_r0:0#011(Slave -> Master server2)
>>>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: process_pe_message:
>>> Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-0.bz2
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
>>> action 1: cancel res_drbd_r0_cancel_30000 on server2 (local)
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
>>> action 56: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=56, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: run_graph: Transition 0
>>> (Complete=3, Pending=0, Fired=0, Skipped=5, Incomplete=4,
>>> Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Stopped
>>>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: unpack_config: On loss of
>>> CCM Quorum: Ignore
>>>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: LogActions: Promote
>>> res_drbd_r0:0#011(Slave -> Master server2)
>>>> Feb 16 18:00:26 server2 pengine[1558]:Â Â notice: process_pe_message:
>>> Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-1.bz2
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â warning: destroy_action: Cancelling
>>> timer for action 1 (src=45)
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
>>> action 55: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Transitional membership event on ring 16: memb=1, new=0,
>>> lost=0
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
>>> pcmk_peer_update: memb: server2 1711385280
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1, lost=0
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
>>> update_member: Node 1694608064/server1 is now: member
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:791 info:
>>> pcmk_peer_update: NEW:Â server1 1694608064
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server1 1694608064
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server2 1711385280
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
>>> send_member_notification: Sending membership update 16 to 2 children
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
>>> processor joined or left the membership and a new membership was formed.
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
>>> Membership 16: quorum acquired
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now member (was
>>> lost)
>>>> Feb 16 18:00:26 server2 cib[1554]:Â Â notice: plugin_handle_membership:
>>> Membership 16: quorum acquired
>>>> Feb 16 18:00:26 server2 cib[1554]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now member (was
>>> lost)
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=59, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
>>> action 14: promote res_drbd_r0_promote_0 on server2 (local)
>>>> Feb 16 18:00:26 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
>>> downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_promote_0 (call=62, rc=0, cib-update=49, confirmed=true)
>>> ok
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
>>> action 56: notify res_drbd_r0_post_notify_promote_0 on server2 (local)
>>>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (10000)
>>>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
>>> update 51: master-res_drbd_r0=10000
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=65, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: te_rsc_command: Initiating
>>> action 15: monitor res_drbd_r0_monitor_10000 on server2 (local)
>>>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (10000)
>>>> Feb 16 18:00:26 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
>>> update 53: master-res_drbd_r0=10000
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_monitor_10000 (call=68, rc=8, cib-update=50,
>>> confirmed=false) master
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: process_lrm_event:
>>> server2-res_drbd_r0_monitor_10000:68 [ \n ]
>>>> Feb 16 18:00:26 server2 crmd[1559]:Â Â notice: run_graph: Transition 1
>>> (Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0,
>>> Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
>>>> Feb 16 18:00:27 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
>>> service synchronization, ready to provide service.
>>>> Feb 16 18:00:28 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1595 A
>>> processor failed, forming new configuration.
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Transitional membership event on ring 20: memb=1, new=0,
>>> lost=1
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
>>> pcmk_peer_update: memb: server2 1711385280
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:779 info:
>>> pcmk_peer_update: lost: server1 1694608064
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Stable membership event on ring 20: memb=1, new=0, lost=0
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server2 1711385280
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:731 info:
>>> ais_mark_unseen_peer_dead: Node server1 was not seen in the previous transition
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
>>> update_member: Node 1694608064/server1 is now: lost
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
>>> send_member_notification: Sending membership update 20 to 2 children
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
>>> processor joined or left the membership and a new membership was formed.
>>>> Feb 16 18:00:29 server2 cib[1554]:Â Â notice: plugin_handle_membership:
>>> Membership 20: quorum lost
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
>>> Membership 20: quorum lost
>>>> Feb 16 18:00:29 server2 cib[1554]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
>>> member)
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â warning: match_down_event: No match for
>>> shutdown action on server1
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: peer_update_callback:
>>> Stonith/shutdown of server1 not matched
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now lost (was
>>> member)
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â warning: match_down_event: No match for
>>> shutdown action on server1
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: peer_update_callback:
>>> Stonith/shutdown of server1 not matched
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
>>> downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
>>>> Feb 16 18:00:29 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
>>> service synchronization, ready to provide service.
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: do_election_count_vote:
>>> Election 4 (current: 2, owner: server1): Processed vote from server1 (Peer is
>>> not part of our cluster)
>>>> Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending
>>> full refresh (origin=crmd)
>>>> Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (10000)
>>>> Feb 16 18:00:29 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: probe_complete (true)
>>>> Feb 16 18:00:29 server2 pengine[1558]:Â Â notice: unpack_config: On loss of
>>> CCM Quorum: Ignore
>>>> Feb 16 18:00:29 server2 pengine[1558]:Â Â notice: process_pe_message:
>>> Calculated Transition 2: /var/lib/pacemaker/pengine/pe-input-2.bz2
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: run_graph: Transition 2
>>> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
>>> Source=/var/lib/pacemaker/pengine/pe-input-2.bz2): Complete
>>>> Feb 16 18:00:29 server2 crmd[1559]:Â Â notice: do_state_transition: State
>>> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
>>> cause=C_FSA_INTERNAL origin=notify_crmd ]
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Transitional membership event on ring 24: memb=1, new=0,
>>> lost=0
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:773 info:
>>> pcmk_peer_update: memb: server2 1711385280
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:760 notice:
>>> pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1, lost=0
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] utils.c:358 info:
>>> update_member: Node 1694608064/server1 is now: member
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:791 info:
>>> pcmk_peer_update: NEW:Â server1 1694608064
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server1 1694608064
>>>> Feb 16 18:00:30 server2 crmd[1559]:Â Â notice: plugin_handle_membership:
>>> Membership 24: quorum acquired
>>>> Feb 16 18:00:30 server2 cib[1554]:Â Â notice: plugin_handle_membership:
>>> Membership 24: quorum acquired
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:808 info:
>>> pcmk_peer_update: MEMB: server2 1711385280
>>>> Feb 16 18:00:30 server2 cib[1554]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now member (was
>>> lost)
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [pcmkÂ ] plugin.c:1368 info:
>>> send_member_notification: Sending membership update 24 to 2 children
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [TOTEM ] totemsrp.c:1892 A
>>> processor joined or left the membership and a new membership was formed.
>>>> Feb 16 18:00:30 server2 crmd[1559]:Â Â notice: crm_update_peer_state:
>>> plugin_handle_membership: Node server1[1694608064] - state is now member (was
>>> lost)
>>>> Feb 16 18:00:30 server2 corosync[1549]:Â Â [CPGÂ Â ] cpg.c:783 chosen
>>> downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
>>>> Feb 16 18:00:31 server2 corosync[1549]:Â Â [MAINÂ ] main.c:278 Completed
>>> service synchronization, ready to provide service.
>>>> Feb 16 18:00:31 server2 attrd[1557]:Â warning: attrd_cib_callback: Update 45
>>> for master-res_drbd_r0=1000 failed: Application of an update diff failed
>>>> Feb 16 18:00:56 server2 crmd[1559]:Â warning: cib_rsc_callback: Resource
>>> update 49 failed: (rc=-62) Timer expired
>>>> Feb 16 18:00:56 server2 crmd[1559]:Â warning: cib_rsc_callback: Resource
>>> update 50 failed: (rc=-62) Timer expired
>>>> Feb 16 18:02:26 server2 attrd[1557]:Â warning: attrd_cib_callback: Update 51
>>> for master-res_drbd_r0=10000 failed: Timer expired
>>>> Feb 16 18:02:27 server2 crmd[1559]:Â warning: crmd_ha_msg_filter: Another DC
>>> detected: server1 (op=join_offer)
>>>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State
>>> transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
>>> origin=crmd_ha_msg_filter ]
>>>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State
>>> transition S_ELECTION -> S_RELEASE_DC [ input=I_RELEASE_DC
>>> cause=C_FSA_INTERNAL origin=do_election_count_vote ]
>>>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: do_state_transition: State
>>> transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE
>>> origin=do_cl_join_finalize_respond ]
>>>> Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_local_callback: Sending
>>> full refresh (origin=crmd)
>>>> Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (10000)
>>>> Feb 16 18:02:27 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: probe_complete (true)
>>>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: process_lrm_event:
>>> server2-res_drbd_r0_monitor_10000:68 [ \n ]
>>>> Feb 16 18:02:27 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=74, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=77, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_demote_0 (call=80, rc=0, cib-update=74, confirmed=true) ok
>>>> Feb 16 18:02:28 server2 attrd[1557]:Â Â notice: attrd_trigger_update: Sending
>>> flush op to all hosts for: master-res_drbd_r0 (1000)
>>>> Feb 16 18:02:28 server2 attrd[1557]:Â Â notice: attrd_perform_update: Sent
>>> update 67: master-res_drbd_r0=1000
>>>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_notify_0 (call=83, rc=0, cib-update=0, confirmed=true) ok
>>>> Feb 16 18:02:28 server2 crmd[1559]:Â Â notice: process_lrm_event: LRM
>>> operation res_drbd_r0_monitor_30000 (call=86, rc=0, cib-update=75,
>>> confirmed=false) ok
>>>>
>>>> ã€global_common.confã€‘
>>>> global {
>>>> Â Â Â Â Â Â Â usage-count no;
>>>> }
>>>>
>>>> common {
>>>> Â Â Â Â Â Â Â protocol C;
>>>> Â Â Â Â Â Â Â handlers {
>>>> Â Â Â Â Â Â Â }
>>>> Â Â Â Â Â Â Â startup {
>>>> Â Â Â Â Â Â Â }
>>>> Â Â Â Â Â Â Â disk {
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â no-disk-barrier;
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â no-disk-flushes;
>>>> Â Â Â Â Â Â Â }
>>>> Â Â Â Â Â Â Â net {
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â max-buffers 8000;
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â max-epoch-size 8000;
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-0pri discard-younger-primary;
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-1pri discard-secondary;
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â after-sb-2pri disconnect;
>>>> Â Â Â Â Â Â Â }
>>>> Â Â Â Â Â Â Â syncer {
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â rate 30M;
>>>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â verify-alg sha1;
>>>> Â Â Â Â Â Â Â }
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Linux-ha-japan mailing list
>>>> Linux-ha-japan@lists.sourceforge.jp
>>>> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Linux-ha-japan mailing list
>>> Linux-ha-japan@lists.sourceforge.jp
>>> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>>>
>>
>>_______________________________________________
>>Linux-ha-japan mailing list
>>Linux-ha-japan@lists.sourceforge.jp
>>http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>>
>
>_______________________________________________
>Linux-ha-japan mailing list
>Linux-ha-japan@lists.sourceforge.jp
>http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>
>
>

_______________________________________________
Linux-ha-japan mailing list
Linux-ha-japan@lists.sourceforge.jp
http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan

Re: DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦ [ In reply to ]

mayama0130 at gmail

Feb 17, 2015, 1:44 AM

Post #7 of 7 (5016 views)

Permalink

å±±å†…æ§˜

ã‚ãã‚„ã¾ã§ã™ã€‚

ã”å›žç”ã‚ã‚ŠãŒã¨ã†ã”ã„ã¾ã™ã€‚

> corosync+PMã®æŽ¨å¥¨å€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã®1000msã ã£ãŸã¨è¨˜æ†¶ã—ã¦ã„ã¾ã™ã€‚
> å¤‰æ›´ã™ã‚‹å ´åˆã®å€¤ã¯ã€æ§‹æˆï¼ˆè² è·çŠ¶æ³ãªã©ï¼‰ã«ã‚ˆã‚Šã¾ã™ã®ã§ã€å€ç¨‹åº¦ã‹ã‚‰ã‚ã’ã¦ã¿ã¦ã¯å¦‚ä½•ã§ã—ã‚‡ã†ã‹ï¼Ÿ
>
> tokenã‚’ä¼¸ã°ã™å ´åˆã®ãƒ‡ãƒ¡ãƒªãƒƒãƒˆã¯ã€æ•…éšœæ¤œçŸ¥ï¼ˆç›¸æ‰‹ã®ãƒ€ã‚¦ãƒ³ï¼‰æ™‚é–“ãŒé•·ããªã‚‹ã“ã¨ã«ãªã‚Šã¾ã™ã€‚
ä¸Šè¨˜ã‚’tokenå€¤ã®å¤‰æ›´ã‚‚æ¤œè¨Žã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚

2015-02-17 14:35 GMT+09:00 <renayama19661014@ybb.ne.jp>:

>
> ã‚ãã‚„ã¾ã•ã‚“
>
> ã“ã‚“ã«ã¡ã¯ã€å±±å†…ã§ã™ã€‚
>
> >tokenå€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆå€¤ï¼ˆ1000msï¼‰ã®ã¾ã¾ã§ã—ãŸã€‚
> >å¤‰æ›´ã™ã‚‹å ´åˆã§ã™ãŒã€æŽ¨å¥¨å€¤ã¯ã‚ã‚‹ã®ã§ã—ã‚‡ã†ã‹ï¼Ÿ
> >ã‚‚ã—ãã¯ç®—å‡ºæ–¹æ³•ç‰ã‚ã‚Œã°ã€ã”æ•™æŽˆé ‚ã‘ã‚‹ã¨åŠ©ã‹ã‚Šã¾ã™ã€‚
>
> corosync+PMã®æŽ¨å¥¨å€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã®1000msã ã£ãŸã¨è¨˜æ†¶ã—ã¦ã„ã¾ã™ã€‚
> å¤‰æ›´ã™ã‚‹å ´åˆã®å€¤ã¯ã€æ§‹æˆï¼ˆè² è·çŠ¶æ³ãªã©ï¼‰ã«ã‚ˆã‚Šã¾ã™ã®ã§ã€å€ç¨‹åº¦ã‹ã‚‰ã‚ã’ã¦ã¿ã¦ã¯å¦‚ä½•ã§ã—ã‚‡ã†ã‹ï¼Ÿ
>
> tokenã‚’ä¼¸ã°ã™å ´åˆã®ãƒ‡ãƒ¡ãƒªãƒƒãƒˆã¯ã€æ•…éšœæ¤œçŸ¥ï¼ˆç›¸æ‰‹ã®ãƒ€ã‚¦ãƒ³ï¼‰æ™‚é–“ãŒé•·ããªã‚‹ã“ã¨ã«ãªã‚Šã¾ã™ã€‚
>
> ä»¥ä¸Šã€å®œã—ããŠé¡˜ã„ã„ãŸã—ã¾ã™ã€‚
>
>
>
> ----- Original Message -----
> >From: ã‚ãã‚„ã¾ã˜ã‚ã† <mayama0130@gmail.com>
> >To: linux-ha-japan@lists.sourceforge.jp
> >Date: 2015/2/17, Tue 14:08
> >Subject: Re: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
> >
> >
> >å±±å†…æ§˜
> >
> >ã‚ãã‚„ã¾ã§ã™ã€‚
> >
> >ã”è¿”ä¿¡ã‚ã‚ŠãŒã¨ã†ã”ã–ã„ã¾ã™ã€‚
> >
> >> ã¡ãªã¿ã«ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã®è² è·ãŒé«˜ã„å¯èƒ½æ€§ã‚‚ã‚ã‚Šã¾ã™ã®ã§ã€
> >> sarã‚„vmstatã§cpuã®çŠ¶æ…‹ã‚‚è¦‹ã¦ã¿ãŸæ–¹ãŒã‚ˆã„ã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
> >> ï¼ƒã‚ã‚‹æ™‚ã«ã€ä½•ã‹ã®cronã‚¸ãƒ§ãƒ–ãªã©ã§è² è·ãŒã‚ãŒã£ãŸã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
> >å½“è©²æ™‚é–“ä»–ã®ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®å‡¦ç†ãŒä»–ã®æ™‚é–“å¸¯ã«æ¯”ã¹ã¦è² è·ãŒé«˜ã‹ã£ãŸã®ã§ã€
> >ãã®å¯èƒ½æ€§ã‚‚å«ã‚ã€çŠ¶æ…‹ã®ç¢ºèªã‚’ã—ã¦ã¿ãŸã„ã¨æ€ã„ã¾ã™ã€‚
> >
> >> corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ
> >tokenå€¤ã¯ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆå€¤ï¼ˆ1000msï¼‰ã®ã¾ã¾ã§ã—ãŸã€‚
> >å¤‰æ›´ã™ã‚‹å ´åˆã§ã™ãŒã€æŽ¨å¥¨å€¤ã¯ã‚ã‚‹ã®ã§ã—ã‚‡ã†ã‹ï¼Ÿ
> >ã‚‚ã—ãã¯ç®—å‡ºæ–¹æ³•ç‰ã‚ã‚Œã°ã€ã”æ•™æŽˆé ‚ã‘ã‚‹ã¨åŠ©ã‹ã‚Šã¾ã™ã€‚
> >
> >
> >
> >2015-02-17 12:58 GMT+09:00 <renayama19661014@ybb.ne.jp>:
> >
> >ã‚ãã‚„ã¾ã•ã‚“
> >>
> >>å±±å†…ã§ã™ã€‚
> >>
> >>ã¡ãªã¿ã«ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã®è² è·ãŒé«˜ã„å¯èƒ½æ€§ã‚‚ã‚ã‚Šã¾ã™ã®ã§ã€
> >>sarã‚„vmstatã§cpuã®çŠ¶æ…‹ã‚‚è¦‹ã¦ã¿ãŸæ–¹ãŒã‚ˆã„ã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
> >>
> >>ï¼ƒã‚ã‚‹æ™‚ã«ã€ä½•ã‹ã®cronã‚¸ãƒ§ãƒ–ãªã©ã§è² è·ãŒã‚ãŒã£ãŸã‹ã‚‚çŸ¥ã‚Œã¾ã›ã‚“ã€‚
> >>
> >>ä»¥ä¸Šã§ã™ã€‚
> >>
> >>
> >>
> >>----- Original Message -----
> >>> From: "renayama19661014@ybb.ne.jp" <renayama19661014@ybb.ne.jp>
> >>> To: "linux-ha-japan@lists.sourceforge.jp" <
> linux-ha-japan@lists.sourceforge.jp>
> >>> Cc:
> >>> Date: 2015/2/17, Tue 12:31
> >>> Subject: Re: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
> >>>
> >>> ã‚ãã‚„ã¾ã•ã‚“
> >>>
> >>> ã“ã‚“ã«ã¡ã¯ã€å±±å†…ã§ã™ã€‚
> >>>
> >>> corosyncã®ãƒ—ãƒ©ã‚¤ãƒžãƒªãƒã‚°ã‚’è¦‹ã‚‹é™ã‚Šã€ã‚»ã‚«ãƒ³ãƒ€ãƒªã¨ã®corosyncå±¤ã®é€šä¿¡ãŒåˆ‡ã‚Œã¦ã€ä¸€æ—¦ã€å†æ§‹æˆãŒã‹ã‹ã£ãŸã‚ˆã†ã§ã™ã€‚
> >>>
> >>> corosync.confã®tokenãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‚’ä¼¸ã°ã—ã¦ã¿ã‚‹ã¨ã©ã†ãªã‚Šã¾ã™ã§ã—ã‚‡ã†ã‹ï¼Ÿ
> >>>
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [MAIN ] main.c:1341
> Corosync main
> >>> process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms).
> Consider
> >>> token timeout increase.
> >>>
> >>>
> >>> ä¸Šè¨˜ã®ãƒã‚°ãŒå‡ºã¦ã„ã¾ã™ã®ã§ã€æœ¬æ¥ã®corosyncã®ã‚¹ã‚±ã‚¸ãƒ¥ãƒ¼ãƒªãƒ³ã‚°ãŒï¼’ç§’å¼·ã€é…ã‚Œã¦ã„ã‚‹çŠ¶æ…‹ãŒé€šçŸ¥ã•ã‚Œã¦ã¾ã™ã€‚
> >>>
> >>> ä»¥ä¸Šã€å®œã—ããŠé¡˜ã„ã„ãŸã—ã¾ã™ã€‚
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: ã‚ãã‚„ã¾ã˜ã‚ã† <mayama0130@gmail.com>
> >>>> To: linux-ha-japan@lists.sourceforge.jp
> >>>> Date: 2015/2/17, Tue 11:38
> >>>> Subject: [Linux-ha-jp] DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ã®ç™ºç”ŸåŽŸå› ã«é–¢ã—ã¦
> >>>>
> >>>>
> >>>> ãŠä¸–è©±ã«ãªã‚Šã¾ã™ã€‚ã‚ãã‚„ã¾ã¨ç”³ã—ã¾ã™ã€‚
> >>>>
> >>>> ä¸‹è¨˜ã®ç’°å¢ƒã§é‹ç”¨ã‚’ã—ã¦ã„ã¾ã™ã€‚
> >>>>
> >>>> ç’°å¢ƒï¼š
> >>>> Red Hat Enterprise Linux Server 6.1 (for x86)
> >>>> kernel-2.6.32-431.23.3.el6.i686
> >>>> corosync-1.4.1-17.el6_5.1.i686
> >>>> pacemaker-1.1.10-14.el6_5.3.i686
> >>>> drbd-8.3.16-1.el6.i386
> >>>>
> >>>> è³ªå•å†…å®¹ï¼š
> >>>> é‹ç”¨ä¸ã«çªç„¶DRBDã‚¹ãƒ—ãƒªãƒƒãƒˆãƒ–ãƒ¬ã‚¤ãƒ³ãŒç™ºç”Ÿã—ã€æ‰‹å‹•ã§å¾©æ—§ã•ã›ã¾ã—ãŸãŒã€
> >>>> ç™ºç”ŸåŽŸå› ãŒä¸æ˜Žã§å›°ã£ã¦ã„ã¾ã™ã€‚
> >>>> ç™ºç”ŸåŽŸå› ã¨å›žé¿ç–ãŒåˆ†ã‹ã‚Œã°ã€ã”æ•™æŽˆé¡˜ãˆã¾ã™ã§ã—ã‚‡ã†ã‹ã€‚
> >>>>
> >>>> ç¾åœ¨ã¾ã§ã«åˆ¤æ˜Žã—ã¦ã„ã‚‹äº‹ï¼š
> >>>> ãƒ»DRBDãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã®pingå¿œç”ã¯å•é¡Œãªã—
> >>>> ãƒ»messagesãƒã‚°ã«ãƒªãƒ³ã‚¯ãƒ€ã‚¦ãƒ³ã®å‡ºåŠ›ãªã—
> >>>>
> >>>> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®messagesãƒã‚°ã€‘
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned
> -110
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary ->
> Unknown )
> >>> conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting
> header on
> >>> sock: r=-110
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
> >>> 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
> >>> Unconnected )
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
> >>> WFConnection )
> >>>> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
> >>> from=::ffff:192.171.1.102
> >>>> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root:
> cmd='echo
> >>> $(date +%Y/%m/%d\ %H:%M:%S)'
> >>>> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
> >>> duration=0(sec)
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful:
> Agreed
> >>> network protocol version 97
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
> >>> WFReportParams )
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread
> (from
> >>> drbd0_receiver [3117])
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
> >>> <not-used>
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut
> down by
> >>> peer.
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
> >>> NetworkFailure )
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
> >>> Unconnected )
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
> >>> WFConnection )
> >>>>
> >>>> ã€ãƒ—ãƒ©ã‚¤ãƒžãƒªå´ã®corosyncãƒã‚°ã€‘
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [MAIN ] main.c:1341
> Corosync main
> >>> process was not scheduled for 3144.5435 ms (threshold is 800.0000 ms).
> Consider
> >>> token timeout increase.
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Transitional membership event on ring 16: memb=1,
> new=0,
> >>> lost=1
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:773 info:
> >>> pcmk_peer_update: memb: server1 1694608064
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:779 info:
> >>> pcmk_peer_update: lost: server2 1711385280
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1,
> lost=0
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:791 info:
> >>> pcmk_peer_update: NEW: server2 1711385280
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server1 1694608064
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server2 1711385280
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [TOTEM ] totemsrp.c:1892 A
> >>> processor joined or left the membership and a new membership was
> formed.
> >>>> Feb 16 18:00:27 server1 corosync[1756]: [CPG ] cpg.c:783 chosen
> >>> downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
> >>>> Feb 16 18:00:28 server1 corosync[1756]: [MAIN ] main.c:1341
> Corosync main
> >>> process was not scheduled for 826.4143 ms (threshold is 800.0000 ms).
> Consider
> >>> token timeout increase.
> >>>> Feb 16 18:00:28 server1 corosync[1756]: [MAIN ] main.c:278
> Completed
> >>> service synchronization, ready to provide service.
> >>>> Feb 16 18:00:28 server1 crmd[1766]: warning: crmd_ha_msg_filter:
> Another DC
> >>> detected: server2 (op=noop)
> >>>> Feb 16 18:00:28 server1 crmd[1766]: notice: do_state_transition:
> State
> >>> transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> >>> origin=crmd_ha_msg_filter ]
> >>>> Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> >>> requesting full refresh in R/W mode
> >>>> Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> >>> requesting full refresh in R/W mode
> >>>> Feb 16 18:00:30 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> >>> requesting full refresh in R/W mode
> >>>> Feb 16 18:00:30 server1 corosync[1756]: [MAIN ] main.c:1341
> Corosync main
> >>> process was not scheduled for 2248.0801 ms (threshold is 800.0000 ms).
> Consider
> >>> token timeout increase.
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Transitional membership event on ring 24: memb=1,
> new=0,
> >>> lost=1
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:773 info:
> >>> pcmk_peer_update: memb: server1 1694608064
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:779 info:
> >>> pcmk_peer_update: lost: server2 1711385280
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1,
> lost=0
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:791 info:
> >>> pcmk_peer_update: NEW: server2 1711385280
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server1 1694608064
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server2 1711385280
> >>>> Feb 16 18:00:31 server1 corosync[1756]: [TOTEM ] totemsrp.c:1892 A
> >>> processor joined or left the membership and a new membership was
> formed.
> >>>> Feb 16 18:00:32 server1 corosync[1756]: [CPG ] cpg.c:783 chosen
> >>> downlist: sender r(0) ip(192.170.1.101) ; members(old:2 left:1)
> >>>> Feb 16 18:00:32 server1 corosync[1756]: [MAIN ] main.c:278
> Completed
> >>> service synchronization, ready to provide service.
> >>>> Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> >>> requesting full refresh in R/W mode
> >>>> Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> >>> requesting full refresh in R/W mode
> >>>> Feb 16 18:00:32 server1 cib[1761]: warning: cib_server_process_diff:
> Not
> >>> requesting full refresh in R/W mode
> >>>> Feb 16 18:02:28 server1 crmd[1766]: error: crm_timer_popped:
> Election
> >>> Timeout (I_ELECTION_DC) just popped in state S_ELECTION! (120000ms)
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: do_state_transition:
> State
> >>> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> >>> cause=C_TIMER_POPPED origin=crm_timer_popped ]
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: do_state_transition:
> State
> >>> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> >>> cause=C_FSA_INTERNAL origin=do_election_check ]
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: do_dc_join_finalize:
> join-4:
> >>> Syncing the CIB from server2 to the rest of the cluster
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: do_dc_join_finalize:
> Requested
> >>> version <generation_tuple epoch="7" num_updates="87"
> >>> admin_epoch="0" validate-with="pacemaker-1.2"
> >>> cib-last-written="Wed Jan 21 00:17:56 2015"
> >>> update-origin="server1" update-client="crmd"
> >>> crm_feature_set="3.0.7" have-quorum="1"
> >>> dc-uuid="server2"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: Diff: ---
> 0.7.69
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: Diff: +++
> 0.7.87
> >>> 1afd9af32d3a574358543e1d4751aaf1
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> <node_state
> >>> crm-debug-origin="do_state_transition" join="member"
> >>> id="server1"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> >>> <lrm_rsc_op operation_key="res_drbd_r0_start_0"
> >>> operation="start"
> >>> transition-key="46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> >>> transition-magic="0:0;46:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> >>> call-id="44" last-run="1421767134"
> >>> last-rc-change="1421767134" exec-time="430"
> >>> id="res_drbd_r0_last_0"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> >>> <lrm_rsc_op id="res_drbd_r0_monitor_30000"
> >>> operation_key="res_drbd_r0_monitor_30000"
> >>> operation="monitor" crm-debug-origin="do_update_resource"
> >>> crm_feature_set="3.0.7"
> >>> transition-key="47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> >>> transition-magic="0:0;47:5:0:6fbace33-40d1-44cc-b09e-e66ef7ce03df"
> >>> call-id="50" rc-code="0" op-status="0"
> >>> interval="30000" last-rc-change="1421767135"
> >>> exec-time="60" queue-time="0"
> >>> op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: --
> >>> <nvpair value="1000"
> >>> id="status-server2-master-res_drbd_r0"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> <node_state
> >>> id="server1" uname="server1" in_ccm="true"
> >>> crmd="online" crm-debug-origin="post_cache_update"
> >>> join="down" expected="member"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> >>> <nvpair id="status-server2-master-res_drbd_r0"
> >>> name="master-res_drbd_r0" value="10000"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> >>> <lrm_rsc_op id="res_drbd_r0_last_0"
> >>> operation_key="res_drbd_r0_promote_0" operation="promote"
> >>> crm-debug-origin="do_update_resource"
> >>> crm_feature_set="3.0.7"
> >>> transition-key="14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> >>> transition-magic="0:0;14:1:0:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> >>> call-id="62" rc-code="0" op-status="0"
> >>> interval="0" last-run="1424077226"
> >>> last-rc-change="1424077226" exec-time="120"
> >>> queue-time="0"
> >>> op-digest="c0e018b73fdf522b6cdd355e125af15e"/>
> >>>> Feb 16 18:02:28 server1 cib[1761]: notice: cib:diff: ++
> >>> <lrm_rsc_op id="res_drbd_r0_monitor_10000"
> >>> operation_key="res_drbd_r0_monitor_10000"
> >>> operation="monitor" crm-debug-origin="do_update_resource"
> >>> crm_feature_set="3.0.7"
> >>> transition-key="15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> >>> transition-magic="0:8;15:1:8:2172d7d8-fe4a-427d-9311-27a13078a0fb"
> >>> call-id="68" rc-code="8" op-status="0"
> >>> interval="10000" last-rc-change="1424077226"
> >>> exec-time="53" queue-time="0"
> >>> op-digest="6f5adcd7f1211cdfc17850827b8582c5"/>
> >>>> Feb 16 18:02:28 server1 stonith-ng[1762]: warning: cib_process_diff:
> Diff
> >>> 0.7.69 -> 0.7.87 from local not applied to 0.7.69: Failed application
> of an
> >>> update diff
> >>>> Feb 16 18:02:28 server1 stonith-ng[1762]: notice:
> update_cib_cache_cb:
> >>> [cib_diff_notify] Patch aborted: Application of an update diff failed
> (-206)
> >>>> Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_local_callback:
> Sending
> >>> full refresh (origin=crmd)
> >>>> Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (10000)
> >>>> Feb 16 18:02:28 server1 attrd[1764]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: probe_complete (true)
> >>>> Feb 16 18:02:28 server1 pengine[1765]: notice: unpack_config: On
> loss of
> >>> CCM Quorum: Ignore
> >>>> Feb 16 18:02:28 server1 pengine[1765]: notice: LogActions: Demote
> >>> res_drbd_r0:1#011(Master -> Slave server2)
> >>>> Feb 16 18:02:28 server1 pengine[1765]: notice: process_pe_message:
> >>> Calculated Transition 2582: /var/lib/pacemaker/pengine/pe-input-24.bz2
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 8: cancel res_drbd_r0_cancel_10000 on server2
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 81: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 83: notify res_drbd_r0_pre_notify_demote_0 on server2
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=113, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: run_graph: Transition
> 2582
> >>> (Complete=4, Pending=0, Fired=0, Skipped=4, Incomplete=5,
> >>> Source=/var/lib/pacemaker/pengine/pe-input-24.bz2): Stopped
> >>>> Feb 16 18:02:28 server1 pengine[1765]: notice: unpack_config: On
> loss of
> >>> CCM Quorum: Ignore
> >>>> Feb 16 18:02:28 server1 pengine[1765]: notice: LogActions: Demote
> >>> res_drbd_r0:1#011(Master -> Slave server2)
> >>>> Feb 16 18:02:28 server1 pengine[1765]: notice: process_pe_message:
> >>> Calculated Transition 2583: /var/lib/pacemaker/pengine/pe-input-25.bz2
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 80: notify res_drbd_r0_pre_notify_demote_0 on server1 (local)
> >>>> Feb 16 18:02:28 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 82: notify res_drbd_r0_pre_notify_demote_0 on server2
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=116, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 39: demote res_drbd_r0_demote_0 on server2
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 81: notify res_drbd_r0_post_notify_demote_0 on server1 (local)
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 83: notify res_drbd_r0_post_notify_demote_0 on server2
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=119, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: run_graph: Transition
> 2583
> >>> (Complete=11, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> >>> Source=/var/lib/pacemaker/pengine/pe-input-25.bz2): Stopped
> >>>> Feb 16 18:02:29 server1 pengine[1765]: notice: unpack_config: On
> loss of
> >>> CCM Quorum: Ignore
> >>>> Feb 16 18:02:29 server1 pengine[1765]: notice: process_pe_message:
> >>> Calculated Transition 2584: /var/lib/pacemaker/pengine/pe-input-26.bz2
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: te_rsc_command:
> Initiating
> >>> action 40: monitor res_drbd_r0_monitor_30000 on server2
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: run_graph: Transition
> 2584
> >>> (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> >>> Source=/var/lib/pacemaker/pengine/pe-input-26.bz2): Complete
> >>>> Feb 16 18:02:29 server1 crmd[1766]: notice: do_state_transition:
> State
> >>> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> >>> cause=C_FSA_INTERNAL origin=notify_crmd ]
> >>>> Feb 16 18:17:29 server1 crmd[1766]: notice: do_state_transition:
> State
> >>> transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_TIMER_POPPED
> >>> origin=crm_timer_popped ]
> >>>> Feb 16 18:17:29 server1 pengine[1765]: notice: unpack_config: On
> loss of
> >>> CCM Quorum: Ignore
> >>>> Feb 16 18:17:29 server1 pengine[1765]: notice: process_pe_message:
> >>> Calculated Transition 2585: /var/lib/pacemaker/pengine/pe-input-27.bz2
> >>>> Feb 16 18:17:29 server1 crmd[1766]: notice: run_graph: Transition
> 2585
> >>> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> >>> Source=/var/lib/pacemaker/pengine/pe-input-27.bz2): Complete
> >>>> Feb 16 18:17:29 server1 crmd[1766]: notice: do_state_transition:
> State
> >>> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> >>> cause=C_FSA_INTERNAL origin=notify_crmd ]
> >>>>
> >>>> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®messagesãƒã‚°ã€‘
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: sock_recvmsg returned
> -110
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: peer( Secondary ->
> Unknown )
> >>> conn( Connected -> BrokenPipe ) pdsk( UpToDate -> DUnknown )
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: short read expecting
> header on
> >>> sock: r=-110
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: new current UUID
> >>> 626E92ABB7AC7487:1CC1ED4094EA9EB7:9A0088783FE79A75:99FF88783FE79A75
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: asender terminated
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: Terminating drbd0_asender
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: Connection closed
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( BrokenPipe ->
> >>> Unconnected )
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver terminated
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: Restarting drbd0_receiver
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: receiver (re)started
> >>>> Feb 16 18:00:05 server1 kernel: block drbd0: conn( Unconnected ->
> >>> WFConnection )
> >>>> Feb 16 18:00:06 server1 xinetd[1940]: START: shell pid=1056
> >>> from=::ffff:192.171.1.102
> >>>> Feb 16 18:00:06 server1 rshd[1057]: root@mdn-cluster2 as root:
> cmd='echo
> >>> $(date +%Y/%m/%d\ %H:%M:%S)'
> >>>> Feb 16 18:00:06 server1 xinetd[1940]: EXIT: shell status=0 pid=1056
> >>> duration=0(sec)
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: Handshake successful:
> Agreed
> >>> network protocol version 97
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFConnection ->
> >>> WFReportParams )
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: Starting asender thread
> (from
> >>> drbd0_receiver [3117])
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: data-integrity-alg:
> >>> <not-used>
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: meta connection shut
> down by
> >>> peer.
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: conn( WFReportParams ->
> >>> NetworkFailure )
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: asender terminated
> >>>> Feb 16 18:00:28 server1 kernel: block drbd0: Terminating drbd0_asender
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: Connection closed
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( NetworkFailure ->
> >>> Unconnected )
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver terminated
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: Restarting drbd0_receiver
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: receiver (re)started
> >>>> Feb 16 18:00:30 server1 kernel: block drbd0: conn( Unconnected ->
> >>> WFConnection )
> >>>>
> >>>> ã€ã‚»ã‚«ãƒ³ãƒ€ãƒªå´ã®corosyncãƒã‚°ã€‘
> >>>> Feb 16 18:00:23 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (1000)
> >>>> Feb 16 18:00:23 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> >>> update 45: master-res_drbd_r0=1000
> >>>> Feb 16 18:00:24 server2 corosync[1549]: [TOTEM ] totemsrp.c:1595 A
> >>> processor failed, forming new configuration.
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Transitional membership event on ring 12: memb=1,
> new=0,
> >>> lost=1
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> >>> pcmk_peer_update: memb: server2 1711385280
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:779 info:
> >>> pcmk_peer_update: lost: server1 1694608064
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Stable membership event on ring 12: memb=1, new=0,
> lost=0
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server2 1711385280
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:731 info:
> >>> ais_mark_unseen_peer_dead: Node server1 was not seen in the previous
> transition
> >>>> Feb 16 18:00:25 server2 cib[1554]: notice: plugin_handle_membership:
> >>> Membership 12: quorum lost
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> >>> update_member: Node 1694608064/server1 is now: lost
> >>>> Feb 16 18:00:25 server2 crmd[1559]: notice:
> plugin_handle_membership:
> >>> Membership 12: quorum lost
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> >>> send_member_notification: Sending membership update 12 to 2 children
> >>>> Feb 16 18:00:25 server2 cib[1554]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> >>> member)
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> >>> processor joined or left the membership and a new membership was
> formed.
> >>>> Feb 16 18:00:25 server2 crmd[1559]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> >>> member)
> >>>> Feb 16 18:00:25 server2 crmd[1559]: warning: reap_dead_nodes: Our DC
> node
> >>> (server1) left the cluster
> >>>> Feb 16 18:00:25 server2 crmd[1559]: notice: do_state_transition:
> State
> >>> transition S_NOT_DC -> S_ELECTION [ input=I_ELECTION
> cause=C_FSA_INTERNAL
> >>> origin=reap_dead_nodes ]
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> >>> downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
> >>>> Feb 16 18:00:25 server2 corosync[1549]: [MAIN ] main.c:278
> Completed
> >>> service synchronization, ready to provide service.
> >>>> Feb 16 18:00:25 server2 crmd[1559]: notice: do_state_transition:
> State
> >>> transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC
> >>> cause=C_FSA_INTERNAL origin=do_election_check ]
> >>>> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_local_callback:
> Sending
> >>> full refresh (origin=crmd)
> >>>> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (1000)
> >>>> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> >>> update 47: master-res_drbd_r0=1000
> >>>> Feb 16 18:00:25 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: probe_complete (true)
> >>>> Feb 16 18:00:26 server2 pengine[1558]: notice: unpack_config: On
> loss of
> >>> CCM Quorum: Ignore
> >>>> Feb 16 18:00:26 server2 pengine[1558]: notice: LogActions: Promote
> >>> res_drbd_r0:0#011(Slave -> Master server2)
> >>>> Feb 16 18:00:26 server2 pengine[1558]: notice: process_pe_message:
> >>> Calculated Transition 0: /var/lib/pacemaker/pengine/pe-input-0.bz2
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command:
> Initiating
> >>> action 1: cancel res_drbd_r0_cancel_30000 on server2 (local)
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command:
> Initiating
> >>> action 56: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=56, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: run_graph: Transition 0
> >>> (Complete=3, Pending=0, Fired=0, Skipped=5, Incomplete=4,
> >>> Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Stopped
> >>>> Feb 16 18:00:26 server2 pengine[1558]: notice: unpack_config: On
> loss of
> >>> CCM Quorum: Ignore
> >>>> Feb 16 18:00:26 server2 pengine[1558]: notice: LogActions: Promote
> >>> res_drbd_r0:0#011(Slave -> Master server2)
> >>>> Feb 16 18:00:26 server2 pengine[1558]: notice: process_pe_message:
> >>> Calculated Transition 1: /var/lib/pacemaker/pengine/pe-input-1.bz2
> >>>> Feb 16 18:00:26 server2 crmd[1559]: warning: destroy_action:
> Cancelling
> >>> timer for action 1 (src=45)
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command:
> Initiating
> >>> action 55: notify res_drbd_r0_pre_notify_promote_0 on server2 (local)
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Transitional membership event on ring 16: memb=1,
> new=0,
> >>> lost=0
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> >>> pcmk_peer_update: memb: server2 1711385280
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Stable membership event on ring 16: memb=2, new=1,
> lost=0
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> >>> update_member: Node 1694608064/server1 is now: member
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:791 info:
> >>> pcmk_peer_update: NEW: server1 1694608064
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server1 1694608064
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server2 1711385280
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> >>> send_member_notification: Sending membership update 16 to 2 children
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> >>> processor joined or left the membership and a new membership was
> formed.
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice:
> plugin_handle_membership:
> >>> Membership 16: quorum acquired
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now
> member (was
> >>> lost)
> >>>> Feb 16 18:00:26 server2 cib[1554]: notice: plugin_handle_membership:
> >>> Membership 16: quorum acquired
> >>>> Feb 16 18:00:26 server2 cib[1554]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now
> member (was
> >>> lost)
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=59, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command:
> Initiating
> >>> action 14: promote res_drbd_r0_promote_0 on server2 (local)
> >>>> Feb 16 18:00:26 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> >>> downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_promote_0 (call=62, rc=0, cib-update=49,
> confirmed=true)
> >>> ok
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command:
> Initiating
> >>> action 56: notify res_drbd_r0_post_notify_promote_0 on server2 (local)
> >>>> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (10000)
> >>>> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> >>> update 51: master-res_drbd_r0=10000
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=65, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: te_rsc_command:
> Initiating
> >>> action 15: monitor res_drbd_r0_monitor_10000 on server2 (local)
> >>>> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (10000)
> >>>> Feb 16 18:00:26 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> >>> update 53: master-res_drbd_r0=10000
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_monitor_10000 (call=68, rc=8, cib-update=50,
> >>> confirmed=false) master
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: process_lrm_event:
> >>> server2-res_drbd_r0_monitor_10000:68 [ \n ]
> >>>> Feb 16 18:00:26 server2 crmd[1559]: notice: run_graph: Transition 1
> >>> (Complete=11, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> >>> Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
> >>>> Feb 16 18:00:27 server2 corosync[1549]: [MAIN ] main.c:278
> Completed
> >>> service synchronization, ready to provide service.
> >>>> Feb 16 18:00:28 server2 corosync[1549]: [TOTEM ] totemsrp.c:1595 A
> >>> processor failed, forming new configuration.
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Transitional membership event on ring 20: memb=1,
> new=0,
> >>> lost=1
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> >>> pcmk_peer_update: memb: server2 1711385280
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:779 info:
> >>> pcmk_peer_update: lost: server1 1694608064
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Stable membership event on ring 20: memb=1, new=0,
> lost=0
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server2 1711385280
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:731 info:
> >>> ais_mark_unseen_peer_dead: Node server1 was not seen in the previous
> transition
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> >>> update_member: Node 1694608064/server1 is now: lost
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> >>> send_member_notification: Sending membership update 20 to 2 children
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> >>> processor joined or left the membership and a new membership was
> formed.
> >>>> Feb 16 18:00:29 server2 cib[1554]: notice: plugin_handle_membership:
> >>> Membership 20: quorum lost
> >>>> Feb 16 18:00:29 server2 crmd[1559]: notice:
> plugin_handle_membership:
> >>> Membership 20: quorum lost
> >>>> Feb 16 18:00:29 server2 cib[1554]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> >>> member)
> >>>> Feb 16 18:00:29 server2 crmd[1559]: warning: match_down_event: No
> match for
> >>> shutdown action on server1
> >>>> Feb 16 18:00:29 server2 crmd[1559]: notice: peer_update_callback:
> >>> Stonith/shutdown of server1 not matched
> >>>> Feb 16 18:00:29 server2 crmd[1559]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now lost
> (was
> >>> member)
> >>>> Feb 16 18:00:29 server2 crmd[1559]: warning: match_down_event: No
> match for
> >>> shutdown action on server1
> >>>> Feb 16 18:00:29 server2 crmd[1559]: notice: peer_update_callback:
> >>> Stonith/shutdown of server1 not matched
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> >>> downlist: sender r(0) ip(192.170.1.102) ; members(old:2 left:1)
> >>>> Feb 16 18:00:29 server2 corosync[1549]: [MAIN ] main.c:278
> Completed
> >>> service synchronization, ready to provide service.
> >>>> Feb 16 18:00:29 server2 crmd[1559]: notice: do_election_count_vote:
> >>> Election 4 (current: 2, owner: server1): Processed vote from server1
> (Peer is
> >>> not part of our cluster)
> >>>> Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_local_callback:
> Sending
> >>> full refresh (origin=crmd)
> >>>> Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (10000)
> >>>> Feb 16 18:00:29 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: probe_complete (true)
> >>>> Feb 16 18:00:29 server2 pengine[1558]: notice: unpack_config: On
> loss of
> >>> CCM Quorum: Ignore
> >>>> Feb 16 18:00:29 server2 pengine[1558]: notice: process_pe_message:
> >>> Calculated Transition 2: /var/lib/pacemaker/pengine/pe-input-2.bz2
> >>>> Feb 16 18:00:29 server2 crmd[1559]: notice: run_graph: Transition 2
> >>> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> >>> Source=/var/lib/pacemaker/pengine/pe-input-2.bz2): Complete
> >>>> Feb 16 18:00:29 server2 crmd[1559]: notice: do_state_transition:
> State
> >>> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> >>> cause=C_FSA_INTERNAL origin=notify_crmd ]
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Transitional membership event on ring 24: memb=1,
> new=0,
> >>> lost=0
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:773 info:
> >>> pcmk_peer_update: memb: server2 1711385280
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:760
> notice:
> >>> pcmk_peer_update: Stable membership event on ring 24: memb=2, new=1,
> lost=0
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] utils.c:358 info:
> >>> update_member: Node 1694608064/server1 is now: member
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:791 info:
> >>> pcmk_peer_update: NEW: server1 1694608064
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server1 1694608064
> >>>> Feb 16 18:00:30 server2 crmd[1559]: notice:
> plugin_handle_membership:
> >>> Membership 24: quorum acquired
> >>>> Feb 16 18:00:30 server2 cib[1554]: notice: plugin_handle_membership:
> >>> Membership 24: quorum acquired
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:808 info:
> >>> pcmk_peer_update: MEMB: server2 1711385280
> >>>> Feb 16 18:00:30 server2 cib[1554]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now
> member (was
> >>> lost)
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [pcmk ] plugin.c:1368 info:
> >>> send_member_notification: Sending membership update 24 to 2 children
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [TOTEM ] totemsrp.c:1892 A
> >>> processor joined or left the membership and a new membership was
> formed.
> >>>> Feb 16 18:00:30 server2 crmd[1559]: notice: crm_update_peer_state:
> >>> plugin_handle_membership: Node server1[1694608064] - state is now
> member (was
> >>> lost)
> >>>> Feb 16 18:00:30 server2 corosync[1549]: [CPG ] cpg.c:783 chosen
> >>> downlist: sender r(0) ip(192.170.1.102) ; members(old:1 left:0)
> >>>> Feb 16 18:00:31 server2 corosync[1549]: [MAIN ] main.c:278
> Completed
> >>> service synchronization, ready to provide service.
> >>>> Feb 16 18:00:31 server2 attrd[1557]: warning: attrd_cib_callback:
> Update 45
> >>> for master-res_drbd_r0=1000 failed: Application of an update diff
> failed
> >>>> Feb 16 18:00:56 server2 crmd[1559]: warning: cib_rsc_callback:
> Resource
> >>> update 49 failed: (rc=-62) Timer expired
> >>>> Feb 16 18:00:56 server2 crmd[1559]: warning: cib_rsc_callback:
> Resource
> >>> update 50 failed: (rc=-62) Timer expired
> >>>> Feb 16 18:02:26 server2 attrd[1557]: warning: attrd_cib_callback:
> Update 51
> >>> for master-res_drbd_r0=10000 failed: Timer expired
> >>>> Feb 16 18:02:27 server2 crmd[1559]: warning: crmd_ha_msg_filter:
> Another DC
> >>> detected: server1 (op=join_offer)
> >>>> Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition:
> State
> >>> transition S_IDLE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL
> >>> origin=crmd_ha_msg_filter ]
> >>>> Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition:
> State
> >>> transition S_ELECTION -> S_RELEASE_DC [ input=I_RELEASE_DC
> >>> cause=C_FSA_INTERNAL origin=do_election_count_vote ]
> >>>> Feb 16 18:02:27 server2 crmd[1559]: notice: do_state_transition:
> State
> >>> transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE
> >>> origin=do_cl_join_finalize_respond ]
> >>>> Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_local_callback:
> Sending
> >>> full refresh (origin=crmd)
> >>>> Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (10000)
> >>>> Feb 16 18:02:27 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: probe_complete (true)
> >>>> Feb 16 18:02:27 server2 crmd[1559]: notice: process_lrm_event:
> >>> server2-res_drbd_r0_monitor_10000:68 [ \n ]
> >>>> Feb 16 18:02:27 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=74, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=77, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_demote_0 (call=80, rc=0, cib-update=74,
> confirmed=true) ok
> >>>> Feb 16 18:02:28 server2 attrd[1557]: notice: attrd_trigger_update:
> Sending
> >>> flush op to all hosts for: master-res_drbd_r0 (1000)
> >>>> Feb 16 18:02:28 server2 attrd[1557]: notice: attrd_perform_update:
> Sent
> >>> update 67: master-res_drbd_r0=1000
> >>>> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_notify_0 (call=83, rc=0, cib-update=0,
> confirmed=true) ok
> >>>> Feb 16 18:02:28 server2 crmd[1559]: notice: process_lrm_event: LRM
> >>> operation res_drbd_r0_monitor_30000 (call=86, rc=0, cib-update=75,
> >>> confirmed=false) ok
> >>>>
> >>>> ã€global_common.confã€‘
> >>>> global {
> >>>> usage-count no;
> >>>> }
> >>>>
> >>>> common {
> >>>> protocol C;
> >>>> handlers {
> >>>> }
> >>>> startup {
> >>>> }
> >>>> disk {
> >>>> no-disk-barrier;
> >>>> no-disk-flushes;
> >>>> }
> >>>> net {
> >>>> max-buffers 8000;
> >>>> max-epoch-size 8000;
> >>>> after-sb-0pri discard-younger-primary;
> >>>> after-sb-1pri discard-secondary;
> >>>> after-sb-2pri disconnect;
> >>>> }
> >>>> syncer {
> >>>> rate 30M;
> >>>> verify-alg sha1;
> >>>> }
> >>>> }
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Linux-ha-japan mailing list
> >>>> Linux-ha-japan@lists.sourceforge.jp
> >>>> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
> >>>>
> >>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> Linux-ha-japan mailing list
> >>> Linux-ha-japan@lists.sourceforge.jp
> >>> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
> >>>
> >>
> >>_______________________________________________
> >>Linux-ha-japan mailing list
> >>Linux-ha-japan@lists.sourceforge.jp
> >>http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
> >>
> >
> >_______________________________________________
> >Linux-ha-japan mailing list
> >Linux-ha-japan@lists.sourceforge.jp
> >http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
> >
> >
> >
>
> _______________________________________________
> Linux-ha-japan mailing list
> Linux-ha-japan@lists.sourceforge.jp
> http://lists.sourceforge.jp/mailman/listinfo/linux-ha-japan
>