Mailing List Archive

Antw: Re: Postgresql RA fails starting master node
>>> Takehiro Matsushima <takehiro.dreamizm@gmail.com> schrieb am 11.09.2014 um
01:15 in Nachricht
<CAAbhU_DtkdGfJ425J3wWrUU7eX_oXs--CMcia-uGVjCnvqHbPQ@mail.gmail.com>:
> Hi.
>
> Let me ask some things,
>
> Can pz01 run under pacemker, without pz02?
> Something differed from running only pz02?
> Could you find some hints in the logs of each postgresql(and pgstartup.log)?

Sometimes it's handy to write a script for ocf-tester (many params to pass) and run it in verbose and RA debugging mode (hoping the RA says something reasonable). But stop the RA in the cluster first. Try the script on both nodes looking for differences.

>
> Good-luck,
>
> Takehiro Matsushima
> _______________________________________________
> Linux-HA mailing list
> Linux-HA@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems



_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
Takehiro, Ulrich, thanks for your responses.
I have some interesting information.

If I only run pz02, then it keeps on starting as Postgres Slave, it is not
promoted to Master:

Online: [ pz02 ]
OFFLINE: [ pz01 ]

Full list of resources:

Master/Slave Set: msPostgresql [pgsql]
Slaves: [ pz02 ]
Stopped: [ pgsql:1 ]
Resource Group: master-group
vip-master (ocf::heartbeat:IPaddr2): Stopped
vip-rep (ocf::heartbeat:IPaddr2): Stopped

Node Attributes:
* Node pz02:
+ master-pgsql:0 : -INFINITY
+ pgsql-data-status : DISCONNECT
+ pgsql-status : HS:alone


Logs:

Sep 11 11:36:00 pz02 pengine: [42962]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Sep 11 11:36:00 pz02 pengine: [42962]: notice: LogActions: Start pgsql:0
(pz02)
Sep 11 11:36:00 pz02 crmd: [42963]: notice: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=handle_response ]
Sep 11 11:36:00 pz02 crmd: [42963]: info: do_te_invoke: Processing graph 0
(ref=pe_calc-dc-1410446160-7) derived from
/var/lib/pengine/pe-input-3875.bz2
Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating action
4: monitor pgsql:0_monitor_0 on pz02 (local)
Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:pgsql:0 probe[2] (pid 42970)
Sep 11 11:36:00 pz02 pengine: [42962]: notice: process_pe_message:
Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-3875.bz2
Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating action
5: monitor vip-master_monitor_0 on pz02 (local)
Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:vip-master probe[3] (pid
42972)
Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating action
6: monitor vip-rep_monitor_0 on pz02 (local)
Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:vip-rep probe[4] (pid 42973)
Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[3] on
vip-master for client 42963: pid 42972 exited with return code 7
Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[4] on vip-rep
for client 42963: pid 42973 exited with return code 7
Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
running
Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[2] on pgsql:0
for client 42963: pid 42970 exited with return code 7
Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
pgsql:0_monitor_0 (call=2, rc=7, cib-update=28, confirmed=true) not running
Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating action
3: probe_complete probe_complete on pz02 (local) - no waiting
Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating action
7: start pgsql:0_start_0 on pz02 (local)
Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
flush op to all hosts for: probe_complete (true)
Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
update 4: probe_complete=true
Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:pgsql:0 start[5] (pid 43082)
Sep 11 11:36:00 pz02 crm_attribute: [43131]: info: Invoked:
/usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-status -v STOP
Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
flush op to all hosts for: pgsql-status (STOP)
Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
update 7: pgsql-status=STOP
Sep 11 11:36:00 pz02 crmd: [42963]: info: abort_transition_graph:
te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
id=status-pz02-pgsql-status, name=pgsql-status, value=STOP, magic=NA,
cib=0.287.18) : Transient attribute: update
Sep 11 11:36:00 pz02 crm_attribute: [43132]: info: Invoked:
/usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-master-baseline -D
Sep 11 11:36:00 pz02 crm_attribute: [43136]: info: Invoked: crm_attribute
-N pz02 -n master-pgsql:0 -l reboot -v -INFINITY
Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
flush op to all hosts for: master-pgsql:0 (-INFINITY)
Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
update 10: master-pgsql:0=-INFINITY
Sep 11 11:36:00 pz02 crmd: [42963]: info: abort_transition_graph:
te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
id=status-pz02-master-pgsql.0, name=master-pgsql:0, value=-INFINITY,
magic=NA, cib=0.287.19) : Transient attribute: update
Sep 11 11:36:00 pz02 crm_attribute: [43146]: info: Invoked:
/usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-xlog-loc -D
Sep 11 11:36:00 pz02 lrmd: [42960]: info: RA output: (pgsql:0:start:stderr)
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?


Sep 11 11:36:02 pz02 crm_attribute: [43259]: info: Invoked:
/usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-status -v HS:alone
Sep 11 11:36:02 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
flush op to all hosts for: pgsql-status (HS:alone)
Sep 11 11:36:02 pz02 lrmd: [42960]: info: operation start[5] on pgsql:0 for
client 42963: pid 43082 exited with return code 0
Sep 11 11:36:02 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
update 12: pgsql-status=HS:alone
Sep 11 11:36:02 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
pgsql:0_start_0 (call=5, rc=0, cib-update=29, confirmed=true) ok
Sep 11 11:36:02 pz02 crmd: [42963]: info: abort_transition_graph:
te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
id=status-pz02-pgsql-status, name=pgsql-status, value=HS:alone, magic=NA,
cib=0.287.20) : Transient attribute: update
Sep 11 11:36:02 pz02 crmd: [42963]: info: te_rsc_command: Initiating action
46: notify pgsql:0_post_notify_start_0 on pz02 (local)
Sep 11 11:36:02 pz02 lrmd: [42960]: info: rsc:pgsql:0 notify[6] (pid 43260)
Sep 11 11:36:02 pz02 lrmd: [42960]: info: operation notify[6] on pgsql:0
for client 42963: pid 43260 exited with return code 0
Sep 11 11:36:02 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
pgsql:0_notify_0 (call=6, rc=0, cib-update=0, confirmed=true) ok
Sep 11 11:36:02 pz02 crmd: [42963]: notice: run_graph: ==== Transition 0
(Complete=13, Pending=0, Fired=0, Skipped=1, Incomplete=0,
Source=/var/lib/pengine/pe-input-3875.bz2): Stopped
But it is correctly DC: [42963]: info: update_dc: Set DC to pz02 (3.0.6)





*I’ve found that using "rep_mode=sync” with my actual configuration then I
get “unknown error” when pgsl try to start Postgres:*



Sep 11 11:50:12 pz01 pengine: [54416]: notice: LogActions: Start pgsql:0
(pz01)
Sep 11 11:50:12 pz01 crmd: [54417]: notice: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=handle_response ]
Sep 11 11:50:12 pz01 crmd: [54417]: info: do_te_invoke: Processing graph 0
(ref=pe_calc-dc-1410450612-7) derived from
/var/lib/pengine/pe-input-1198.bz2
Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating action
4: monitor pgsql:0_monitor_0 on pz01 (local)
Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 probe[2] (pid 54424)
Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating action
5: monitor vip-master_monitor_0 on pz01 (local)
Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:vip-master probe[3] (pid
54425)
Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating action
6: monitor vip-rep_monitor_0 on pz01 (local)
Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:vip-rep probe[4] (pid 54427)
Sep 11 11:50:12 pz01 pengine: [54416]: notice: process_pe_message:
Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-1198.bz2
Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[3] on
vip-master for client 54417: pid 54425 exited with return code 7
Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
running
Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[4] on vip-rep
for client 54417: pid 54427 exited with return code 7
Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[2] on pgsql:0
for client 54417: pid 54424 exited with return code 7
Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
pgsql:0_monitor_0 (call=2, rc=7, cib-update=28, confirmed=true) not running
Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating action
3: probe_complete probe_complete on pz01 (local) - no waiting
Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating action
7: start pgsql:0_start_0 on pz01 (local)
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
flush op to all hosts for: probe_complete (true)
Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 start[5] (pid 54537)
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
update 4: probe_complete=true
Sep 11 11:50:12 pz01 crm_attribute: [54586]: info: Invoked:
/usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-status -v STOP
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
flush op to all hosts for: pgsql-status (STOP)
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
update 7: pgsql-status=STOP
Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
id=status-pz01-pgsql-status, name=pgsql-status, value=STOP, magic=NA,
cib=0.330.18) : Transient attribute: update
Sep 11 11:50:12 pz01 crm_attribute: [54587]: info: Invoked:
/usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-master-baseline -D
Sep 11 11:50:12 pz01 crm_attribute: [54591]: info: Invoked: crm_attribute
-N pz01 -n master-pgsql:0 -l reboot -v -INFINITY
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
flush op to all hosts for: master-pgsql:0 (-INFINITY)
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
update 10: master-pgsql:0=-INFINITY
Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
id=status-pz01-master-pgsql.0, name=master-pgsql:0, value=-INFINITY,
magic=NA, cib=0.330.19) : Transient attribute: update
Sep 11 11:50:12 pz01 crm_attribute: [54601]: info: Invoked:
/usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-xlog-loc -D
*Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation start[5] on pgsql:0
for client 54417: pid 54537 exited with return code 1*
*Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
pgsql:0_start_0 (call=5, rc=1, cib-update=29, confirmed=true) unknown error*
Sep 11 11:50:12 pz01 crmd: [54417]: WARN: status_from_rc: Action 7
(pgsql:0_start_0) on pz01 failed (target: 0 vs. rc: 1): Error
Sep 11 11:50:12 pz01 crmd: [54417]: WARN: update_failcount: Updating
failcount for pgsql:0 on pz01 after failed start: rc=1 (update=INFINITY,
time=1410450612)
Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
match_graph_event:277 - Triggered transition abort (complete=0,
tag=lrm_rsc_op, id=pgsql:0_last_failure_0,
magic=0:1;7:0:0:af61405d-b520-4e01-8f85-0443bd508a04, cib=0.330.20) : Event
failed
Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating action
46: notify pgsql:0_post_notify_start_0 on pz01 (local)
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
flush op to all hosts for: fail-count-pgsql:0 (INFINITY)
Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 notify[6] (pid 54616)
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
update 13: fail-count-pgsql:0=INFINITY
Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
id=status-pz01-fail-count-pgsql.0, name=fail-count-pgsql:0, value=INFINITY,
magic=NA, cib=0.330.21) : Transient attribute: update
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
flush op to all hosts for: last-failure-pgsql:0 (1410450612)
Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
update 16: last-failure-pgsql:0=1410450612
Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
id=status-pz01-last-failure-pgsql.0, name=last-failure-pgsql:0,
value=1410450612, magic=NA, cib=0.330.22) : Transient attribute: update
Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation notify[6] on pgsql:0
for client 54417: pid 54616 exited with return code 0
Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
pgsql:0_notify_0 (call=6, rc=0, cib-update=0, confirmed=true) ok
Sep 11 11:50:12 pz01 crmd: [54417]: notice: run_graph: ==== Transition 0
(Complete=13, Pending=0, Fired=0, Skipped=1, Incomplete=0,
Source=/var/lib/pengine/pe-input-1198.bz2): Stopped
Sep 11 11:50:12 pz01 crmd: [54417]: notice: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_FSA_INTERNAL origin=notify_crmd ]
Sep 11 11:50:12 pz01 pengine: [54416]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Sep 11 11:50:12 pz01 pengine: [54416]: WARN: unpack_rsc_op: Processing
failed op pgsql:0_last_failure_0 on pz01: unknown error (1)
Sep 11 11:50:12 pz01 pengine: [54416]: WARN: common_apply_stickiness:
Forcing msPostgresql away from pz01 after 1000000 failures (max=1)
Sep 11 11:50:12 pz01 pengine: [54416]: WARN: common_apply_stickiness:
Forcing msPostgresql away from pz01 after 1000000 failures (max=1)
Sep 11 11:50:12 pz01 pengine: [54416]: notice: LogActions: Stop pgsql:0
(pz01)




*But If I change to “rep_mode=async” then I get 'invalid parameter’ from *
*pgsql:0_monitor_0*

Sep 11 11:54:27 pz01 pengine: [54884]: notice: LogActions: Start pgsql:0
(pz01)
Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=handle_response ]
Sep 11 11:54:27 pz01 crmd: [54885]: info: do_te_invoke: Processing graph 0
(ref=pe_calc-dc-1410450867-7) derived from
/var/lib/pengine/pe-input-1202.bz2
Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating action
4: monitor pgsql:0_monitor_0 on pz01 (local)
Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:pgsql:0 probe[2] (pid 54892)
Sep 11 11:54:27 pz01 pengine: [54884]: notice: process_pe_message:
Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-1202.bz2
Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating action
5: monitor vip-master_monitor_0 on pz01 (local)
Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:vip-master probe[3] (pid
54894)
Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating action
6: monitor vip-rep_monitor_0 on pz01 (local)
Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:vip-rep probe[4] (pid 54895)
Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[3] on
vip-master for client 54885: pid 54894 exited with return code 7
Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM operation
vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
running
Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[4] on vip-rep
for client 54885: pid 54895 exited with return code 7
Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM operation
vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
*Sep 11 11:54:27 pz01 lrmd: [54882]: info: RA output:
(pgsql:0:probe:stderr) /usr/lib/ocf/resource.d//heartbeat/pgsql: 1646:
/usr/lib/ocf/resource.d//heartbeat/pgsql: Bad substitution*

*Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[2] on pgsql:0
for client 54885: pid 54892 exited with return code 2*
*Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM operation
pgsql:0_monitor_0 (call=2, rc=2, cib-update=28, confirmed=true) invalid
parameter*
*Sep 11 11:54:27 pz01 crmd: [54885]: WARN: status_from_rc: Action 4
(pgsql:0_monitor_0) on pz01 failed (target: 7 vs. rc: 2): Error*
Sep 11 11:54:27 pz01 crmd: [54885]: info: abort_transition_graph:
match_graph_event:277 - Triggered transition abort (complete=0,
tag=lrm_rsc_op, id=pgsql:0_last_failure_0,
magic=0:2;4:0:7:58ccffc4-db11-44a2-aa1a-bc79cd4924ae, cib=0.331.16) : Event
failed
Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating action
3: probe_complete probe_complete on pz01 (local) - no waiting
Sep 11 11:54:27 pz01 crmd: [54885]: notice: run_graph: ==== Transition 0
(Complete=7, Pending=0, Fired=0, Skipped=3, Incomplete=4,
Source=/var/lib/pengine/pe-input-1202.bz2): Stopped
Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_FSA_INTERNAL origin=notify_crmd ]
Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_trigger_update: Sending
flush op to all hosts for: probe_complete (true)
Sep 11 11:54:27 pz01 pengine: [54884]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Sep 11 11:54:27 pz01 pengine: [54884]: ERROR: unpack_rsc_op: Preventing
msPostgresql from re-starting on pz01: operation monitor failed 'invalid
parameter' (rc=2)
Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=handle_response ]
Sep 11 11:54:27 pz01 crmd: [54885]: info: do_te_invoke: Processing graph 1
(ref=pe_calc-dc-1410450867-12) derived from
/var/lib/pengine/pe-input-1203.bz2
Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating action
3: probe_complete probe_complete on pz01 (local) - no waiting
Sep 11 11:54:27 pz01 crmd: [54885]: notice: run_graph: ==== Transition 1
(Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pengine/pe-input-1203.bz2): Complete
Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
cause=C_FSA_INTERNAL origin=notify_crmd ]
Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_perform_update: Sent
update 4: probe_complete=true
Sep 11 11:54:27 pz01 pengine: [54884]: notice: process_pe_message:
Transition 1: PEngine Input stored in: /var/lib/pengine/pe-input-1203.bz2
Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_trigger_update: Sending
flush op to all hosts for: probe_complete (true)
Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_perform_update: Sent
update 6: probe_complete=true



*Some questions:*

1. Could my issue be related with some erroneous postgres configuration?
(I’ll check my postgres conf and I’ll post it)
2. How can I debug values from pgsl?
For example to check values in pgsl:L1646
sed -i "/${rep_mode_string//\//\\/}/d" $OCF_RESKEY_config

If I use: *ocf_log err "$OCF_RESKEY_config”* in pgsql
Where do I have to check this print? Because I’m not seeing it in
corosync.log.
Do I have to enable anything else?

Thanks for your help!!




2014-09-11 3:06 GMT-03:00 Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de>:

> >>> Takehiro Matsushima <takehiro.dreamizm@gmail.com> schrieb am
> 11.09.2014 um
> 01:15 in Nachricht
> <CAAbhU_DtkdGfJ425J3wWrUU7eX_oXs--CMcia-uGVjCnvqHbPQ@mail.gmail.com>:
> > Hi.
> >
> > Let me ask some things,
> >
> > Can pz01 run under pacemker, without pz02?
> > Something differed from running only pz02?
> > Could you find some hints in the logs of each postgresql(and
> pgstartup.log)?
>
> Sometimes it's handy to write a script for ocf-tester (many params to
> pass) and run it in verbose and RA debugging mode (hoping the RA says
> something reasonable). But stop the RA in the cluster first. Try the script
> on both nodes looking for differences.
>
> >
> > Good-luck,
> >
> > Takehiro Matsushima
> > _______________________________________________
> > Linux-HA mailing list
> > Linux-HA@lists.linux-ha.org
> > http://lists.linux-ha.org/mailman/listinfo/linux-ha
> > See also: http://linux-ha.org/ReportingProblems
>
>
>
> _______________________________________________
> Linux-HA mailing list
> Linux-HA@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
Using ocf-tester I get:

ocf-tester -n pgsql -o repuser="ha" -o pgdba="postgres" -o
restart_on_promote="true" -o pgctl="/usr/lib/postgresql/9.1/bin/pg_ctl" -o
psql="/usr/lib/postgresql/9.1/bin/psql" -o
pgdata="/var/lib/postgresql/9.1/main/" -o
config="/etc/postgresql/9.1/main/postgresql.conf" -o rep_mode="async" -o
node_list="pz01 pz02" -o restore_command="cp
/var/lib/postgresql/9.1/main/archive/%f %p" -o
primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5
keepalives_count=5" -o master_ip="10.10.10.80" -o stop_escalate="0"
/usr/lib/ocf/resource.d/heartbeat/pgsql
Beginning tests for /usr/lib/ocf/resource.d/heartbeat/pgsql...
/usr/sbin/ocf-tester: 268: export: /var/lib/postgresql/9.1/main/archive/%f:
bad variable name

Is this the reason why I get `invalid parameter` ? Do you know what is
wrong there?

And without restore_command:

ocf-tester -n msPgsql -o repuser="ha" -o pgdba="postgres" -o
restart_on_promote="true" -o pgctl="/usr/lib/postgresql/9.1/bin/pg_ctl" -o
psql="/usr/lib/postgresql/9.1/bin/psql" -o
pgdata="/var/lib/postgresql/9.1/main/" -o
config="/etc/postgresql/9.1/main/postgresql.conf" -o rep_mode="async" -o
node_list="pz01 pz02" -o primary_conninfo_opt="keepalives_idle=60
keepalives_interval=5 keepalives_count=5" -o master_ip="10.10.10.80" -o
stop_escalate="0" /usr/lib/ocf/resource.d/heartbeat/pgsql
Beginning tests for /usr/lib/ocf/resource.d/heartbeat/pgsql...
* rc=6: Validation failed. Did you supply enough options with -o ?
pgsql[57665]: ERROR: Replication(rep_mode=async or sync) requires
Master/Slave configuration.
Aborting tests

Thanks in advanced




2014-09-11 12:07 GMT-03:00 Federico Castro <fcastelco@gmail.com>:

> Takehiro, Ulrich, thanks for your responses.
> I have some interesting information.
>
> If I only run pz02, then it keeps on starting as Postgres Slave, it is not
> promoted to Master:
>
> Online: [ pz02 ]
> OFFLINE: [ pz01 ]
>
> Full list of resources:
>
> Master/Slave Set: msPostgresql [pgsql]
> Slaves: [ pz02 ]
> Stopped: [ pgsql:1 ]
> Resource Group: master-group
> vip-master (ocf::heartbeat:IPaddr2): Stopped
> vip-rep (ocf::heartbeat:IPaddr2): Stopped
>
> Node Attributes:
> * Node pz02:
> + master-pgsql:0 : -INFINITY
> + pgsql-data-status : DISCONNECT
> + pgsql-status : HS:alone
>
>
> Logs:
>
> Sep 11 11:36:00 pz02 pengine: [42962]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 11 11:36:00 pz02 pengine: [42962]: notice: LogActions: Start pgsql:0
> (pz02)
> Sep 11 11:36:00 pz02 crmd: [42963]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:36:00 pz02 crmd: [42963]: info: do_te_invoke: Processing graph 0
> (ref=pe_calc-dc-1410446160-7) derived from
> /var/lib/pengine/pe-input-3875.bz2
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 4: monitor pgsql:0_monitor_0 on pz02 (local)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:pgsql:0 probe[2] (pid 42970)
> Sep 11 11:36:00 pz02 pengine: [42962]: notice: process_pe_message:
> Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-3875.bz2
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 5: monitor vip-master_monitor_0 on pz02 (local)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:vip-master probe[3] (pid
> 42972)
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 6: monitor vip-rep_monitor_0 on pz02 (local)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:vip-rep probe[4] (pid 42973)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[3] on
> vip-master for client 42963: pid 42972 exited with return code 7
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[4] on vip-rep
> for client 42963: pid 42973 exited with return code 7
> Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
> running
> Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[2] on pgsql:0
> for client 42963: pid 42970 exited with return code 7
> Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> pgsql:0_monitor_0 (call=2, rc=7, cib-update=28, confirmed=true) not running
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz02 (local) - no waiting
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 7: start pgsql:0_start_0 on pz02 (local)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 4: probe_complete=true
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:pgsql:0 start[5] (pid 43082)
> Sep 11 11:36:00 pz02 crm_attribute: [43131]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-status -v STOP
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: pgsql-status (STOP)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 7: pgsql-status=STOP
> Sep 11 11:36:00 pz02 crmd: [42963]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz02-pgsql-status, name=pgsql-status, value=STOP, magic=NA,
> cib=0.287.18) : Transient attribute: update
> Sep 11 11:36:00 pz02 crm_attribute: [43132]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-master-baseline -D
> Sep 11 11:36:00 pz02 crm_attribute: [43136]: info: Invoked: crm_attribute
> -N pz02 -n master-pgsql:0 -l reboot -v -INFINITY
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-pgsql:0 (-INFINITY)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 10: master-pgsql:0=-INFINITY
> Sep 11 11:36:00 pz02 crmd: [42963]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz02-master-pgsql.0, name=master-pgsql:0, value=-INFINITY,
> magic=NA, cib=0.287.19) : Transient attribute: update
> Sep 11 11:36:00 pz02 crm_attribute: [43146]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-xlog-loc -D
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: RA output:
> (pgsql:0:start:stderr) psql: could not connect to server: No such file or
> directory
> Is the server running locally and accepting
> connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
>
>
> Sep 11 11:36:02 pz02 crm_attribute: [43259]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-status -v HS:alone
> Sep 11 11:36:02 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: pgsql-status (HS:alone)
> Sep 11 11:36:02 pz02 lrmd: [42960]: info: operation start[5] on pgsql:0
> for client 42963: pid 43082 exited with return code 0
> Sep 11 11:36:02 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 12: pgsql-status=HS:alone
> Sep 11 11:36:02 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> pgsql:0_start_0 (call=5, rc=0, cib-update=29, confirmed=true) ok
> Sep 11 11:36:02 pz02 crmd: [42963]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz02-pgsql-status, name=pgsql-status, value=HS:alone, magic=NA,
> cib=0.287.20) : Transient attribute: update
> Sep 11 11:36:02 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 46: notify pgsql:0_post_notify_start_0 on pz02 (local)
> Sep 11 11:36:02 pz02 lrmd: [42960]: info: rsc:pgsql:0 notify[6] (pid 43260)
> Sep 11 11:36:02 pz02 lrmd: [42960]: info: operation notify[6] on pgsql:0
> for client 42963: pid 43260 exited with return code 0
> Sep 11 11:36:02 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> pgsql:0_notify_0 (call=6, rc=0, cib-update=0, confirmed=true) ok
> Sep 11 11:36:02 pz02 crmd: [42963]: notice: run_graph: ==== Transition 0
> (Complete=13, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> Source=/var/lib/pengine/pe-input-3875.bz2): Stopped
> But it is correctly DC: [42963]: info: update_dc: Set DC to pz02 (3.0.6)
>
>
>
>
>
> *I’ve found that using "rep_mode=sync” with my actual configuration then I
> get “unknown error” when pgsl try to start Postgres:*
>
>
>
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: LogActions: Start pgsql:0
> (pz01)
> Sep 11 11:50:12 pz01 crmd: [54417]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:50:12 pz01 crmd: [54417]: info: do_te_invoke: Processing graph 0
> (ref=pe_calc-dc-1410450612-7) derived from
> /var/lib/pengine/pe-input-1198.bz2
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 4: monitor pgsql:0_monitor_0 on pz01 (local)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 probe[2] (pid 54424)
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 5: monitor vip-master_monitor_0 on pz01 (local)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:vip-master probe[3] (pid
> 54425)
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 6: monitor vip-rep_monitor_0 on pz01 (local)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:vip-rep probe[4] (pid 54427)
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: process_pe_message:
> Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-1198.bz2
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[3] on
> vip-master for client 54417: pid 54425 exited with return code 7
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
> running
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[4] on vip-rep
> for client 54417: pid 54427 exited with return code 7
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[2] on pgsql:0
> for client 54417: pid 54424 exited with return code 7
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> pgsql:0_monitor_0 (call=2, rc=7, cib-update=28, confirmed=true) not running
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz01 (local) - no waiting
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 7: start pgsql:0_start_0 on pz01 (local)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 start[5] (pid 54537)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 4: probe_complete=true
> Sep 11 11:50:12 pz01 crm_attribute: [54586]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-status -v STOP
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: pgsql-status (STOP)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 7: pgsql-status=STOP
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-pgsql-status, name=pgsql-status, value=STOP, magic=NA,
> cib=0.330.18) : Transient attribute: update
> Sep 11 11:50:12 pz01 crm_attribute: [54587]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-master-baseline -D
> Sep 11 11:50:12 pz01 crm_attribute: [54591]: info: Invoked: crm_attribute
> -N pz01 -n master-pgsql:0 -l reboot -v -INFINITY
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-pgsql:0 (-INFINITY)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 10: master-pgsql:0=-INFINITY
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-master-pgsql.0, name=master-pgsql:0, value=-INFINITY,
> magic=NA, cib=0.330.19) : Transient attribute: update
> Sep 11 11:50:12 pz01 crm_attribute: [54601]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-xlog-loc -D
> *Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation start[5] on pgsql:0
> for client 54417: pid 54537 exited with return code 1*
> *Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM
> operation pgsql:0_start_0 (call=5, rc=1, cib-update=29, confirmed=true)
> unknown error*
> Sep 11 11:50:12 pz01 crmd: [54417]: WARN: status_from_rc: Action 7
> (pgsql:0_start_0) on pz01 failed (target: 0 vs. rc: 1): Error
> Sep 11 11:50:12 pz01 crmd: [54417]: WARN: update_failcount: Updating
> failcount for pgsql:0 on pz01 after failed start: rc=1 (update=INFINITY,
> time=1410450612)
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> match_graph_event:277 - Triggered transition abort (complete=0,
> tag=lrm_rsc_op, id=pgsql:0_last_failure_0,
> magic=0:1;7:0:0:af61405d-b520-4e01-8f85-0443bd508a04, cib=0.330.20) : Event
> failed
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 46: notify pgsql:0_post_notify_start_0 on pz01 (local)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: fail-count-pgsql:0 (INFINITY)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 notify[6] (pid 54616)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 13: fail-count-pgsql:0=INFINITY
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-fail-count-pgsql.0, name=fail-count-pgsql:0, value=INFINITY,
> magic=NA, cib=0.330.21) : Transient attribute: update
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: last-failure-pgsql:0 (1410450612)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 16: last-failure-pgsql:0=1410450612
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-last-failure-pgsql.0, name=last-failure-pgsql:0,
> value=1410450612, magic=NA, cib=0.330.22) : Transient attribute: update
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation notify[6] on pgsql:0
> for client 54417: pid 54616 exited with return code 0
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> pgsql:0_notify_0 (call=6, rc=0, cib-update=0, confirmed=true) ok
> Sep 11 11:50:12 pz01 crmd: [54417]: notice: run_graph: ==== Transition 0
> (Complete=13, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> Source=/var/lib/pengine/pe-input-1198.bz2): Stopped
> Sep 11 11:50:12 pz01 crmd: [54417]: notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 11 11:50:12 pz01 pengine: [54416]: WARN: unpack_rsc_op: Processing
> failed op pgsql:0_last_failure_0 on pz01: unknown error (1)
> Sep 11 11:50:12 pz01 pengine: [54416]: WARN: common_apply_stickiness:
> Forcing msPostgresql away from pz01 after 1000000 failures (max=1)
> Sep 11 11:50:12 pz01 pengine: [54416]: WARN: common_apply_stickiness:
> Forcing msPostgresql away from pz01 after 1000000 failures (max=1)
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: LogActions: Stop pgsql:0
> (pz01)
>
>
>
>
> *But If I change to “rep_mode=async” then I get 'invalid parameter’ from *
> *pgsql:0_monitor_0*
>
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: LogActions: Start pgsql:0
> (pz01)
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:54:27 pz01 crmd: [54885]: info: do_te_invoke: Processing graph 0
> (ref=pe_calc-dc-1410450867-7) derived from
> /var/lib/pengine/pe-input-1202.bz2
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 4: monitor pgsql:0_monitor_0 on pz01 (local)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:pgsql:0 probe[2] (pid 54892)
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: process_pe_message:
> Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-1202.bz2
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 5: monitor vip-master_monitor_0 on pz01 (local)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:vip-master probe[3] (pid
> 54894)
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 6: monitor vip-rep_monitor_0 on pz01 (local)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:vip-rep probe[4] (pid 54895)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[3] on
> vip-master for client 54885: pid 54894 exited with return code 7
> Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM operation
> vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
> running
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[4] on vip-rep
> for client 54885: pid 54895 exited with return code 7
> Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM operation
> vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
> *Sep 11 11:54:27 pz01 lrmd: [54882]: info: RA output:
> (pgsql:0:probe:stderr) /usr/lib/ocf/resource.d//heartbeat/pgsql: 1646:
> /usr/lib/ocf/resource.d//heartbeat/pgsql: Bad substitution*
>
> *Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[2] on pgsql:0
> for client 54885: pid 54892 exited with return code 2*
> *Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM
> operation pgsql:0_monitor_0 (call=2, rc=2, cib-update=28, confirmed=true)
> invalid parameter*
> *Sep 11 11:54:27 pz01 crmd: [54885]: WARN: status_from_rc: Action 4
> (pgsql:0_monitor_0) on pz01 failed (target: 7 vs. rc: 2): Error*
> Sep 11 11:54:27 pz01 crmd: [54885]: info: abort_transition_graph:
> match_graph_event:277 - Triggered transition abort (complete=0,
> tag=lrm_rsc_op, id=pgsql:0_last_failure_0,
> magic=0:2;4:0:7:58ccffc4-db11-44a2-aa1a-bc79cd4924ae, cib=0.331.16) : Event
> failed
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz01 (local) - no waiting
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: run_graph: ==== Transition 0
> (Complete=7, Pending=0, Fired=0, Skipped=3, Incomplete=4,
> Source=/var/lib/pengine/pe-input-1202.bz2): Stopped
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 11 11:54:27 pz01 pengine: [54884]: ERROR: unpack_rsc_op: Preventing
> msPostgresql from re-starting on pz01: operation monitor failed 'invalid
> parameter' (rc=2)
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:54:27 pz01 crmd: [54885]: info: do_te_invoke: Processing graph 1
> (ref=pe_calc-dc-1410450867-12) derived from
> /var/lib/pengine/pe-input-1203.bz2
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz01 (local) - no waiting
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: run_graph: ==== Transition 1
> (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pengine/pe-input-1203.bz2): Complete
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_perform_update: Sent
> update 4: probe_complete=true
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: process_pe_message:
> Transition 1: PEngine Input stored in: /var/lib/pengine/pe-input-1203.bz2
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_perform_update: Sent
> update 6: probe_complete=true
>
>
>
> *Some questions:*
>
> 1. Could my issue be related with some erroneous postgres configuration?
> (I’ll check my postgres conf and I’ll post it)
> 2. How can I debug values from pgsl?
> For example to check values in pgsl:L1646
> sed -i "/${rep_mode_string//\//\\/}/d" $OCF_RESKEY_config
>
> If I use: *ocf_log err "$OCF_RESKEY_config”* in pgsql
> Where do I have to check this print? Because I’m not seeing it in
> corosync.log.
> Do I have to enable anything else?
>
> Thanks for your help!!
>
>
>
>
> 2014-09-11 3:06 GMT-03:00 Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de>
> :
>
>> 11.09.2014 um
>> 01:15 in Nachricht
>> <CAAbhU_DtkdGfJ425J3wWrUU7eX_oXs--CMcia-uGVjCnvqHbPQ@mail.gmail.com>:
>> > Hi.
>> >
>> > Let me ask some things,
>> >
>> > Can pz01 run under pacemker, without pz02?
>> > Something differed from running only pz02?
>> > Could you find some hints in the logs of each postgresql(and
>> pgstartup.log)?
>>
>> Sometimes it's handy to write a script for ocf-tester (many params to
>> pass) and run it in verbose and RA debugging mode (hoping the RA says
>> something reasonable). But stop the RA in the cluster first. Try the script
>> on both nodes looking for differences.
>>
>> >
>> > Good-luck,
>> >
>> > Takehiro Matsushima
>> > _______________________________________________
>> > Linux-HA mailing list
>> > Linux-HA@lists.linux-ha.org
>> > http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> > See also: http://linux-ha.org/ReportingProblems
>>
>>
>>
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA@lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
>>
>
>
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
Using ocf-tester I get:

ocf-tester -n pgsql -o repuser="ha" -o pgdba="postgres" -o
restart_on_promote="true" -o pgctl="/usr/lib/postgresql/9.1/bin/pg_ctl" -o
psql="/usr/lib/postgresql/9.1/bin/psql" -o
pgdata="/var/lib/postgresql/9.1/main/" -o
config="/etc/postgresql/9.1/main/postgresql.conf" -o rep_mode="async" -o
node_list="pz01 pz02" -o restore_command="cp
/var/lib/postgresql/9.1/main/archive/%f %p" -o
primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5
keepalives_count=5" -o master_ip="10.10.10.80" -o stop_escalate="0"
/usr/lib/ocf/resource.d/heartbeat/pgsql
Beginning tests for /usr/lib/ocf/resource.d/heartbeat/pgsql...
/usr/sbin/ocf-tester: 268: export: /var/lib/postgresql/9.1/main/archive/%f:
bad variable name

Is this the reason why I get `invalid parameter` ? Do you know what is
wrong there?

And without restore_command:

ocf-tester -n msPgsql -o repuser="ha" -o pgdba="postgres" -o
restart_on_promote="true" -o pgctl="/usr/lib/postgresql/9.1/bin/pg_ctl" -o
psql="/usr/lib/postgresql/9.1/bin/psql" -o
pgdata="/var/lib/postgresql/9.1/main/" -o
config="/etc/postgresql/9.1/main/postgresql.conf" -o rep_mode="async" -o
node_list="pz01 pz02" -o primary_conninfo_opt="keepalives_idle=60
keepalives_interval=5 keepalives_count=5" -o master_ip="10.10.10.80" -o
stop_escalate="0" /usr/lib/ocf/resource.d/heartbeat/pgsql
Beginning tests for /usr/lib/ocf/resource.d/heartbeat/pgsql...
* rc=6: Validation failed. Did you supply enough options with -o ?
pgsql[57665]: ERROR: Replication(rep_mode=async or sync) requires
Master/Slave configuration.
Aborting tests

Thanks in advanced

2014-09-11 12:07 GMT-03:00 Federico Castro <fcastelco@gmail.com>:

> Takehiro, Ulrich, thanks for your responses.
> I have some interesting information.
>
> If I only run pz02, then it keeps on starting as Postgres Slave, it is not
> promoted to Master:
>
> Online: [ pz02 ]
> OFFLINE: [ pz01 ]
>
> Full list of resources:
>
> Master/Slave Set: msPostgresql [pgsql]
> Slaves: [ pz02 ]
> Stopped: [ pgsql:1 ]
> Resource Group: master-group
> vip-master (ocf::heartbeat:IPaddr2): Stopped
> vip-rep (ocf::heartbeat:IPaddr2): Stopped
>
> Node Attributes:
> * Node pz02:
> + master-pgsql:0 : -INFINITY
> + pgsql-data-status : DISCONNECT
> + pgsql-status : HS:alone
>
>
> Logs:
>
> Sep 11 11:36:00 pz02 pengine: [42962]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 11 11:36:00 pz02 pengine: [42962]: notice: LogActions: Start pgsql:0
> (pz02)
> Sep 11 11:36:00 pz02 crmd: [42963]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:36:00 pz02 crmd: [42963]: info: do_te_invoke: Processing graph 0
> (ref=pe_calc-dc-1410446160-7) derived from
> /var/lib/pengine/pe-input-3875.bz2
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 4: monitor pgsql:0_monitor_0 on pz02 (local)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:pgsql:0 probe[2] (pid 42970)
> Sep 11 11:36:00 pz02 pengine: [42962]: notice: process_pe_message:
> Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-3875.bz2
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 5: monitor vip-master_monitor_0 on pz02 (local)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:vip-master probe[3] (pid
> 42972)
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 6: monitor vip-rep_monitor_0 on pz02 (local)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:vip-rep probe[4] (pid 42973)
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[3] on
> vip-master for client 42963: pid 42972 exited with return code 7
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[4] on vip-rep
> for client 42963: pid 42973 exited with return code 7
> Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
> running
> Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: operation monitor[2] on pgsql:0
> for client 42963: pid 42970 exited with return code 7
> Sep 11 11:36:00 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> pgsql:0_monitor_0 (call=2, rc=7, cib-update=28, confirmed=true) not running
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz02 (local) - no waiting
> Sep 11 11:36:00 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 7: start pgsql:0_start_0 on pz02 (local)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 4: probe_complete=true
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: rsc:pgsql:0 start[5] (pid 43082)
> Sep 11 11:36:00 pz02 crm_attribute: [43131]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-status -v STOP
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: pgsql-status (STOP)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 7: pgsql-status=STOP
> Sep 11 11:36:00 pz02 crmd: [42963]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz02-pgsql-status, name=pgsql-status, value=STOP, magic=NA,
> cib=0.287.18) : Transient attribute: update
> Sep 11 11:36:00 pz02 crm_attribute: [43132]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-master-baseline -D
> Sep 11 11:36:00 pz02 crm_attribute: [43136]: info: Invoked: crm_attribute
> -N pz02 -n master-pgsql:0 -l reboot -v -INFINITY
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-pgsql:0 (-INFINITY)
> Sep 11 11:36:00 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 10: master-pgsql:0=-INFINITY
> Sep 11 11:36:00 pz02 crmd: [42963]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz02-master-pgsql.0, name=master-pgsql:0, value=-INFINITY,
> magic=NA, cib=0.287.19) : Transient attribute: update
> Sep 11 11:36:00 pz02 crm_attribute: [43146]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-xlog-loc -D
> Sep 11 11:36:00 pz02 lrmd: [42960]: info: RA output:
> (pgsql:0:start:stderr) psql: could not connect to server: No such file or
> directory
> Is the server running locally and accepting
> connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
>
>
> Sep 11 11:36:02 pz02 crm_attribute: [43259]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz02 -n pgsql-status -v HS:alone
> Sep 11 11:36:02 pz02 attrd: [42961]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: pgsql-status (HS:alone)
> Sep 11 11:36:02 pz02 lrmd: [42960]: info: operation start[5] on pgsql:0
> for client 42963: pid 43082 exited with return code 0
> Sep 11 11:36:02 pz02 attrd: [42961]: notice: attrd_perform_update: Sent
> update 12: pgsql-status=HS:alone
> Sep 11 11:36:02 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> pgsql:0_start_0 (call=5, rc=0, cib-update=29, confirmed=true) ok
> Sep 11 11:36:02 pz02 crmd: [42963]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz02-pgsql-status, name=pgsql-status, value=HS:alone, magic=NA,
> cib=0.287.20) : Transient attribute: update
> Sep 11 11:36:02 pz02 crmd: [42963]: info: te_rsc_command: Initiating
> action 46: notify pgsql:0_post_notify_start_0 on pz02 (local)
> Sep 11 11:36:02 pz02 lrmd: [42960]: info: rsc:pgsql:0 notify[6] (pid 43260)
> Sep 11 11:36:02 pz02 lrmd: [42960]: info: operation notify[6] on pgsql:0
> for client 42963: pid 43260 exited with return code 0
> Sep 11 11:36:02 pz02 crmd: [42963]: info: process_lrm_event: LRM operation
> pgsql:0_notify_0 (call=6, rc=0, cib-update=0, confirmed=true) ok
> Sep 11 11:36:02 pz02 crmd: [42963]: notice: run_graph: ==== Transition 0
> (Complete=13, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> Source=/var/lib/pengine/pe-input-3875.bz2): Stopped
> But it is correctly DC: [42963]: info: update_dc: Set DC to pz02 (3.0.6)
>
>
>
>
>
> *I’ve found that using "rep_mode=sync” with my actual configuration then I
> get “unknown error” when pgsl try to start Postgres:*
>
>
>
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: LogActions: Start pgsql:0
> (pz01)
> Sep 11 11:50:12 pz01 crmd: [54417]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:50:12 pz01 crmd: [54417]: info: do_te_invoke: Processing graph 0
> (ref=pe_calc-dc-1410450612-7) derived from
> /var/lib/pengine/pe-input-1198.bz2
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 4: monitor pgsql:0_monitor_0 on pz01 (local)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 probe[2] (pid 54424)
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 5: monitor vip-master_monitor_0 on pz01 (local)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:vip-master probe[3] (pid
> 54425)
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 6: monitor vip-rep_monitor_0 on pz01 (local)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:vip-rep probe[4] (pid 54427)
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: process_pe_message:
> Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-1198.bz2
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[3] on
> vip-master for client 54417: pid 54425 exited with return code 7
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
> running
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[4] on vip-rep
> for client 54417: pid 54427 exited with return code 7
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation monitor[2] on pgsql:0
> for client 54417: pid 54424 exited with return code 7
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> pgsql:0_monitor_0 (call=2, rc=7, cib-update=28, confirmed=true) not running
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz01 (local) - no waiting
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 7: start pgsql:0_start_0 on pz01 (local)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 start[5] (pid 54537)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 4: probe_complete=true
> Sep 11 11:50:12 pz01 crm_attribute: [54586]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-status -v STOP
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: pgsql-status (STOP)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 7: pgsql-status=STOP
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-pgsql-status, name=pgsql-status, value=STOP, magic=NA,
> cib=0.330.18) : Transient attribute: update
> Sep 11 11:50:12 pz01 crm_attribute: [54587]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-master-baseline -D
> Sep 11 11:50:12 pz01 crm_attribute: [54591]: info: Invoked: crm_attribute
> -N pz01 -n master-pgsql:0 -l reboot -v -INFINITY
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: master-pgsql:0 (-INFINITY)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 10: master-pgsql:0=-INFINITY
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-master-pgsql.0, name=master-pgsql:0, value=-INFINITY,
> magic=NA, cib=0.330.19) : Transient attribute: update
> Sep 11 11:50:12 pz01 crm_attribute: [54601]: info: Invoked:
> /usr/sbin/crm_attribute -l reboot -N pz01 -n pgsql-xlog-loc -D
> *Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation start[5] on pgsql:0
> for client 54417: pid 54537 exited with return code 1*
> *Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM
> operation pgsql:0_start_0 (call=5, rc=1, cib-update=29, confirmed=true)
> unknown error*
> Sep 11 11:50:12 pz01 crmd: [54417]: WARN: status_from_rc: Action 7
> (pgsql:0_start_0) on pz01 failed (target: 0 vs. rc: 1): Error
> Sep 11 11:50:12 pz01 crmd: [54417]: WARN: update_failcount: Updating
> failcount for pgsql:0 on pz01 after failed start: rc=1 (update=INFINITY,
> time=1410450612)
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> match_graph_event:277 - Triggered transition abort (complete=0,
> tag=lrm_rsc_op, id=pgsql:0_last_failure_0,
> magic=0:1;7:0:0:af61405d-b520-4e01-8f85-0443bd508a04, cib=0.330.20) : Event
> failed
> Sep 11 11:50:12 pz01 crmd: [54417]: info: te_rsc_command: Initiating
> action 46: notify pgsql:0_post_notify_start_0 on pz01 (local)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: fail-count-pgsql:0 (INFINITY)
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: rsc:pgsql:0 notify[6] (pid 54616)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 13: fail-count-pgsql:0=INFINITY
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-fail-count-pgsql.0, name=fail-count-pgsql:0, value=INFINITY,
> magic=NA, cib=0.330.21) : Transient attribute: update
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: last-failure-pgsql:0 (1410450612)
> Sep 11 11:50:12 pz01 attrd: [54415]: notice: attrd_perform_update: Sent
> update 16: last-failure-pgsql:0=1410450612
> Sep 11 11:50:12 pz01 crmd: [54417]: info: abort_transition_graph:
> te_update_diff:176 - Triggered transition abort (complete=0, tag=nvpair,
> id=status-pz01-last-failure-pgsql.0, name=last-failure-pgsql:0,
> value=1410450612, magic=NA, cib=0.330.22) : Transient attribute: update
> Sep 11 11:50:12 pz01 lrmd: [54414]: info: operation notify[6] on pgsql:0
> for client 54417: pid 54616 exited with return code 0
> Sep 11 11:50:12 pz01 crmd: [54417]: info: process_lrm_event: LRM operation
> pgsql:0_notify_0 (call=6, rc=0, cib-update=0, confirmed=true) ok
> Sep 11 11:50:12 pz01 crmd: [54417]: notice: run_graph: ==== Transition 0
> (Complete=13, Pending=0, Fired=0, Skipped=1, Incomplete=0,
> Source=/var/lib/pengine/pe-input-1198.bz2): Stopped
> Sep 11 11:50:12 pz01 crmd: [54417]: notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 11 11:50:12 pz01 pengine: [54416]: WARN: unpack_rsc_op: Processing
> failed op pgsql:0_last_failure_0 on pz01: unknown error (1)
> Sep 11 11:50:12 pz01 pengine: [54416]: WARN: common_apply_stickiness:
> Forcing msPostgresql away from pz01 after 1000000 failures (max=1)
> Sep 11 11:50:12 pz01 pengine: [54416]: WARN: common_apply_stickiness:
> Forcing msPostgresql away from pz01 after 1000000 failures (max=1)
> Sep 11 11:50:12 pz01 pengine: [54416]: notice: LogActions: Stop pgsql:0
> (pz01)
>
>
>
>
> *But If I change to “rep_mode=async” then I get 'invalid parameter’ from *
> *pgsql:0_monitor_0*
>
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: LogActions: Start pgsql:0
> (pz01)
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:54:27 pz01 crmd: [54885]: info: do_te_invoke: Processing graph 0
> (ref=pe_calc-dc-1410450867-7) derived from
> /var/lib/pengine/pe-input-1202.bz2
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 4: monitor pgsql:0_monitor_0 on pz01 (local)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:pgsql:0 probe[2] (pid 54892)
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: process_pe_message:
> Transition 0: PEngine Input stored in: /var/lib/pengine/pe-input-1202.bz2
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 5: monitor vip-master_monitor_0 on pz01 (local)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:vip-master probe[3] (pid
> 54894)
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 6: monitor vip-rep_monitor_0 on pz01 (local)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: rsc:vip-rep probe[4] (pid 54895)
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[3] on
> vip-master for client 54885: pid 54894 exited with return code 7
> Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM operation
> vip-master_monitor_0 (call=3, rc=7, cib-update=26, confirmed=true) not
> running
> Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[4] on vip-rep
> for client 54885: pid 54895 exited with return code 7
> Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM operation
> vip-rep_monitor_0 (call=4, rc=7, cib-update=27, confirmed=true) not running
> *Sep 11 11:54:27 pz01 lrmd: [54882]: info: RA output:
> (pgsql:0:probe:stderr) /usr/lib/ocf/resource.d//heartbeat/pgsql: 1646:
> /usr/lib/ocf/resource.d//heartbeat/pgsql: Bad substitution*
>
> *Sep 11 11:54:27 pz01 lrmd: [54882]: info: operation monitor[2] on pgsql:0
> for client 54885: pid 54892 exited with return code 2*
> *Sep 11 11:54:27 pz01 crmd: [54885]: info: process_lrm_event: LRM
> operation pgsql:0_monitor_0 (call=2, rc=2, cib-update=28, confirmed=true)
> invalid parameter*
> *Sep 11 11:54:27 pz01 crmd: [54885]: WARN: status_from_rc: Action 4
> (pgsql:0_monitor_0) on pz01 failed (target: 7 vs. rc: 2): Error*
> Sep 11 11:54:27 pz01 crmd: [54885]: info: abort_transition_graph:
> match_graph_event:277 - Triggered transition abort (complete=0,
> tag=lrm_rsc_op, id=pgsql:0_last_failure_0,
> magic=0:2;4:0:7:58ccffc4-db11-44a2-aa1a-bc79cd4924ae, cib=0.331.16) : Event
> failed
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz01 (local) - no waiting
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: run_graph: ==== Transition 0
> (Complete=7, Pending=0, Fired=0, Skipped=3, Incomplete=4,
> Source=/var/lib/pengine/pe-input-1202.bz2): Stopped
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: unpack_config: On loss of
> CCM Quorum: Ignore
> Sep 11 11:54:27 pz01 pengine: [54884]: ERROR: unpack_rsc_op: Preventing
> msPostgresql from re-starting on pz01: operation monitor failed 'invalid
> parameter' (rc=2)
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Sep 11 11:54:27 pz01 crmd: [54885]: info: do_te_invoke: Processing graph 1
> (ref=pe_calc-dc-1410450867-12) derived from
> /var/lib/pengine/pe-input-1203.bz2
> Sep 11 11:54:27 pz01 crmd: [54885]: info: te_rsc_command: Initiating
> action 3: probe_complete probe_complete on pz01 (local) - no waiting
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: run_graph: ==== Transition 1
> (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pengine/pe-input-1203.bz2): Complete
> Sep 11 11:54:27 pz01 crmd: [54885]: notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_perform_update: Sent
> update 4: probe_complete=true
> Sep 11 11:54:27 pz01 pengine: [54884]: notice: process_pe_message:
> Transition 1: PEngine Input stored in: /var/lib/pengine/pe-input-1203.bz2
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_trigger_update: Sending
> flush op to all hosts for: probe_complete (true)
> Sep 11 11:54:27 pz01 attrd: [54883]: notice: attrd_perform_update: Sent
> update 6: probe_complete=true
>
>
>
> *Some questions:*
>
> 1. Could my issue be related with some erroneous postgres configuration?
> (I’ll check my postgres conf and I’ll post it)
> 2. How can I debug values from pgsl?
> For example to check values in pgsl:L1646
> sed -i "/${rep_mode_string//\//\\/}/d" $OCF_RESKEY_config
>
> If I use: *ocf_log err "$OCF_RESKEY_config”* in pgsql
> Where do I have to check this print? Because I’m not seeing it in
> corosync.log.
> Do I have to enable anything else?
>
> Thanks for your help!!
>
>
>
>
> 2014-09-11 3:06 GMT-03:00 Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de>
> :
>
>> 11.09.2014 um
>> 01:15 in Nachricht
>> <CAAbhU_DtkdGfJ425J3wWrUU7eX_oXs--CMcia-uGVjCnvqHbPQ@mail.gmail.com>:
>> > Hi.
>> >
>> > Let me ask some things,
>> >
>> > Can pz01 run under pacemker, without pz02?
>> > Something differed from running only pz02?
>> > Could you find some hints in the logs of each postgresql(and
>> pgstartup.log)?
>>
>> Sometimes it's handy to write a script for ocf-tester (many params to
>> pass) and run it in verbose and RA debugging mode (hoping the RA says
>> something reasonable). But stop the RA in the cluster first. Try the script
>> on both nodes looking for differences.
>>
>> >
>> > Good-luck,
>> >
>> > Takehiro Matsushima
>> > _______________________________________________
>> > Linux-HA mailing list
>> > Linux-HA@lists.linux-ha.org
>> > http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> > See also: http://linux-ha.org/ReportingProblems
>>
>>
>>
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA@lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
>>
>
>
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
[...]
If I use: *ocf_log err "$OCF_RESKEY_config”* in pgsql
Where do I have to check this print? Because I’m not seeing it in
corosync.log.
[...]

It depends what log you configured. In my configuration (and probably yours
also) these messages should go to syslog. Maybe try ;-)
ocf_log err "HEY, LOOK here: $OCF_RESKEY_config”

_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
Hi!

my hints were for debugging your own RA, but you can also debug other people's RAs. Consider this script I wrote for one of my own RAs:
(blocktune-test.sh)
#RA=/usr/lib/ocf/resource.d/xola/blocktune
RA=./blocktune
if [ "$1" = "manual" ]; then
shift
OCF_ROOT=/usr/lib/ocf OCF_RESOURCE_INSTANCE=blocktune \
OCF_RESKEY_blockdev="loop0" \
OCF_RESKEY_parameter="queue/scheduler" \
OCF_RESKEY_onvalue="noop" \
OCF_RESKEY_offvalue="cfq" \
sh -x $RA "$@"
echo "Exit status is $?"
else
/usr/sbin/ocf-tester -n multipath \
-o blockdev="loop0" \
-o parameter="queue/scheduler" \
-o onvalue="noop" \
-o offvalue="cfq" \
$RA
fi
---
Note the "sh -x"! You'd use "./blocktune-test.sh manual start/stop/status/whatever" to test a single operation, and without parameters the usual ocf-tester would be used.
I guess you get the idea of how to adjust my sample script for your needs.
Hint: If you have installed the RA you should use the official path; other wise use the RA in the current directory

Regards,
Ulrich


_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
Great! Thanks for your tips. I found the problem.

Bad substitution error in line 1646

L1646: sed -i "/${rep_mode_string//\//\\/}/d" $OCF_RESKEY_config

was when trying to delete string "include
'/var/lib/pgsql/tmp/rep_mode.conf' # added by pgsql RA’” in my config file
"/etc/postgresql/9.1/main/postgresql.conf”

It was caused because Debian 7 /bin/sh is a symlink to /bin/dash (/bin/sh
-> dash) and substitution is not working there, I changed script
interpreter to bash and it is working.

Thanks again.


2014-09-12 3:13 GMT-03:00 Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de>:

> Hi!
>
> my hints were for debugging your own RA, but you can also debug other
> people's RAs. Consider this script I wrote for one of my own RAs:
> (blocktune-test.sh)
> #RA=/usr/lib/ocf/resource.d/xola/blocktune
> RA=./blocktune
> if [ "$1" = "manual" ]; then
> shift
> OCF_ROOT=/usr/lib/ocf OCF_RESOURCE_INSTANCE=blocktune \
> OCF_RESKEY_blockdev="loop0" \
> OCF_RESKEY_parameter="queue/scheduler" \
> OCF_RESKEY_onvalue="noop" \
> OCF_RESKEY_offvalue="cfq" \
> sh -x $RA "$@"
> echo "Exit status is $?"
> else
> /usr/sbin/ocf-tester -n multipath \
> -o blockdev="loop0" \
> -o parameter="queue/scheduler" \
> -o onvalue="noop" \
> -o offvalue="cfq" \
> $RA
> fi
> ---
> Note the "sh -x"! You'd use "./blocktune-test.sh manual
> start/stop/status/whatever" to test a single operation, and without
> parameters the usual ocf-tester would be used.
> I guess you get the idea of how to adjust my sample script for your needs.
> Hint: If you have installed the RA you should use the official path; other
> wise use the RA in the current directory
>
> Regards,
> Ulrich
>
>
>
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
Hi,

On Fri, Sep 12, 2014 at 05:16:57PM -0300, Federico Castro wrote:
> Great! Thanks for your tips. I found the problem.
>
> Bad substitution error in line 1646
>
> L1646: sed -i "/${rep_mode_string//\//\\/}/d" $OCF_RESKEY_config
>
> was when trying to delete string "include
> '/var/lib/pgsql/tmp/rep_mode.conf' # added by pgsql RA’” in my config file
> "/etc/postgresql/9.1/main/postgresql.conf”
>
> It was caused because Debian 7 /bin/sh is a symlink to /bin/dash (/bin/sh
> -> dash) and substitution is not working there, I changed script
> interpreter to bash and it is working.

You should report this at github.com (or open bug with Debian).
It looks like a bashism and pgsql is a POSIX shell.

Thanks,

Dejan

> Thanks again.
>
>
> 2014-09-12 3:13 GMT-03:00 Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de>:
>
> > Hi!
> >
> > my hints were for debugging your own RA, but you can also debug other
> > people's RAs. Consider this script I wrote for one of my own RAs:
> > (blocktune-test.sh)
> > #RA=/usr/lib/ocf/resource.d/xola/blocktune
> > RA=./blocktune
> > if [ "$1" = "manual" ]; then
> > shift
> > OCF_ROOT=/usr/lib/ocf OCF_RESOURCE_INSTANCE=blocktune \
> > OCF_RESKEY_blockdev="loop0" \
> > OCF_RESKEY_parameter="queue/scheduler" \
> > OCF_RESKEY_onvalue="noop" \
> > OCF_RESKEY_offvalue="cfq" \
> > sh -x $RA "$@"
> > echo "Exit status is $?"
> > else
> > /usr/sbin/ocf-tester -n multipath \
> > -o blockdev="loop0" \
> > -o parameter="queue/scheduler" \
> > -o onvalue="noop" \
> > -o offvalue="cfq" \
> > $RA
> > fi
> > ---
> > Note the "sh -x"! You'd use "./blocktune-test.sh manual
> > start/stop/status/whatever" to test a single operation, and without
> > parameters the usual ocf-tester would be used.
> > I guess you get the idea of how to adjust my sample script for your needs.
> > Hint: If you have installed the RA you should use the official path; other
> > wise use the RA in the current directory
> >
> > Regards,
> > Ulrich
> >
> >
> >
> _______________________________________________
> Linux-HA mailing list
> Linux-HA@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
Re: Antw: Re: Postgresql RA fails starting master node [ In reply to ]
Hi,

>> L1646: sed -i "/${rep_mode_string//\//\\/}/d" $OCF_RESKEY_config

This bashism was reported in the following issue and it was fixed now.

https://github.com/ClusterLabs/resource-agents/issues/403

It is recommended that using the latest pgsql RA if you configure
replication.

On 2014/09/16 2:06, Dejan Muhamedagic wrote:
> Hi,
>
> On Fri, Sep 12, 2014 at 05:16:57PM -0300, Federico Castro wrote:
>> Great! Thanks for your tips. I found the problem.
>>
>> Bad substitution error in line 1646
>>
>> L1646: sed -i "/${rep_mode_string//\//\\/}/d" $OCF_RESKEY_config
>>
>> was when trying to delete string "include
>> '/var/lib/pgsql/tmp/rep_mode.conf' # added by pgsql RA’” in my config file
>> "/etc/postgresql/9.1/main/postgresql.conf”
>>
>> It was caused because Debian 7 /bin/sh is a symlink to /bin/dash (/bin/sh
>> -> dash) and substitution is not working there, I changed script
>> interpreter to bash and it is working.
>
> You should report this at github.com (or open bug with Debian).
> It looks like a bashism and pgsql is a POSIX shell.
>
> Thanks,
>
> Dejan
>
>> Thanks again.
>>
>>
>> 2014-09-12 3:13 GMT-03:00 Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de>:
>>
>>> Hi!
>>>
>>> my hints were for debugging your own RA, but you can also debug other
>>> people's RAs. Consider this script I wrote for one of my own RAs:
>>> (blocktune-test.sh)
>>> #RA=/usr/lib/ocf/resource.d/xola/blocktune
>>> RA=./blocktune
>>> if [ "$1" = "manual" ]; then
>>> shift
>>> OCF_ROOT=/usr/lib/ocf OCF_RESOURCE_INSTANCE=blocktune \
>>> OCF_RESKEY_blockdev="loop0" \
>>> OCF_RESKEY_parameter="queue/scheduler" \
>>> OCF_RESKEY_onvalue="noop" \
>>> OCF_RESKEY_offvalue="cfq" \
>>> sh -x $RA "$@"
>>> echo "Exit status is $?"
>>> else
>>> /usr/sbin/ocf-tester -n multipath \
>>> -o blockdev="loop0" \
>>> -o parameter="queue/scheduler" \
>>> -o onvalue="noop" \
>>> -o offvalue="cfq" \
>>> $RA
>>> fi
>>> ---
>>> Note the "sh -x"! You'd use "./blocktune-test.sh manual
>>> start/stop/status/whatever" to test a single operation, and without
>>> parameters the usual ocf-tester would be used.
>>> I guess you get the idea of how to adjust my sample script for your needs.
>>> Hint: If you have installed the RA you should use the official path; other
>>> wise use the RA in the current directory
>>>
>>> Regards,
>>> Ulrich
>>>
>>>
>>>
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA@lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
> _______________________________________________
> Linux-HA mailing list
> Linux-HA@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>
>


--
NTTデータ先端技術株式会社
中平 和友
TEL: 03-5860-5135 FAX: 03-5463-6490
Mail: nakahira_kazutomo_b1@lab.ntt.co.jp
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems