Mailing List Archive

Hi Dejan, all,

02.02.2012 19:44, Dejan Muhamedagic wrote:
> Hello all,
>
> Sorry for crossposting, but can anybody comment on the matter
> bellow? Thanks!

Running LVM operations can fail monitor op due timeouts. I experienced
that many times before I switched to home-brew RA for LVM.
There I only check for existence of /dev/VG[/LV].
Of course you need to obtain real status from LVM stack for start and
stop ops.

Please look at attached stripped-down version of RA I actually use (I
quickly removed bits of code which are very site-specific or too
experimental and are no interest for anyone). I wanted to send it long
ago, but you all know, some guys need "activation" to do something they
wish but do not actually need ;)

I'd say that RA is (near) production-quality and had been extensively
tested on several clusters. What is attached is probably twentieth
revision/rewrite, and it runs almost a year without modifications, not
causing any problems (unlike "stock" LVM RA for me).

If you wish, you may include some ideas from it into LVM RA or just
include attached as an alternative implementation (after some light
testing because of removed code).

It has enough comments/logs in critical sections so I hope it should be
clean to reader.

Main ideas lying behind that RA are:
* Do not run LVM commands on monitor (they are simply not needed). This
also helps to be tolerant to iSCSI link failures.
* Skip LVM locking where it is not needed (borrowed from RedHat's
lvm.sh). Useful when clvm waits for fencing (it would not allow any
command to succeed until fencing is done, so RA may timeout on monitor
without any matter when LV is actually available to system).
* Use timeout to not hang forever. Better is to try again.
* Use realtime scheduling priority (because otherwise LVM commands may
run for ages under high load even with well-tuned filter in LVM.conf).
This helps to reduce run time up to scale 20 - 60 to 3 secs in some
circumstances.
* Allow to separate VG/LV management:
** Allow VG to be just made known to system without activating any LVs
in it.
** Allow per-LV management (managing single LVs requires operation from
previous item to be done before).

The only limitation is that empty VGs are not supported (there is a
comment in code describing why).

And, it requires bash.

Hope you find that useful,

Best,
Vladislav

>
> Dejan
>
> On Tue, Jan 10, 2012 at 02:22:35PM +0100, Dejan Muhamedagic wrote:
>> Hi Hideo-san,
>>
>> On Tue, Jan 10, 2012 at 11:28:12AM +0900, renayama19661014@ybb.ne.jp wrote:
>>> Hi Dejan,
>>>
>>> How do you think about this matter?
>>
>> I'm still inclined to drop vgck from monitor and use it just
>> before start. I wouldn't even consider that a regression.
>>
>> I'm also not sure what does vgck offer in comparison with
>> vgdisplay and if both actually work with the on-disk lvm
>> meta-data. In that case we should drop vgdisplay as well and find
>> another (and better) way to monitor VGs.
>>
>> Anybody with deeper knowledge on LVM?
>>
>> Cheers,
>>
>> Dejan
>>
>>> Best Regards,
>>> Hideo Yamauchi.
>>>
>>>
>>> --- On Thu, 2011/12/8, renayama19661014@ybb.ne.jp <renayama19661014@ybb.ne.jp> wrote:
>>>
>>>> Hi Dejan,
>>>>
>>>> Thank you for comment.
>>>> We examine a correction of LVM_validate_all.
>>>> Because the handling of vgck influences it, I am going to obey the decision of this argument.
>>>>
>>>> For example, even the following simple choice may be good.
>>>> * Add "exec_vgck" parameter
>>>> * true(default) : Exec vgck command.
>>>> * false : Not exec vgck command.
>>>>
>>>> Best Regards,
>>>> Hideo Yamauchi.
>>>>
>>>> --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan@suse.de> wrote:
>>>>
>>>>> Hi Hideo-san,
>>>>>
>>>>> On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014@ybb.ne.jp wrote:
>>>>>> Hi Dejan,
>>>>>> Hi Xinwei,
>>>>>>
>>>>>> We pay attention to this correction, too.
>>>>>>
>>>>>> Did the discussion advance afterwards?
>>>>>> Was the method of the correction decided?
>>>>>
>>>>> Actually not, it somehow slipped off my radar.
>>>>>
>>>>> Xinwei suggested in his latest message to run vgck on probe, but
>>>>> this may result in error if the device isn't ready yet (it may
>>>>> happen say with iscsi). So, I think that we should just drop vgck
>>>>> altogether.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Dejan
>>>>>
>>>>>> Best Regards,
>>>>>> Hideo Yamauchi.
>>>>>>
>>>>>> _______________________________________________
>>>>>> ha-wg-technical mailing list
>>>>>> ha-wg-technical@lists.linux-foundation.org
>>>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>>>> _______________________________________________
>>>>> ha-wg-technical mailing list
>>>>> ha-wg-technical@lists.linux-foundation.org
>>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>>>>
>>>> _______________________________________________
>>>> ha-wg-technical mailing list
>>>> ha-wg-technical@lists.linux-foundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>>>
>>> _______________________________________________
>>> ha-wg-technical mailing list
>>> ha-wg-technical@lists.linux-foundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>> _______________________________________________
>> ha-wg-technical mailing list
>> ha-wg-technical@lists.linux-foundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> _______________________________________________________
> Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
> Home Page: http://linux-ha.org/

Hi Vladislav,

On Fri, Feb 03, 2012 at 10:33:52AM +0300, Vladislav Bogdanov wrote:
> Hi Dejan, all,
>
> 02.02.2012 19:44, Dejan Muhamedagic wrote:
> > Hello all,
> >
> > Sorry for crossposting, but can anybody comment on the matter
> > bellow? Thanks!
>
> Running LVM operations can fail monitor op due timeouts. I experienced
> that many times before I switched to home-brew RA for LVM.
> There I only check for existence of /dev/VG[/LV].
> Of course you need to obtain real status from LVM stack for start and
> stop ops.
>
> Please look at attached stripped-down version of RA I actually use (I
> quickly removed bits of code which are very site-specific or too
> experimental and are no interest for anyone). I wanted to send it long
> ago, but you all know, some guys need "activation" to do something they
> wish but do not actually need ;)
>
> I'd say that RA is (near) production-quality and had been extensively
> tested on several clusters. What is attached is probably twentieth
> revision/rewrite, and it runs almost a year without modifications, not
> causing any problems (unlike "stock" LVM RA for me).
>
> If you wish, you may include some ideas from it into LVM RA or just
> include attached as an alternative implementation (after some light
> testing because of removed code).
>
> It has enough comments/logs in critical sections so I hope it should be
> clean to reader.
>
> Main ideas lying behind that RA are:
> * Do not run LVM commands on monitor (they are simply not needed). This
> also helps to be tolerant to iSCSI link failures.

Thanks for confirming this.

> * Skip LVM locking where it is not needed (borrowed from RedHat's
> lvm.sh). Useful when clvm waits for fencing (it would not allow any
> command to succeed until fencing is done, so RA may timeout on monitor
> without any matter when LV is actually available to system).

Good point.

> * Use timeout to not hang forever. Better is to try again.

I'm sure it's better, but it would be good to know why. timeout
from coreutils is still fairly new.

> * Use realtime scheduling priority (because otherwise LVM commands may
> run for ages under high load even with well-tuned filter in LVM.conf).
> This helps to reduce run time up to scale 20 - 60 to 3 secs in some
> circumstances.

In case monitors are this light, how does this help? For start/stop?

> * Allow to separate VG/LV management:
> ** Allow VG to be just made known to system without activating any LVs
> in it.
> ** Allow per-LV management (managing single LVs requires operation from
> previous item to be done before).
>
> The only limitation is that empty VGs are not supported (there is a
> comment in code describing why).
>
> And, it requires bash.
>
> Hope you find that useful,

Probably, but just as an example. LVM2 is out of question:
IPaddr2/IPaddr turned out not to be such a great idea.

Thanks,

Dejan

> Best,
> Vladislav
>
> >
> > Dejan
> >
> > On Tue, Jan 10, 2012 at 02:22:35PM +0100, Dejan Muhamedagic wrote:
> >> Hi Hideo-san,
> >>
> >> On Tue, Jan 10, 2012 at 11:28:12AM +0900, renayama19661014@ybb.ne.jp wrote:
> >>> Hi Dejan,
> >>>
> >>> How do you think about this matter?
> >>
> >> I'm still inclined to drop vgck from monitor and use it just
> >> before start. I wouldn't even consider that a regression.
> >>
> >> I'm also not sure what does vgck offer in comparison with
> >> vgdisplay and if both actually work with the on-disk lvm
> >> meta-data. In that case we should drop vgdisplay as well and find
> >> another (and better) way to monitor VGs.
> >>
> >> Anybody with deeper knowledge on LVM?
> >>
> >> Cheers,
> >>
> >> Dejan
> >>
> >>> Best Regards,
> >>> Hideo Yamauchi.
> >>>
> >>>
> >>> --- On Thu, 2011/12/8, renayama19661014@ybb.ne.jp <renayama19661014@ybb.ne.jp> wrote:
> >>>
> >>>> Hi Dejan,
> >>>>
> >>>> Thank you for comment.
> >>>> We examine a correction of LVM_validate_all.
> >>>> Because the handling of vgck influences it, I am going to obey the decision of this argument.
> >>>>
> >>>> For example, even the following simple choice may be good.
> >>>> * Add "exec_vgck" parameter
> >>>> * true(default) : Exec vgck command.
> >>>> * false : Not exec vgck command.
> >>>>
> >>>> Best Regards,
> >>>> Hideo Yamauchi.
> >>>>
> >>>> --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan@suse.de> wrote:
> >>>>
> >>>>> Hi Hideo-san,
> >>>>>
> >>>>> On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014@ybb.ne.jp wrote:
> >>>>>> Hi Dejan,
> >>>>>> Hi Xinwei,
> >>>>>>
> >>>>>> We pay attention to this correction, too.
> >>>>>>
> >>>>>> Did the discussion advance afterwards?
> >>>>>> Was the method of the correction decided?
> >>>>>
> >>>>> Actually not, it somehow slipped off my radar.
> >>>>>
> >>>>> Xinwei suggested in his latest message to run vgck on probe, but
> >>>>> this may result in error if the device isn't ready yet (it may
> >>>>> happen say with iscsi). So, I think that we should just drop vgck
> >>>>> altogether.
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Dejan
> >>>>>
> >>>>>> Best Regards,
> >>>>>> Hideo Yamauchi.
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> ha-wg-technical mailing list
> >>>>>> ha-wg-technical@lists.linux-foundation.org
> >>>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> >>>>> _______________________________________________
> >>>>> ha-wg-technical mailing list
> >>>>> ha-wg-technical@lists.linux-foundation.org
> >>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> >>>>>
> >>>> _______________________________________________
> >>>> ha-wg-technical mailing list
> >>>> ha-wg-technical@lists.linux-foundation.org
> >>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> >>>>
> >>> _______________________________________________
> >>> ha-wg-technical mailing list
> >>> ha-wg-technical@lists.linux-foundation.org
> >>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> >> _______________________________________________
> >> ha-wg-technical mailing list
> >> ha-wg-technical@lists.linux-foundation.org
> >> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > _______________________________________________________
> > Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
> > http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
> > Home Page: http://linux-ha.org/
>

> #!/bin/bash
> #
> # LV management RA
> #
> # Copyright (c) 2011 Vladislav Bogdanov <bubble@hoster-ok.com>
> #
> # Partially based on LVM RA by Alan Robertson (Copyright: (C) 2002 - 2005
> # International Business Machines, Inc.) and lvm.sh RA by Redhat.
> #
> #######################################################################
> # Initialization:
>
> : ${OCF_FUNCTIONS_DIR=${OCF_ROOT}/lib/heartbeat}
> . ${OCF_FUNCTIONS_DIR}/ocf-shellfuncs
>
> #######################################################################
>
> OCF_RESKEY_activation_mode_default="auto"
>
> : ${OCF_RESKEY_activation_mode=${OCF_RESKEY_activation_mode_default}}
> : ${OCF_RESKEY_force_stop=0}
> : ${OCF_RESKEY_verify_stopped_on_stop=0}
>
> need_real_status=0
>
> usage() {
> cat <<EOF
> usage: $0 {start|stop|reload|monitor|validate-all|meta-data}
> EOF
> }
>
> meta_data() {
> cat <<EOF
> <?xml version="1.0"?>
> <!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
> <resource-agent name="LVM2">
> <version>1.0</version>
>
> <longdesc lang="en">
> Resource script for LVM. It manages an Linux Volume Manager volume (LVM)
> as an HA resource.
> </longdesc>
> <shortdesc lang="en">Controls the availability of an LVM Volume or Group</shortdesc>
>
> <parameters>
> <parameter name="vg_name" unique="0" required="1">
> <longdesc lang="en">
> The name of volume group.
> </longdesc>
> <shortdesc lang="en">Volume group name</shortdesc>
> <content type="string" default="" />
> </parameter>
>
> <parameter name="lv_name" unique="0">
> <longdesc lang="en">
> The name of the only logical volume to activate.
> If empty, then all volumes will be activated unless activation_mode
> is set to "none".
> </longdesc>
> <shortdesc lang="en">Logical volume name</shortdesc>
> <content type="string" default="" />
> </parameter>
>
> <parameter name="activation_mode" unique="0" required="0">
> <longdesc lang="en">
> Specifies activation mode for VG (LV).
> Could be one of:
> auto - Activate all volumes if none is specified by 'lv_name',
> otherwise activate only specified volume. Clustered
> volumes and groups are activated in local mode if
> resource is running as clone and in exclusive mode
> otherwise.
> none - only for VGs, do not activate/deactivate volumes,
> just make sure VG is known to kernel on start, and
> look for active volumes on monitor. Useful if one
> wants to separate VG and LV monitoring.
> local - only for (volumes in) clustered groups. Make local
> activation (-aly). Default if resource is run as clone.
> exclusive - only for (volumes in) clustered groups. Make exclusive
> activation (-aey). Default if resource is not run as
> clone. RA will complain if this is specified for cloned
> resource.
> </longdesc>
> <shortdesc lang="en">VG/LV activation mode</shortdesc>
> <content type="string" default="${OCF_RESKEY_activation_mode_default}" />
> </parameter>
>
> <parameter name="force_stop" unique="0">
> <longdesc lang="en">
> Force all logical volumes in group to be deactivated if
> activation_mode is set to "none". RA will fail in this case if deactivation failed.
> Only for VG-level resources (lv_name is empty).
> </longdesc>
> <shortdesc lang="en">Force deactivation of all volumes</shortdesc>
> <content type="boolean" default="0" />
> </parameter>
>
> <parameter name="verify_stopped_on_stop" unique="0">
> <longdesc lang="en">
> Fail on stop if activation_mode is set to "none" and VG has active volumes.
> Only for VG-level resources (lv_name is empty).
> </longdesc>
> <shortdesc lang="en">Fail on stop if VG has active volumes</shortdesc>
> <content type="boolean" default="0" />
> </parameter>
>
> <actions>
> <action name="start" timeout="240" />
> <action name="stop" timeout="240" />
> <action name="reload" timeout="120" />
> <action name="monitor" depth="0" timeout="60" interval="30" />
> <action name="meta-data" timeout="5" />
> <action name="validate-all" timeout="5" />
> </actions>
> </resource-agent>
> EOF
> }
>
> # Global vars
> clustered=
> activation_modifier=""
>
> check_activation_mode() {
>
> case ${OCF_RESKEY_activation_mode} in
> local)
> if [ ${clustered} -eq 0 ] ; then
> ocf_log err "Rejecting to operate in local activation mode for non-clustered volume, use activation_mode={auto|none} instead."
> return $OCF_ERR_CONFIGURED
> fi
> activation_modifier="l"
> ;;
> exclusive)
> if [ ${clustered} -eq 0 ] ; then
> ocf_log err "Rejecting to operate in exclusive activation mode for non-clustered volume, use activation_mode={auto|none} instead."
> return $OCF_ERR_CONFIGURED
> elif [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
> ocf_log err "Rejecting to operate in exclusive activation mode for clone resource."
> return $OCF_ERR_CONFIGURED
> fi
> activation_modifier="e"
> ;;
> none)
> if [ -n "${OCF_RESKEY_lv_name}" ] ; then
> ocf_log err "activation_mode=none cannot be used for volumes, only for volume groups."
> return $OCF_ERR_CONFIGURED
> fi
> ;;
> auto)
> if [ ${clustered} -eq 1 ] ; then
> if [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
> activation_modifier="l"
> else
> activation_modifier="e"
> fi
> else
> if [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
> ocf_log err "Rejecting to operate as clone for non-clustered volume group ${OCF_RESKEY_vg_name}."
> return $OCF_ERR_CONFIGURED
> fi
> fi
> ;;
> *)
> ocf_log err "Incorrect parameter activation_mode='${OCF_RESKEY_activation_mode}'."
> return $OCF_ERR_CONFIGURED
> ;;
> esac
>
> return $OCF_SUCCESS
> }
>
> # Version of status() function which does not use any of lvm utilities (except for start/stop).
> # Reason for this is weird LVM behavior whan utils do not work at all if
> # some PV becomes temporarily inaccessible and underlying layers (dm-multipath, iscsi)
> # block when accessing that device. This is extremely not-helpful for monitor operations.
> status() {
> local vg=$1
> local lv=$2
> local count
> local ret
> local output
> local tries
>
> if [ ! -d "/dev/${vg}" ] && [ ${need_real_status} = 1 ] ; then
> return $OCF_NOT_RUNNING
> fi
>
> if [ -z "${lv}" ] ; then
> # VG mode
> # Very special case: if vg is not activated on start, and we are not in stop
> if [ "${OCF_RESKEY_activation_mode}" = "none" ] && [ ${need_real_status} != 1 ] ; then
> ha_pseudo_resource ${ha_pseudo_resource_name} monitor
> return $?
> fi
>
> # All other cases:
> # Look if there are active LVs
> : $(( active = 0 ))
> if [ -d "/dev/${vg}" ] ; then # Otherwise string "/dev/${vg}/*" itself will be counted.
> for i in "/dev/${vg}/"* ; do
> : $(( active++ ))
> done
> fi
>
> if [ ${active} -eq 0 ] ; then
> # ACHTUNG! ACHTUNG!!! Empty VGs are NOT SUPPORTED
> # Longer version - it is impossible to distinguish between empty VG and not-running VG
> # without looking into physical block devices (f.e. with 'vgs').
> # But we can not look there, because in the case of iSCSI connection failure
> # we timeout and fail.
> ret=$OCF_NOT_RUNNING
> if [ "$__OCF_ACTION" = "start" ] ; then
> # Damn, (vg|lv)change are not a synchronous operation when it comes to /dev entries
> # Give chance for udev to create/delete them on start/stop on busy node
> : $(( tries = 0 ))
> while : ; do
> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg} 2>/dev/null )
> : $(( count = 0 ))
> : $(( active = 0 ))
> if [ $? -eq 124 ] || [ -z "${output}" ] ; then
> continue
> fi
> for i in ${output} ; do
> : $(( count++ ))
> : ${count} total volumes
> if [[ $i =~ ....a. ]] ; then
> : $(( active++ ))
> : ${active} active volumes
> fi
> done
> if (( active > 0 )) ; then
> ret=$OCF_SUCCESS
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> break
> fi
> sleep 1
> done
> fi
> else
> ret=$OCF_SUCCESS
> if [ "$__OCF_ACTION" = "stop" ] ; then
> : $(( tries = 0 ))
> while : ; do
> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg} 2>/dev/null )
> : $(( count = 0 ))
> : $(( active = 0 ))
> if [ $? -eq 124 ] || [ -z "${output}" ] ; then
> continue
> fi
> for i in ${output} ; do
> : $(( count++ ))
> : ${count} total volumes
> if [[ $i =~ ....a. ]] ; then
> : $(( active++ ))
> : ${active} active volumes
> fi
> done
> if (( active == 0 )) ; then
> ret=$OCF_NOT_RUNNING
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> break
> fi
> sleep 1
> done
> fi
> fi
> else
> if [ ! -e "/dev/${vg}/${lv}" ] ; then
> ret=$OCF_NOT_RUNNING
> if [ "$__OCF_ACTION" = "start" ] ; then
> : $(( tries = 0 ))
> while : ; do
> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
> if [[ $output =~ ....a. ]]; then
> ret=$OCF_SUCCESS
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> break
> fi
> sleep 1
> done
> fi
> else
> ret=$OCF_SUCCESS
> if [ "$__OCF_ACTION" = "stop" ] ; then
> : $(( tries = 0 ))
> while : ; do
> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
> if [ $? -ne 124 ] && [ -n "$output" ] && [[ ! $output =~ ....a. ]] ; then
> ret=$OCF_NOT_RUNNING
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> break
> fi
> sleep 1
> done
> fi
> fi
> fi
>
> return ${ret}
> }
>
> monitor() {
> local vg=$1
> local lv=$2
> local rc
> local output
>
> status ${vg} ${lv}
> rc=$?
>
> if [ ${rc} != $OCF_SUCCESS ] && ! ocf_is_probe ; then
> ocf_log warn "Volume group [${OCF_RESKEY_vg_name}] does not exist or is not accessible."
> ocf_log warn "For clustered volume groups verify that clvmd is started and working."
> fi
>
> return ${rc}
> }
>
> start() {
> local vg=$1
> local lv=$2
> local rc
> local scanned
> local flags
> local output
> local tries
>
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_SUCCESS ] ; then
> return ${rc}
> fi
>
> # We need to run vgscan in all cases, even if activation_mode = "none"
> scanned=0
> while : ; do
> while : ; do
> flags=$( timeout -s KILL 15 vgs --config 'log{command_names=0 prefix=""}' --nolocking -o attr --noheadings ${vg} 2>/dev/null )
> if [ $? -ne 124 ] ; then
> break
> fi
> done
> if [ -z "${flags}" ] ; then
> if [ ${scanned} -eq 1 ] ; then
> return $OCF_ERR_GENERIC
> else
> ocf_run vgscan
> scanned=1
> continue
> fi
> fi
> ocf_run timeout -s KILL 60 vgck ${vg}
> rc=$?
> if [ ${rc} -ne 0 ]; then
> if [ ${rc} -ne 124 ] ; then
> ocf_log err "Volume group [${vg}] contains errors."
> return $OCF_ERR_GENERIC
> fi
> fi
>
> # VG is known to system and is clean, break
> break
> done
>
> if [[ $flags =~ .....c ]]; then
> clustered=1
> else
> clustered=0
> fi
>
> echo "clustered=${clustered}" > "${STATEFILE}"
>
> check_activation_mode
> rc=$?
> if [ ${rc} != $OCF_SUCCESS ] ; then
> return ${rc}
> fi
>
> if [ -z "${lv}" ] ; then
> # VG mode
> if [ "${OCF_RESKEY_activation_mode}" != "none" ] ; then
> ocf_log info "Activating volume group [${vg}] (-a${activation_modifier}y)"
>
> : $(( tries = 0 ))
> while : ; do
> ocf_run timeout -s KILL 40 vgchange "-a${activation_modifier}y" ${vg}
> rc=$?
> if [ ${rc} -eq 124 ] ; then
> continue
> fi
> if [ ${rc} -ne 0 ] ; then
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_SUCCESS ] ; then
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> rc=$OCF_ERR_GENERIC
> break
> fi
> sleep 1
> else
> status ${vg} ${lv}
> rc=$?
> break
> fi
> done
> else
> ha_pseudo_resource ${ha_pseudo_resource_name} start
> status ${vg} ${lv}
> rc=$?
> fi
>
> if [ ${rc} -ne $OCF_SUCCESS ] ; then
> ocf_log err "Volume group [${vg}] did not activate correctly"
> output=$( timeout -s KILL 15 lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg} 2>/dev/null )
> if [ ${rc} -ne 124 ] ; then
> ocf_log info "${output}"
> fi
> fi
> else
> ocf_log info "Activating volume [${vg}/${lv}] (-a${activation_modifier}y)"
> : $(( tries = 0 ))
> while : ; do
> ocf_run timeout -s KILL 25 lvchange "-a${activation_modifier}y" ${vg}/${lv}
> rc=$?
> if [ ${rc} -eq 124 ] ; then
> continue
> fi
> if [ $rc -ne 0 ] ; then
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_SUCCESS ] ; then
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> rc=$OCF_ERR_GENERIC
> break
> fi
> sleep 1
> else
> status ${vg} ${lv}
> rc=$?
> break
> fi
> done
>
> if [ ${rc} -ne $OCF_SUCCESS ] ; then
> ocf_log err "Volume [${vg}/${lv}] did not activate correctly"
> output=$( timeout -s KILL 15 lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
> if [ ${rc} -ne 124 ] ; then
> ocf_log info "${output}"
> fi
> fi
> fi
>
> return ${rc}
> }
>
> stop() {
> local vg=$1
> local lv=$2
> local rc
> local output
>
> # Quick hack to introduce needed features. Global variable...
> if [ "${OCF_RESKEY_activation_mode}" != "none" ] || \
> ocf_is_true ${OCF_RESKEY_force_stop} || \
> ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
> need_real_status=1
> fi
>
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
> return $OCF_SUCCESS
> elif [ ${rc} = $OCF_SUCCESS ] && [ "${OCF_RESKEY_activation_mode}" = "none" ] && ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
> ocf_log err "Not-managed volume group [${vg}] has active volumes on stop and verify_stopped_on_stop is true:"
> ocf_run timeout -s KILL 15 lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}
> return $OCF_ERR_GENERIC
> fi
>
> if [ -z "${lv}" ] ; then
> # VG mode
> if [ ${need_real_status} = 1 ] ; then
> if [ "${OCF_RESKEY_activation_mode}" != "none" ] || ocf_is_true ${OCF_RESKEY_force_stop} ; then
> ocf_log info "Deactivating volume group [${vg}] (-a${activation_modifier}n)"
> : $(( tries = 0 ))
> while : ; do
> ocf_run timeout -s KILL 40 vgchange "-a${activation_modifier}n" ${vg}
> rc=$?
> if [ ${rc} -ne 0 ] ; then
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
> rc=$OCF_SUCCESS
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> rc=$OCF_ERR_GENERIC
> break
> fi
> sleep 1
> else
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
> rc=$OCF_SUCCESS
> elif [ ${rc} -eq $OCF_SUCCESS ] ; then
> rc=$OCF_ERR_GENERIC
> fi
> break
> fi
> done
> if [ "${OCF_RESKEY_activation_mode}" = "none" ] ; then
> ha_pseudo_resource ${ha_pseudo_resource_name} stop
> fi
> else
> if [ "${OCF_RESKEY_activation_mode}" = "none" ] ; then
> ha_pseudo_resource ${ha_pseudo_resource_name} stop
> fi
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
> rc=$OCF_SUCCESS
> elif [ ${rc} -eq $OCF_SUCCESS ] ; then
> rc=$OCF_ERR_GENERIC
> fi
> fi
>
> if [ ${rc} -ne $OCF_SUCCESS ] ; then
> ocf_log err "Volume group [${vg}] did not stop correctly"
> output=$( lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
> ocf_log info "${output}"
> fi
> fi
> else
> ocf_log info "Deactivating volume [${vg}/${lv}] (-a${activation_modifier}n)"
> : $(( tries = 0 ))
> while : ; do
> ocf_run timeout -s KILL 25 lvchange "-a${activation_modifier}n" ${vg}/${lv}
> rc=$?
> if [ ${rc} -ne 0 ] ; then
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
> rc=$OCF_SUCCESS
> break
> fi
> : $(( tries++ ))
> if [ ${tries} -gt 10 ] ; then
> rc=$OCF_ERR_GENERIC
> break
> fi
> sleep 1
> else
> status ${vg} ${lv}
> rc=$?
> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
> rc=$OCF_SUCCESS
> elif [ ${rc} -eq $OCF_SUCCESS ] ; then
> rc=$OCF_ERR_GENERIC
> fi
> break
> fi
> done
>
> if [ ${rc} -ne $OCF_SUCCESS ] ; then
> ocf_log err "Volume [${vg}/${lv}] did not stop correctly"
> output=$( lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
> ocf_log info "${output}"
> fi
> fi
>
> return ${rc}
> }
>
> validate_all() {
> check_binary vgs
> check_binary vgscan
> check_binary vgck
> check_binary vgchange
> check_binary lvchange
> check_binary lvs
>
> if [ -z "${OCF_RESKEY_vg_name}" ] ; then
> exit $OCF_ERR_CONFIGURED
> fi
> case ${OCF_RESKEY_activation_mode} in
> exclusive)
> if [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
> ocf_log err "Rejecting to operate in exclusive activation mode for clone resource."
> return $OCF_ERR_CONFIGURED
> fi
> ;;
> none)
> if [ -n "${OCF_RESKEY_lv_name}" ] ; then
> ocf_log err "activation_mode=none cannot be used for volumes, only for volume groups."
> return $OCF_ERR_CONFIGURED
> fi
> ;;
> auto|local)
> ;;
> *)
> ocf_log err "Incorrect parameter activation_mode='${OCF_RESKEY_activation_mode}'."
> return $OCF_ERR_CONFIGURED
> ;;
> esac
>
> if ocf_is_true ${OCF_RESKEY_force_stop} || ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
> if [ -n "${OCF_RESKRY_lv_name}" ] ; then
> ocf_log err "'force_stop' or 'verify_stopped_on_stop' should be specified only for VG-level resources."
> return $OCF_ERR_CONFIGURED
> fi
> if [ "${OCF_RESKEY_activation_mode}" != "none" ] ; then
> ocf_log warn "'force_stop' and 'verify_stopped_on_stop' don't affect operation if activation_mode != 'none', please fix parameters for this resource."
> fi
> if ocf_is_true ${OCF_RESKEY_force_stop} && ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
> ocf_log err "'force_stop' and 'verify_stopped_on_stop' parameters are mutually exclusive."
> return $OCF_ERR_CONFIGURED
> fi
> fi
> }
>
>
> case $1 in
> meta-data)
> meta_data
> exit $OCF_SUCCESS
> ;;
> usage|help)
> usage
> exit $OCF_SUCCESS
> ;;
> esac
>
> # Everything except usage and meta-data must pass the validate test
> validate_all || exit $?
>
>
> : ${ha_pseudo_resource_name:=LVM2-${OCF_RESOURCE_INSTANCE}}
> STATEFILE="${HA_RSCTMP}/LVM2-${OCF_RESOURCE_INSTANCE}.state"
>
> if [ "$__OCF_ACTION" != "start" ] && [ "$__OCF_ACTION" != "reload" ] && [ -s "${STATEFILE}" ] ; then
> . "${STATEFILE}"
> check_activation_mode
> rc=$?
> if [ ${rc} != $OCF_SUCCESS ] ; then
> return ${rc}
> fi
> fi
>
> # Request high priority (very helpful under high load)
> chrt -p -r 10 $$
>
> # What kind of method was invoked?
> case "$__OCF_ACTION" in
> start|reload)
> start ${OCF_RESKEY_vg_name} ${OCF_RESKEY_lv_name}
> rc=$?
> ;;
> stop)
> stop ${OCF_RESKEY_vg_name} ${OCF_RESKEY_lv_name}
> rc=$?
>
> if [ ${rc} -eq 0 ] ; then
> rm -f "${STATEFILE}"
> fi
> ;;
> monitor)
> monitor ${OCF_RESKEY_vg_name} ${OCF_RESKEY_lv_name}
> rc=$?
> ;;
> validate-all)
> rc=$OCF_SUCCESS
> ;;
> *)
> usage
> exit $OCF_ERR_UNIMPLEMENTED
> ;;
> esac
>
> exit ${rc}

> _______________________________________________________
> Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
> Home Page: http://linux-ha.org/

_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/

Hi Dejan,

20.02.2012 20:23, Dejan Muhamedagic wrote:
> Hi Vladislav,
>
> On Fri, Feb 03, 2012 at 10:33:52AM +0300, Vladislav Bogdanov wrote:
>> Hi Dejan, all,
>>
>> 02.02.2012 19:44, Dejan Muhamedagic wrote:
>>> Hello all,
>>>
>>> Sorry for crossposting, but can anybody comment on the matter
>>> bellow? Thanks!
>>
>> Running LVM operations can fail monitor op due timeouts. I experienced
>> that many times before I switched to home-brew RA for LVM.
>> There I only check for existence of /dev/VG[/LV].
>> Of course you need to obtain real status from LVM stack for start and
>> stop ops.
>>
>> Please look at attached stripped-down version of RA I actually use (I
>> quickly removed bits of code which are very site-specific or too
>> experimental and are no interest for anyone). I wanted to send it long
>> ago, but you all know, some guys need "activation" to do something they
>> wish but do not actually need ;)
>>
>> I'd say that RA is (near) production-quality and had been extensively
>> tested on several clusters. What is attached is probably twentieth
>> revision/rewrite, and it runs almost a year without modifications, not
>> causing any problems (unlike "stock" LVM RA for me).
>>
>> If you wish, you may include some ideas from it into LVM RA or just
>> include attached as an alternative implementation (after some light
>> testing because of removed code).
>>
>> It has enough comments/logs in critical sections so I hope it should be
>> clean to reader.
>>
>> Main ideas lying behind that RA are:
>> * Do not run LVM commands on monitor (they are simply not needed). This
>> also helps to be tolerant to iSCSI link failures.
>
> Thanks for confirming this.
>
>> * Skip LVM locking where it is not needed (borrowed from RedHat's
>> lvm.sh). Useful when clvm waits for fencing (it would not allow any
>> command to succeed until fencing is done, so RA may timeout on monitor
>> without any matter when LV is actually available to system).
>
> Good point.
>
>> * Use timeout to not hang forever. Better is to try again.
>
> I'm sure it's better, but it would be good to know why. timeout
> from coreutils is still fairly new.

At least to print something sane to logs. "RA is timed out" is not very
informative. "LVM command XXX is timed out" has much more info.
I also suspect that some cLVM commands highly depend on a cluster and
DLM state which may change over the time (between two command runs). I
can't tell for sure, but I suspect possible deadlocks if command arrived
at the wrong time (when cluster state was wrong). Please note, that
pacemaker and DLM points of view on a cluster state may differ (f.e.
pacemaker with openais plugin allows quick node leave-join while DLM
does not). So pacemaker may run commands while dlm waits for fencing. I
fixed this for my setups, but upstream dlm_controld.pcmk (and version in
suse) is affected by this. This should change with corosync 2.0 where
pacemaker will use CPG too.

>
>> * Use realtime scheduling priority (because otherwise LVM commands may
>> run for ages under high load even with well-tuned filter in LVM.conf).
>> This helps to reduce run time up to scale 20 - 60 to 3 secs in some
>> circumstances.
>
> In case monitors are this light, how does this help? For start/stop?

Just for all LVM commands, and yes, vgchange/lvchange for start/stop are
critical ones.

>
>> * Allow to separate VG/LV management:
>> ** Allow VG to be just made known to system without activating any LVs
>> in it.
>> ** Allow per-LV management (managing single LVs requires operation from
>> previous item to be done before).
>>
>> The only limitation is that empty VGs are not supported (there is a
>> comment in code describing why).
>>
>> And, it requires bash.
>>
>> Hope you find that useful,
>
> Probably, but just as an example. LVM2 is out of question:
> IPaddr2/IPaddr turned out not to be such a great idea.

It's up to you.
BTW software package this RA works with is named LVM2 ;)

Best,
Vladislav

>
> Thanks,
>
> Dejan
>
>> Best,
>> Vladislav
>>
>>>
>>> Dejan
>>>
>>> On Tue, Jan 10, 2012 at 02:22:35PM +0100, Dejan Muhamedagic wrote:
>>>> Hi Hideo-san,
>>>>
>>>> On Tue, Jan 10, 2012 at 11:28:12AM +0900, renayama19661014@ybb.ne.jp wrote:
>>>>> Hi Dejan,
>>>>>
>>>>> How do you think about this matter?
>>>>
>>>> I'm still inclined to drop vgck from monitor and use it just
>>>> before start. I wouldn't even consider that a regression.
>>>>
>>>> I'm also not sure what does vgck offer in comparison with
>>>> vgdisplay and if both actually work with the on-disk lvm
>>>> meta-data. In that case we should drop vgdisplay as well and find
>>>> another (and better) way to monitor VGs.
>>>>
>>>> Anybody with deeper knowledge on LVM?
>>>>
>>>> Cheers,
>>>>
>>>> Dejan
>>>>
>>>>> Best Regards,
>>>>> Hideo Yamauchi.
>>>>>
>>>>>
>>>>> --- On Thu, 2011/12/8, renayama19661014@ybb.ne.jp <renayama19661014@ybb.ne.jp> wrote:
>>>>>
>>>>>> Hi Dejan,
>>>>>>
>>>>>> Thank you for comment.
>>>>>> We examine a correction of LVM_validate_all.
>>>>>> Because the handling of vgck influences it, I am going to obey the decision of this argument.
>>>>>>
>>>>>> For example, even the following simple choice may be good.
>>>>>> * Add "exec_vgck" parameter
>>>>>> * true(default) : Exec vgck command.
>>>>>> * false : Not exec vgck command.
>>>>>>
>>>>>> Best Regards,
>>>>>> Hideo Yamauchi.
>>>>>>
>>>>>> --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan@suse.de> wrote:
>>>>>>
>>>>>>> Hi Hideo-san,
>>>>>>>
>>>>>>> On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014@ybb.ne.jp wrote:
>>>>>>>> Hi Dejan,
>>>>>>>> Hi Xinwei,
>>>>>>>>
>>>>>>>> We pay attention to this correction, too.
>>>>>>>>
>>>>>>>> Did the discussion advance afterwards?
>>>>>>>> Was the method of the correction decided?
>>>>>>>
>>>>>>> Actually not, it somehow slipped off my radar.
>>>>>>>
>>>>>>> Xinwei suggested in his latest message to run vgck on probe, but
>>>>>>> this may result in error if the device isn't ready yet (it may
>>>>>>> happen say with iscsi). So, I think that we should just drop vgck
>>>>>>> altogether.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Dejan
>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Hideo Yamauchi.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ha-wg-technical mailing list
>>>>>>>> ha-wg-technical@lists.linux-foundation.org
>>>>>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>>>>>> _______________________________________________
>>>>>>> ha-wg-technical mailing list
>>>>>>> ha-wg-technical@lists.linux-foundation.org
>>>>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>>>>>>
>>>>>> _______________________________________________
>>>>>> ha-wg-technical mailing list
>>>>>> ha-wg-technical@lists.linux-foundation.org
>>>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>>>>>
>>>>> _______________________________________________
>>>>> ha-wg-technical mailing list
>>>>> ha-wg-technical@lists.linux-foundation.org
>>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>>> _______________________________________________
>>>> ha-wg-technical mailing list
>>>> ha-wg-technical@lists.linux-foundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>>> _______________________________________________________
>>> Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
>>> Home Page: http://linux-ha.org/
>>
>
>> #!/bin/bash
>> #
>> # LV management RA
>> #
>> # Copyright (c) 2011 Vladislav Bogdanov <bubble@hoster-ok.com>
>> #
>> # Partially based on LVM RA by Alan Robertson (Copyright: (C) 2002 - 2005
>> # International Business Machines, Inc.) and lvm.sh RA by Redhat.
>> #
>> #######################################################################
>> # Initialization:
>>
>> : ${OCF_FUNCTIONS_DIR=${OCF_ROOT}/lib/heartbeat}
>> . ${OCF_FUNCTIONS_DIR}/ocf-shellfuncs
>>
>> #######################################################################
>>
>> OCF_RESKEY_activation_mode_default="auto"
>>
>> : ${OCF_RESKEY_activation_mode=${OCF_RESKEY_activation_mode_default}}
>> : ${OCF_RESKEY_force_stop=0}
>> : ${OCF_RESKEY_verify_stopped_on_stop=0}
>>
>> need_real_status=0
>>
>> usage() {
>> cat <<EOF
>> usage: $0 {start|stop|reload|monitor|validate-all|meta-data}
>> EOF
>> }
>>
>> meta_data() {
>> cat <<EOF
>> <?xml version="1.0"?>
>> <!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
>> <resource-agent name="LVM2">
>> <version>1.0</version>
>>
>> <longdesc lang="en">
>> Resource script for LVM. It manages an Linux Volume Manager volume (LVM)
>> as an HA resource.
>> </longdesc>
>> <shortdesc lang="en">Controls the availability of an LVM Volume or Group</shortdesc>
>>
>> <parameters>
>> <parameter name="vg_name" unique="0" required="1">
>> <longdesc lang="en">
>> The name of volume group.
>> </longdesc>
>> <shortdesc lang="en">Volume group name</shortdesc>
>> <content type="string" default="" />
>> </parameter>
>>
>> <parameter name="lv_name" unique="0">
>> <longdesc lang="en">
>> The name of the only logical volume to activate.
>> If empty, then all volumes will be activated unless activation_mode
>> is set to "none".
>> </longdesc>
>> <shortdesc lang="en">Logical volume name</shortdesc>
>> <content type="string" default="" />
>> </parameter>
>>
>> <parameter name="activation_mode" unique="0" required="0">
>> <longdesc lang="en">
>> Specifies activation mode for VG (LV).
>> Could be one of:
>> auto - Activate all volumes if none is specified by 'lv_name',
>> otherwise activate only specified volume. Clustered
>> volumes and groups are activated in local mode if
>> resource is running as clone and in exclusive mode
>> otherwise.
>> none - only for VGs, do not activate/deactivate volumes,
>> just make sure VG is known to kernel on start, and
>> look for active volumes on monitor. Useful if one
>> wants to separate VG and LV monitoring.
>> local - only for (volumes in) clustered groups. Make local
>> activation (-aly). Default if resource is run as clone.
>> exclusive - only for (volumes in) clustered groups. Make exclusive
>> activation (-aey). Default if resource is not run as
>> clone. RA will complain if this is specified for cloned
>> resource.
>> </longdesc>
>> <shortdesc lang="en">VG/LV activation mode</shortdesc>
>> <content type="string" default="${OCF_RESKEY_activation_mode_default}" />
>> </parameter>
>>
>> <parameter name="force_stop" unique="0">
>> <longdesc lang="en">
>> Force all logical volumes in group to be deactivated if
>> activation_mode is set to "none". RA will fail in this case if deactivation failed.
>> Only for VG-level resources (lv_name is empty).
>> </longdesc>
>> <shortdesc lang="en">Force deactivation of all volumes</shortdesc>
>> <content type="boolean" default="0" />
>> </parameter>
>>
>> <parameter name="verify_stopped_on_stop" unique="0">
>> <longdesc lang="en">
>> Fail on stop if activation_mode is set to "none" and VG has active volumes.
>> Only for VG-level resources (lv_name is empty).
>> </longdesc>
>> <shortdesc lang="en">Fail on stop if VG has active volumes</shortdesc>
>> <content type="boolean" default="0" />
>> </parameter>
>>
>> <actions>
>> <action name="start" timeout="240" />
>> <action name="stop" timeout="240" />
>> <action name="reload" timeout="120" />
>> <action name="monitor" depth="0" timeout="60" interval="30" />
>> <action name="meta-data" timeout="5" />
>> <action name="validate-all" timeout="5" />
>> </actions>
>> </resource-agent>
>> EOF
>> }
>>
>> # Global vars
>> clustered=
>> activation_modifier=""
>>
>> check_activation_mode() {
>>
>> case ${OCF_RESKEY_activation_mode} in
>> local)
>> if [ ${clustered} -eq 0 ] ; then
>> ocf_log err "Rejecting to operate in local activation mode for non-clustered volume, use activation_mode={auto|none} instead."
>> return $OCF_ERR_CONFIGURED
>> fi
>> activation_modifier="l"
>> ;;
>> exclusive)
>> if [ ${clustered} -eq 0 ] ; then
>> ocf_log err "Rejecting to operate in exclusive activation mode for non-clustered volume, use activation_mode={auto|none} instead."
>> return $OCF_ERR_CONFIGURED
>> elif [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
>> ocf_log err "Rejecting to operate in exclusive activation mode for clone resource."
>> return $OCF_ERR_CONFIGURED
>> fi
>> activation_modifier="e"
>> ;;
>> none)
>> if [ -n "${OCF_RESKEY_lv_name}" ] ; then
>> ocf_log err "activation_mode=none cannot be used for volumes, only for volume groups."
>> return $OCF_ERR_CONFIGURED
>> fi
>> ;;
>> auto)
>> if [ ${clustered} -eq 1 ] ; then
>> if [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
>> activation_modifier="l"
>> else
>> activation_modifier="e"
>> fi
>> else
>> if [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
>> ocf_log err "Rejecting to operate as clone for non-clustered volume group ${OCF_RESKEY_vg_name}."
>> return $OCF_ERR_CONFIGURED
>> fi
>> fi
>> ;;
>> *)
>> ocf_log err "Incorrect parameter activation_mode='${OCF_RESKEY_activation_mode}'."
>> return $OCF_ERR_CONFIGURED
>> ;;
>> esac
>>
>> return $OCF_SUCCESS
>> }
>>
>> # Version of status() function which does not use any of lvm utilities (except for start/stop).
>> # Reason for this is weird LVM behavior whan utils do not work at all if
>> # some PV becomes temporarily inaccessible and underlying layers (dm-multipath, iscsi)
>> # block when accessing that device. This is extremely not-helpful for monitor operations.
>> status() {
>> local vg=$1
>> local lv=$2
>> local count
>> local ret
>> local output
>> local tries
>>
>> if [ ! -d "/dev/${vg}" ] && [ ${need_real_status} = 1 ] ; then
>> return $OCF_NOT_RUNNING
>> fi
>>
>> if [ -z "${lv}" ] ; then
>> # VG mode
>> # Very special case: if vg is not activated on start, and we are not in stop
>> if [ "${OCF_RESKEY_activation_mode}" = "none" ] && [ ${need_real_status} != 1 ] ; then
>> ha_pseudo_resource ${ha_pseudo_resource_name} monitor
>> return $?
>> fi
>>
>> # All other cases:
>> # Look if there are active LVs
>> : $(( active = 0 ))
>> if [ -d "/dev/${vg}" ] ; then # Otherwise string "/dev/${vg}/*" itself will be counted.
>> for i in "/dev/${vg}/"* ; do
>> : $(( active++ ))
>> done
>> fi
>>
>> if [ ${active} -eq 0 ] ; then
>> # ACHTUNG! ACHTUNG!!! Empty VGs are NOT SUPPORTED
>> # Longer version - it is impossible to distinguish between empty VG and not-running VG
>> # without looking into physical block devices (f.e. with 'vgs').
>> # But we can not look there, because in the case of iSCSI connection failure
>> # we timeout and fail.
>> ret=$OCF_NOT_RUNNING
>> if [ "$__OCF_ACTION" = "start" ] ; then
>> # Damn, (vg|lv)change are not a synchronous operation when it comes to /dev entries
>> # Give chance for udev to create/delete them on start/stop on busy node
>> : $(( tries = 0 ))
>> while : ; do
>> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg} 2>/dev/null )
>> : $(( count = 0 ))
>> : $(( active = 0 ))
>> if [ $? -eq 124 ] || [ -z "${output}" ] ; then
>> continue
>> fi
>> for i in ${output} ; do
>> : $(( count++ ))
>> : ${count} total volumes
>> if [[ $i =~ ....a. ]] ; then
>> : $(( active++ ))
>> : ${active} active volumes
>> fi
>> done
>> if (( active > 0 )) ; then
>> ret=$OCF_SUCCESS
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> break
>> fi
>> sleep 1
>> done
>> fi
>> else
>> ret=$OCF_SUCCESS
>> if [ "$__OCF_ACTION" = "stop" ] ; then
>> : $(( tries = 0 ))
>> while : ; do
>> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg} 2>/dev/null )
>> : $(( count = 0 ))
>> : $(( active = 0 ))
>> if [ $? -eq 124 ] || [ -z "${output}" ] ; then
>> continue
>> fi
>> for i in ${output} ; do
>> : $(( count++ ))
>> : ${count} total volumes
>> if [[ $i =~ ....a. ]] ; then
>> : $(( active++ ))
>> : ${active} active volumes
>> fi
>> done
>> if (( active == 0 )) ; then
>> ret=$OCF_NOT_RUNNING
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> break
>> fi
>> sleep 1
>> done
>> fi
>> fi
>> else
>> if [ ! -e "/dev/${vg}/${lv}" ] ; then
>> ret=$OCF_NOT_RUNNING
>> if [ "$__OCF_ACTION" = "start" ] ; then
>> : $(( tries = 0 ))
>> while : ; do
>> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
>> if [[ $output =~ ....a. ]]; then
>> ret=$OCF_SUCCESS
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> break
>> fi
>> sleep 1
>> done
>> fi
>> else
>> ret=$OCF_SUCCESS
>> if [ "$__OCF_ACTION" = "stop" ] ; then
>> : $(( tries = 0 ))
>> while : ; do
>> output=$( timeout -s KILL 10 lvs --config 'log{command_names=0 prefix=""}' -o attr --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
>> if [ $? -ne 124 ] && [ -n "$output" ] && [[ ! $output =~ ....a. ]] ; then
>> ret=$OCF_NOT_RUNNING
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> break
>> fi
>> sleep 1
>> done
>> fi
>> fi
>> fi
>>
>> return ${ret}
>> }
>>
>> monitor() {
>> local vg=$1
>> local lv=$2
>> local rc
>> local output
>>
>> status ${vg} ${lv}
>> rc=$?
>>
>> if [ ${rc} != $OCF_SUCCESS ] && ! ocf_is_probe ; then
>> ocf_log warn "Volume group [${OCF_RESKEY_vg_name}] does not exist or is not accessible."
>> ocf_log warn "For clustered volume groups verify that clvmd is started and working."
>> fi
>>
>> return ${rc}
>> }
>>
>> start() {
>> local vg=$1
>> local lv=$2
>> local rc
>> local scanned
>> local flags
>> local output
>> local tries
>>
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_SUCCESS ] ; then
>> return ${rc}
>> fi
>>
>> # We need to run vgscan in all cases, even if activation_mode = "none"
>> scanned=0
>> while : ; do
>> while : ; do
>> flags=$( timeout -s KILL 15 vgs --config 'log{command_names=0 prefix=""}' --nolocking -o attr --noheadings ${vg} 2>/dev/null )
>> if [ $? -ne 124 ] ; then
>> break
>> fi
>> done
>> if [ -z "${flags}" ] ; then
>> if [ ${scanned} -eq 1 ] ; then
>> return $OCF_ERR_GENERIC
>> else
>> ocf_run vgscan
>> scanned=1
>> continue
>> fi
>> fi
>> ocf_run timeout -s KILL 60 vgck ${vg}
>> rc=$?
>> if [ ${rc} -ne 0 ]; then
>> if [ ${rc} -ne 124 ] ; then
>> ocf_log err "Volume group [${vg}] contains errors."
>> return $OCF_ERR_GENERIC
>> fi
>> fi
>>
>> # VG is known to system and is clean, break
>> break
>> done
>>
>> if [[ $flags =~ .....c ]]; then
>> clustered=1
>> else
>> clustered=0
>> fi
>>
>> echo "clustered=${clustered}" > "${STATEFILE}"
>>
>> check_activation_mode
>> rc=$?
>> if [ ${rc} != $OCF_SUCCESS ] ; then
>> return ${rc}
>> fi
>>
>> if [ -z "${lv}" ] ; then
>> # VG mode
>> if [ "${OCF_RESKEY_activation_mode}" != "none" ] ; then
>> ocf_log info "Activating volume group [${vg}] (-a${activation_modifier}y)"
>>
>> : $(( tries = 0 ))
>> while : ; do
>> ocf_run timeout -s KILL 40 vgchange "-a${activation_modifier}y" ${vg}
>> rc=$?
>> if [ ${rc} -eq 124 ] ; then
>> continue
>> fi
>> if [ ${rc} -ne 0 ] ; then
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_SUCCESS ] ; then
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> rc=$OCF_ERR_GENERIC
>> break
>> fi
>> sleep 1
>> else
>> status ${vg} ${lv}
>> rc=$?
>> break
>> fi
>> done
>> else
>> ha_pseudo_resource ${ha_pseudo_resource_name} start
>> status ${vg} ${lv}
>> rc=$?
>> fi
>>
>> if [ ${rc} -ne $OCF_SUCCESS ] ; then
>> ocf_log err "Volume group [${vg}] did not activate correctly"
>> output=$( timeout -s KILL 15 lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg} 2>/dev/null )
>> if [ ${rc} -ne 124 ] ; then
>> ocf_log info "${output}"
>> fi
>> fi
>> else
>> ocf_log info "Activating volume [${vg}/${lv}] (-a${activation_modifier}y)"
>> : $(( tries = 0 ))
>> while : ; do
>> ocf_run timeout -s KILL 25 lvchange "-a${activation_modifier}y" ${vg}/${lv}
>> rc=$?
>> if [ ${rc} -eq 124 ] ; then
>> continue
>> fi
>> if [ $rc -ne 0 ] ; then
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_SUCCESS ] ; then
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> rc=$OCF_ERR_GENERIC
>> break
>> fi
>> sleep 1
>> else
>> status ${vg} ${lv}
>> rc=$?
>> break
>> fi
>> done
>>
>> if [ ${rc} -ne $OCF_SUCCESS ] ; then
>> ocf_log err "Volume [${vg}/${lv}] did not activate correctly"
>> output=$( timeout -s KILL 15 lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
>> if [ ${rc} -ne 124 ] ; then
>> ocf_log info "${output}"
>> fi
>> fi
>> fi
>>
>> return ${rc}
>> }
>>
>> stop() {
>> local vg=$1
>> local lv=$2
>> local rc
>> local output
>>
>> # Quick hack to introduce needed features. Global variable...
>> if [ "${OCF_RESKEY_activation_mode}" != "none" ] || \
>> ocf_is_true ${OCF_RESKEY_force_stop} || \
>> ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
>> need_real_status=1
>> fi
>>
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
>> return $OCF_SUCCESS
>> elif [ ${rc} = $OCF_SUCCESS ] && [ "${OCF_RESKEY_activation_mode}" = "none" ] && ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
>> ocf_log err "Not-managed volume group [${vg}] has active volumes on stop and verify_stopped_on_stop is true:"
>> ocf_run timeout -s KILL 15 lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}
>> return $OCF_ERR_GENERIC
>> fi
>>
>> if [ -z "${lv}" ] ; then
>> # VG mode
>> if [ ${need_real_status} = 1 ] ; then
>> if [ "${OCF_RESKEY_activation_mode}" != "none" ] || ocf_is_true ${OCF_RESKEY_force_stop} ; then
>> ocf_log info "Deactivating volume group [${vg}] (-a${activation_modifier}n)"
>> : $(( tries = 0 ))
>> while : ; do
>> ocf_run timeout -s KILL 40 vgchange "-a${activation_modifier}n" ${vg}
>> rc=$?
>> if [ ${rc} -ne 0 ] ; then
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
>> rc=$OCF_SUCCESS
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> rc=$OCF_ERR_GENERIC
>> break
>> fi
>> sleep 1
>> else
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
>> rc=$OCF_SUCCESS
>> elif [ ${rc} -eq $OCF_SUCCESS ] ; then
>> rc=$OCF_ERR_GENERIC
>> fi
>> break
>> fi
>> done
>> if [ "${OCF_RESKEY_activation_mode}" = "none" ] ; then
>> ha_pseudo_resource ${ha_pseudo_resource_name} stop
>> fi
>> else
>> if [ "${OCF_RESKEY_activation_mode}" = "none" ] ; then
>> ha_pseudo_resource ${ha_pseudo_resource_name} stop
>> fi
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
>> rc=$OCF_SUCCESS
>> elif [ ${rc} -eq $OCF_SUCCESS ] ; then
>> rc=$OCF_ERR_GENERIC
>> fi
>> fi
>>
>> if [ ${rc} -ne $OCF_SUCCESS ] ; then
>> ocf_log err "Volume group [${vg}] did not stop correctly"
>> output=$( lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
>> ocf_log info "${output}"
>> fi
>> fi
>> else
>> ocf_log info "Deactivating volume [${vg}/${lv}] (-a${activation_modifier}n)"
>> : $(( tries = 0 ))
>> while : ; do
>> ocf_run timeout -s KILL 25 lvchange "-a${activation_modifier}n" ${vg}/${lv}
>> rc=$?
>> if [ ${rc} -ne 0 ] ; then
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
>> rc=$OCF_SUCCESS
>> break
>> fi
>> : $(( tries++ ))
>> if [ ${tries} -gt 10 ] ; then
>> rc=$OCF_ERR_GENERIC
>> break
>> fi
>> sleep 1
>> else
>> status ${vg} ${lv}
>> rc=$?
>> if [ ${rc} -eq $OCF_NOT_RUNNING ] ; then
>> rc=$OCF_SUCCESS
>> elif [ ${rc} -eq $OCF_SUCCESS ] ; then
>> rc=$OCF_ERR_GENERIC
>> fi
>> break
>> fi
>> done
>>
>> if [ ${rc} -ne $OCF_SUCCESS ] ; then
>> ocf_log err "Volume [${vg}/${lv}] did not stop correctly"
>> output=$( lvs --config 'log{command_names=0 prefix=""}' --nolocking --noheadings ${vg}/${lv} 2>/dev/null )
>> ocf_log info "${output}"
>> fi
>> fi
>>
>> return ${rc}
>> }
>>
>> validate_all() {
>> check_binary vgs
>> check_binary vgscan
>> check_binary vgck
>> check_binary vgchange
>> check_binary lvchange
>> check_binary lvs
>>
>> if [ -z "${OCF_RESKEY_vg_name}" ] ; then
>> exit $OCF_ERR_CONFIGURED
>> fi
>> case ${OCF_RESKEY_activation_mode} in
>> exclusive)
>> if [ -n "${OCF_RESKEY_CRM_meta_clone}" ] ; then
>> ocf_log err "Rejecting to operate in exclusive activation mode for clone resource."
>> return $OCF_ERR_CONFIGURED
>> fi
>> ;;
>> none)
>> if [ -n "${OCF_RESKEY_lv_name}" ] ; then
>> ocf_log err "activation_mode=none cannot be used for volumes, only for volume groups."
>> return $OCF_ERR_CONFIGURED
>> fi
>> ;;
>> auto|local)
>> ;;
>> *)
>> ocf_log err "Incorrect parameter activation_mode='${OCF_RESKEY_activation_mode}'."
>> return $OCF_ERR_CONFIGURED
>> ;;
>> esac
>>
>> if ocf_is_true ${OCF_RESKEY_force_stop} || ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
>> if [ -n "${OCF_RESKRY_lv_name}" ] ; then
>> ocf_log err "'force_stop' or 'verify_stopped_on_stop' should be specified only for VG-level resources."
>> return $OCF_ERR_CONFIGURED
>> fi
>> if [ "${OCF_RESKEY_activation_mode}" != "none" ] ; then
>> ocf_log warn "'force_stop' and 'verify_stopped_on_stop' don't affect operation if activation_mode != 'none', please fix parameters for this resource."
>> fi
>> if ocf_is_true ${OCF_RESKEY_force_stop} && ocf_is_true ${OCF_RESKEY_verify_stopped_on_stop} ; then
>> ocf_log err "'force_stop' and 'verify_stopped_on_stop' parameters are mutually exclusive."
>> return $OCF_ERR_CONFIGURED
>> fi
>> fi
>> }
>>
>>
>> case $1 in
>> meta-data)
>> meta_data
>> exit $OCF_SUCCESS
>> ;;
>> usage|help)
>> usage
>> exit $OCF_SUCCESS
>> ;;
>> esac
>>
>> # Everything except usage and meta-data must pass the validate test
>> validate_all || exit $?
>>
>>
>> : ${ha_pseudo_resource_name:=LVM2-${OCF_RESOURCE_INSTANCE}}
>> STATEFILE="${HA_RSCTMP}/LVM2-${OCF_RESOURCE_INSTANCE}.state"
>>
>> if [ "$__OCF_ACTION" != "start" ] && [ "$__OCF_ACTION" != "reload" ] && [ -s "${STATEFILE}" ] ; then
>> . "${STATEFILE}"
>> check_activation_mode
>> rc=$?
>> if [ ${rc} != $OCF_SUCCESS ] ; then
>> return ${rc}
>> fi
>> fi
>>
>> # Request high priority (very helpful under high load)
>> chrt -p -r 10 $$
>>
>> # What kind of method was invoked?
>> case "$__OCF_ACTION" in
>> start|reload)
>> start ${OCF_RESKEY_vg_name} ${OCF_RESKEY_lv_name}
>> rc=$?
>> ;;
>> stop)
>> stop ${OCF_RESKEY_vg_name} ${OCF_RESKEY_lv_name}
>> rc=$?
>>
>> if [ ${rc} -eq 0 ] ; then
>> rm -f "${STATEFILE}"
>> fi
>> ;;
>> monitor)
>> monitor ${OCF_RESKEY_vg_name} ${OCF_RESKEY_lv_name}
>> rc=$?
>> ;;
>> validate-all)
>> rc=$OCF_SUCCESS
>> ;;
>> *)
>> usage
>> exit $OCF_ERR_UNIMPLEMENTED
>> ;;
>> esac
>>
>> exit ${rc}
>
>> _______________________________________________________
>> Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
>> Home Page: http://linux-ha.org/
>
> _______________________________________________________
> Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
> Home Page: http://linux-ha.org/

_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/