Mailing List Archive

Hi,

On Thu, Nov 13, 2014 at 01:57:08PM +0100, Heiner Meier wrote:
> Hello,
>
> i need an Cluster with drbd, the active Cluster Member should hold a
> running kvm instance, started via libvirt.
>
> A virtual IP is not needet.
>
> It runs, but from time to Time it doesnt take over correctly when i
> reboot the "master" System, normaly all resources after the machine is
> up again should migrate back to the master System (via location statement).
>
> In the most cases this works, but from time to time drbd failed and the
> ressources stay on the slave Server, after rebooting the master Server
> one time more, all is OK.
>
> What i later still need ist a automatic drbd Split Brain recovery, if
> anyone have a working config for this it should be interesting to see it.
>
> Here is my pacemaker configuration:
>
> node $id="1084777473" master \
> attributes standby="off" maintenance="off"
> node $id="1084777474" slave \
> attributes maintenance="off" standby="off"
> primitive libvirt upstart:libvirt-bin \
> op start timeout="120s" interval="0" \
> op stop timeout="120s" interval="0" \
> op monitor interval="30s" \
> meta target-role="Started"
> primitive vmdata ocf:linbit:drbd \
> params drbd_resource="vmdata" \
> op monitor interval="29s" role="Master" \
> op monitor interval="31s" role="Slave"
> primitive vmdata_fs ocf:heartbeat:Filesystem \
> params device="/dev/drbd0" directory="/vmdata" fstype="ext4" \
> meta target-role="Started"
> ms drbd_master_slave vmdata \
> meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1"
> notify="true"
> location PrimaryNode-libvirt libvirt 200: master
> location PrimaryNode-vmdata_fs vmdata_fs 200: master
> location SecondaryNode-libvirt libvirt 10: slave
> location SecondaryNode-vmdata_fs vmdata_fs 10: slave
> colocation services_colo inf: drbd_master_slave:Master vmdata_fs

This one should be the other way around:

colocation services_colo inf: vmdata_fs drbd_master_slave:Master

> order fs_after_drbd inf: drbd_master_slave:promote vmdata_fs:start
> libvirt:start

And you need one more collocation:

colocation libvirt-with-fs inf: libvirt vmdata_fs

HTH,

Dejan

> property $id="cib-bootstrap-options" \
> dc-version="1.1.10-42f2063" \
> cluster-infrastructure="corosync" \
> stonith-enabled="false" \
> no-quorum-policy="ignore" \
> last-lrm-refresh="1415619869"
>
>
> There must be an Error in this configuration, but i dont know in which
> part.
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

And you need to configure your cluster fencing and you need to be sure
sure to configure drbd to use the pacemaker fencing
http://www.drbd.org/users-guide/s-pacemaker-fencing.html

2014-11-13 14:58 GMT+01:00 Dejan Muhamedagic <dejanmm@fastmail.fm>:
> Hi,
>
> On Thu, Nov 13, 2014 at 01:57:08PM +0100, Heiner Meier wrote:
>> Hello,
>>
>> i need an Cluster with drbd, the active Cluster Member should hold a
>> running kvm instance, started via libvirt.
>>
>> A virtual IP is not needet.
>>
>> It runs, but from time to Time it doesnt take over correctly when i
>> reboot the "master" System, normaly all resources after the machine is
>> up again should migrate back to the master System (via location statement).
>>
>> In the most cases this works, but from time to time drbd failed and the
>> ressources stay on the slave Server, after rebooting the master Server
>> one time more, all is OK.
>>
>> What i later still need ist a automatic drbd Split Brain recovery, if
>> anyone have a working config for this it should be interesting to see it.
>>
>> Here is my pacemaker configuration:
>>
>> node $id="1084777473" master \
>> attributes standby="off" maintenance="off"
>> node $id="1084777474" slave \
>> attributes maintenance="off" standby="off"
>> primitive libvirt upstart:libvirt-bin \
>> op start timeout="120s" interval="0" \
>> op stop timeout="120s" interval="0" \
>> op monitor interval="30s" \
>> meta target-role="Started"
>> primitive vmdata ocf:linbit:drbd \
>> params drbd_resource="vmdata" \
>> op monitor interval="29s" role="Master" \
>> op monitor interval="31s" role="Slave"
>> primitive vmdata_fs ocf:heartbeat:Filesystem \
>> params device="/dev/drbd0" directory="/vmdata" fstype="ext4" \
>> meta target-role="Started"
>> ms drbd_master_slave vmdata \
>> meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1"
>> notify="true"
>> location PrimaryNode-libvirt libvirt 200: master
>> location PrimaryNode-vmdata_fs vmdata_fs 200: master
>> location SecondaryNode-libvirt libvirt 10: slave
>> location SecondaryNode-vmdata_fs vmdata_fs 10: slave
>> colocation services_colo inf: drbd_master_slave:Master vmdata_fs
>
> This one should be the other way around:
>
> colocation services_colo inf: vmdata_fs drbd_master_slave:Master
>
>> order fs_after_drbd inf: drbd_master_slave:promote vmdata_fs:start
>> libvirt:start
>
> And you need one more collocation:
>
> colocation libvirt-with-fs inf: libvirt vmdata_fs
>
> HTH,
>
> Dejan
>
>> property $id="cib-bootstrap-options" \
>> dc-version="1.1.10-42f2063" \
>> cluster-infrastructure="corosync" \
>> stonith-enabled="false" \
>> no-quorum-policy="ignore" \
>> last-lrm-refresh="1415619869"
>>
>>
>> There must be an Error in this configuration, but i dont know in which
>> part.
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org



--
esta es mi vida e me la vivo hasta que dios quiera

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

On 13/11/14 07:57 AM, Heiner Meier wrote:
> stonith-enabled="false" \

This is your problem. DRBD absolutely requires fencing/stonith. Please
configure it in pacemaker, confirm that it works properly, then
configure DRBD to use the crm-fence-peer.sh fence handler (and the
crm-unfence-peer.sh unfence handler), tell DRBD to use the
'resource-and-stonith' fencing policy and things should start working
predictably.

If not, please reply back with log snippets.

--
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Hello,

i now have configured fencing in drbd:

disk {
fencing resource-only;
}
handlers {
fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
after-resync-target "/usr/lib/drbd/crm-unfence-peer.sh";
}

And changed the config to:

node $id="1084777473" master \
attributes standby="off" maintenance="off"
node $id="1084777474" slave \
attributes maintenance="off" standby="off"
primitive libvirt upstart:libvirt-bin \
op start timeout="120s" interval="0" \
op stop timeout="120s" interval="0" \
op monitor interval="30s" \
meta target-role="Started"
primitive vmdata ocf:linbit:drbd \
params drbd_resource="vmdata" \
op monitor interval="29s" role="Master" \
op monitor interval="31s" role="Slave"
primitive vmdata_fs ocf:heartbeat:Filesystem \
params device="/dev/drbd0" directory="/vmdata" fstype="ext4" \
meta target-role="Started"
ms drbd_master_slave vmdata \
meta master-max="1" master-node-max="1" clone-max="2"
clone-node-max="1" notify="true" target-role="Started"
location PrimaryNode-libvirt libvirt 200: master
location PrimaryNode-vmdata_fs vmdata_fs 200: master
location SecondaryNode-libvirt libvirt 10: slave
location SecondaryNode-vmdata_fs vmdata_fs 10: slave
colocation libvirt-with-fs inf: libvirt vmdata_fs
colocation services_colo inf: vmdata_fs drbd_master_slave:Master
order fs_after_drbd inf: drbd_master_slave:promote vmdata_fs:start
libvirt:start
property $id="cib-bootstrap-options" \
dc-version="1.1.10-42f2063" \
cluster-infrastructure="corosync" \
stonith-enabled="false" \
no-quorum-policy="ignore" \
last-lrm-refresh="1415964693"


But now the Cluster wont work anymore, no failover - drbd / libvirt.
Both members stay always in slave state:

When i try to start ressources with crm - no drbd Filesystem will
mounted, but the machine ist now "master" - after a reboot both
stays slave...

Also i cant see the ressources with "crm status" on the shell,
with the old config i can see them both??



_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

you need to configure the fencing in pacemaker

2014-11-14 13:04 GMT+01:00 Heiner Meier <linuxforums@erwo.net>:
> Hello,
>
> i now have configured fencing in drbd:
>
> disk {
> fencing resource-only;
> }
> handlers {
> fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
> after-resync-target "/usr/lib/drbd/crm-unfence-peer.sh";
> }
>
> And changed the config to:
>
> node $id="1084777473" master \
> attributes standby="off" maintenance="off"
> node $id="1084777474" slave \
> attributes maintenance="off" standby="off"
> primitive libvirt upstart:libvirt-bin \
> op start timeout="120s" interval="0" \
> op stop timeout="120s" interval="0" \
> op monitor interval="30s" \
> meta target-role="Started"
> primitive vmdata ocf:linbit:drbd \
> params drbd_resource="vmdata" \
> op monitor interval="29s" role="Master" \
> op monitor interval="31s" role="Slave"
> primitive vmdata_fs ocf:heartbeat:Filesystem \
> params device="/dev/drbd0" directory="/vmdata" fstype="ext4" \
> meta target-role="Started"
> ms drbd_master_slave vmdata \
> meta master-max="1" master-node-max="1" clone-max="2"
> clone-node-max="1" notify="true" target-role="Started"
> location PrimaryNode-libvirt libvirt 200: master
> location PrimaryNode-vmdata_fs vmdata_fs 200: master
> location SecondaryNode-libvirt libvirt 10: slave
> location SecondaryNode-vmdata_fs vmdata_fs 10: slave
> colocation libvirt-with-fs inf: libvirt vmdata_fs
> colocation services_colo inf: vmdata_fs drbd_master_slave:Master
> order fs_after_drbd inf: drbd_master_slave:promote vmdata_fs:start
> libvirt:start
> property $id="cib-bootstrap-options" \
> dc-version="1.1.10-42f2063" \
> cluster-infrastructure="corosync" \
> stonith-enabled="false" \
> no-quorum-policy="ignore" \
> last-lrm-refresh="1415964693"
>
>
> But now the Cluster wont work anymore, no failover - drbd / libvirt.
> Both members stay always in slave state:
>
> When i try to start ressources with crm - no drbd Filesystem will
> mounted, but the machine ist now "master" - after a reboot both
> stays slave...
>
> Also i cant see the ressources with "crm status" on the shell,
> with the old config i can see them both??
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org



--
esta es mi vida e me la vivo hasta que dios quiera

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org