Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Linux-HA>
  3. Pacemaker
Avoid monitoring of resources on nodes
daniel.dehennin at baby-gnu
Nov 25, 2014, 1:06 AM
Post #1 of 12 (4110 views)
Permalink
Hello,

I have a 4 nodes cluster and some resources are only installed on 2 of
them.

I set cluster asymmetry and infinity location:

primitive Mysqld upstart:mysql \
op monitor interval="60"
primitive OpenNebula-Sunstone-Sysv lsb:opennebula-sunstone \
op monitor interval="60"
primitive OpenNebula-Sysv lsb:opennebula \
op monitor interval="60"
group OpenNebula Mysqld OpenNebula-Sysv OpenNebula-Sunstone-Sysv \
meta target-role="Started"
location OpenNebula-runs-on-Frontend OpenNebula inf: one-frontend
property $id="cib-bootstrap-options" \
dc-version="1.1.10-42f2063" \
cluster-infrastructure="corosync" \
symmetric-cluster="false" \
stonith-enabled="true" \
stonith-timeout="30" \
last-lrm-refresh="1416817941" \
no-quorum-policy="stop" \
stop-all-resources="off"

But I have a lot of failing monitoring on other nodes of these resources
because they are not installed on them.

Is there a way to completely exclude the resources from nodes, even the
monitoring?

Regards.

Ubuntu Trusty Tahr (amd64):
- corosync 2.3.3-1ubuntu1
- pacemaker 1.1.10+git20130802-1ubuntu2.1

--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF

Attached Files:

Re: Avoid monitoring of resources on nodes [ In reply to ]
daniel.dehennin at baby-gnu
Nov 25, 2014, 7:54 AM
Post #2 of 12 (4090 views)
Permalink
Daniel Dehennin <daniel.dehennin@baby-gnu.org> writes:

> Hello,

Hello,

> I have a 4 nodes cluster and some resources are only installed on 2 of
> them.
>
> I set cluster asymmetry and infinity location:
>
> primitive Mysqld upstart:mysql \
> op monitor interval="60"
> primitive OpenNebula-Sunstone-Sysv lsb:opennebula-sunstone \
> op monitor interval="60"
> primitive OpenNebula-Sysv lsb:opennebula \
> op monitor interval="60"
> group OpenNebula Mysqld OpenNebula-Sysv OpenNebula-Sunstone-Sysv \
> meta target-role="Started"
> location OpenNebula-runs-on-Frontend OpenNebula inf: one-frontend
> property $id="cib-bootstrap-options" \
> dc-version="1.1.10-42f2063" \
> cluster-infrastructure="corosync" \
> symmetric-cluster="false" \
> stonith-enabled="true" \
> stonith-timeout="30" \
> last-lrm-refresh="1416817941" \
> no-quorum-policy="stop" \
> stop-all-resources="off"
>
> But I have a lot of failing monitoring on other nodes of these resources
> because they are not installed on them.
>
> Is there a way to completely exclude the resources from nodes, even the
> monitoring?

This cause troubles on my setup, as resources fails, my nodes are all
fenced.

Any hints?

Regards.
--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF

Attached Files:

Re: Avoid monitoring of resources on nodes [ In reply to ]
dvossel at redhat
Nov 25, 2014, 12:36 PM
Post #3 of 12 (4082 views)
Permalink
----- Original Message -----
> Daniel Dehennin <daniel.dehennin@baby-gnu.org> writes:
>
> > Hello,
>
> Hello,
>
> > I have a 4 nodes cluster and some resources are only installed on 2 of
> > them.
> >
> > I set cluster asymmetry and infinity location:
> >
> > primitive Mysqld upstart:mysql \
> > op monitor interval="60"
> > primitive OpenNebula-Sunstone-Sysv lsb:opennebula-sunstone \
> > op monitor interval="60"
> > primitive OpenNebula-Sysv lsb:opennebula \
> > op monitor interval="60"
> > group OpenNebula Mysqld OpenNebula-Sysv OpenNebula-Sunstone-Sysv \
> > meta target-role="Started"
> > location OpenNebula-runs-on-Frontend OpenNebula inf: one-frontend
> > property $id="cib-bootstrap-options" \
> > dc-version="1.1.10-42f2063" \
> > cluster-infrastructure="corosync" \
> > symmetric-cluster="false" \
> > stonith-enabled="true" \
> > stonith-timeout="30" \
> > last-lrm-refresh="1416817941" \
> > no-quorum-policy="stop" \
> > stop-all-resources="off"
> >
> > But I have a lot of failing monitoring on other nodes of these resources
> > because they are not installed on them.
> >
> > Is there a way to completely exclude the resources from nodes, even the
> > monitoring?

actually, this is possible now. I am unaware of any configuration tools (pcs or
crmsh) that support this feature yet though. You might have to edit the cib xml
manually.

There's a new 'resource-discovery' option you can set on a location constraint
that help prevent resources from ever being started or monitored on a node.

Example: never start or monitor the resource FAKE1 on 18node2.

<rsc_location id="location-FAKE1-18node2" node="18node2" resource-discovery="never" rsc="FAKE1" score="-INFINITY"/>

There are more examples in this regression test.
https://github.com/ClusterLabs/pacemaker/blob/master/pengine/test10/resource-discovery.xml#L99

-- Vossel


> This cause troubles on my setup, as resources fails, my nodes are all
> fenced.
>
> Any hints?
>
> Regards.
> --
> Daniel Dehennin
> Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
> Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>

_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: Avoid monitoring of resources on nodes [ In reply to ]
daniel.dehennin at baby-gnu
Nov 25, 2014, 2:27 PM
Post #4 of 12 (4084 views)
Permalink
David Vossel <dvossel@redhat.com> writes:

> actually, this is possible now. I am unaware of any configuration tools (pcs or
> crmsh) that support this feature yet though. You might have to edit the cib xml
> manually.
>
> There's a new 'resource-discovery' option you can set on a location constraint
> that help prevent resources from ever being started or monitored on a node.
>
> Example: never start or monitor the resource FAKE1 on 18node2.
>
> <rsc_location id="location-FAKE1-18node2" node="18node2" resource-discovery="never" rsc="FAKE1" score="-INFINITY"/>
>
> There are more examples in this regression test.
> https://github.com/ClusterLabs/pacemaker/blob/master/pengine/test10/resource-discovery.xml#L99

Thanks a lot.

I'll try find how to make the change directly in XML.

Regards.
--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF

Attached Files:

Re: Avoid monitoring of resources on nodes [ In reply to ]
daniel.dehennin at baby-gnu
Nov 25, 2014, 2:46 PM
Post #5 of 12 (4088 views)
Permalink
Daniel Dehennin <daniel.dehennin@baby-gnu.org> writes:

>> There's a new 'resource-discovery' option you can set on a location constraint
>> that help prevent resources from ever being started or monitored on a node.
>>
>> Example: never start or monitor the resource FAKE1 on 18node2.
>>
>> <rsc_location id="location-FAKE1-18node2" node="18node2" resource-discovery="never" rsc="FAKE1" score="-INFINITY"/>
>>
>> There are more examples in this regression test.
>> https://github.com/ClusterLabs/pacemaker/blob/master/pengine/test10/resource-discovery.xml#L99
>
> Thanks a lot.
>
> I'll try find how to make the change directly in XML.

Ok, looking at git history this feature seems only available on master
branch and not yet released.

Thanks.
--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF

Attached Files:

Re: Avoid monitoring of resources on nodes [ In reply to ]
bubble at hoster-ok
Nov 25, 2014, 9:52 PM
Post #6 of 12 (4075 views)
Permalink
25.11.2014 23:36, David Vossel wrote:
>
>
> ----- Original Message -----
>> Daniel Dehennin <daniel.dehennin@baby-gnu.org> writes:
>>
>>> Hello,
>>
>> Hello,
>>
>>> I have a 4 nodes cluster and some resources are only installed on 2 of
>>> them.
>>>
>>> I set cluster asymmetry and infinity location:
>>>
>>> primitive Mysqld upstart:mysql \
>>> op monitor interval="60"
>>> primitive OpenNebula-Sunstone-Sysv lsb:opennebula-sunstone \
>>> op monitor interval="60"
>>> primitive OpenNebula-Sysv lsb:opennebula \
>>> op monitor interval="60"
>>> group OpenNebula Mysqld OpenNebula-Sysv OpenNebula-Sunstone-Sysv \
>>> meta target-role="Started"
>>> location OpenNebula-runs-on-Frontend OpenNebula inf: one-frontend
>>> property $id="cib-bootstrap-options" \
>>> dc-version="1.1.10-42f2063" \
>>> cluster-infrastructure="corosync" \
>>> symmetric-cluster="false" \
>>> stonith-enabled="true" \
>>> stonith-timeout="30" \
>>> last-lrm-refresh="1416817941" \
>>> no-quorum-policy="stop" \
>>> stop-all-resources="off"
>>>
>>> But I have a lot of failing monitoring on other nodes of these resources
>>> because they are not installed on them.
>>>
>>> Is there a way to completely exclude the resources from nodes, even the
>>> monitoring?
>
> actually, this is possible now. I am unaware of any configuration tools (pcs or
> crmsh) that support this feature yet though. You might have to edit the cib xml
> manually.
>
> There's a new 'resource-discovery' option you can set on a location constraint
> that help prevent resources from ever being started or monitored on a node.

crmsh git master supports that.

One note is that pacemaker validation schema should be set to
'pacemaker-next'.

>
> Example: never start or monitor the resource FAKE1 on 18node2.
>
> <rsc_location id="location-FAKE1-18node2" node="18node2" resource-discovery="never" rsc="FAKE1" score="-INFINITY"/>
>
> There are more examples in this regression test.
> https://github.com/ClusterLabs/pacemaker/blob/master/pengine/test10/resource-discovery.xml#L99
>
> -- Vossel
>
>
>> This cause troubles on my setup, as resources fails, my nodes are all
>> fenced.
>>
>> Any hints?
>>
>> Regards.
>> --
>> Daniel Dehennin
>> Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
>> Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF
>>
>> _______________________________________________
>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>


_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: Avoid monitoring of resources on nodes [ In reply to ]
daniel.dehennin at baby-gnu
Nov 26, 2014, 3:21 AM
Post #7 of 12 (4078 views)
Permalink
Daniel Dehennin <daniel.dehennin@baby-gnu.org> writes:

>> I'll try find how to make the change directly in XML.
>
> Ok, looking at git history this feature seems only available on master
> branch and not yet released.

I do not have that feature on my pacemaker version.

Does it sounds normal, I have:

- asymmetrical Opt-in cluster[1]

- a group of resources with INFINITY location on a specific node

And the nodes excluded are fenced because of many monitor errors about
this resource.

Regards.

Footnotes:
[1] http://clusterlabs.org/doc/en-US/Pacemaker/1.1-plugin/html/Pacemaker_Explained/_asymmetrical_opt_in_clusters.html

--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF

Attached Files:

Re: Avoid monitoring of resources on nodes [ In reply to ]
bubble at hoster-ok
Nov 26, 2014, 4:04 AM
Post #8 of 12 (4077 views)
Permalink
26.11.2014 14:21, Daniel Dehennin wrote:
> Daniel Dehennin <daniel.dehennin@baby-gnu.org> writes:
>
>>> I'll try find how to make the change directly in XML.
>>
>> Ok, looking at git history this feature seems only available on master
>> branch and not yet released.
>
> I do not have that feature on my pacemaker version.
>
> Does it sounds normal, I have:
>
> - asymmetrical Opt-in cluster[1]
>
> - a group of resources with INFINITY location on a specific node
>
> And the nodes excluded are fenced because of many monitor errors about
> this resource.

Nodes may be fenced because of resource _only_ if resource fails to
stop. I can only guess what exactly happens:
* cluster probes all resource on all nodes (to prevent that you need
feature mentioned by David)
* some of resource probes return something except "not running"
* cluster tries to stop that resources
* stop fails
* node is fenced

You need to locate what exactly resource returns error on probe and fix
that agent (actually you do not use OCF agents but rather upstart jobs
and LSB scripts).

Above is for the case if all nodes have mysql job and both scripts
installed.

If pacemaker decides to fence because one of them is missing - that
should be a bug.

>
> Regards.
>
> Footnotes:
> [1] http://clusterlabs.org/doc/en-US/Pacemaker/1.1-plugin/html/Pacemaker_Explained/_asymmetrical_opt_in_clusters.html
>
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>


_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: Avoid monitoring of resources on nodes [ In reply to ]
andrew at beekhof
Dec 3, 2014, 7:07 PM
Post #9 of 12 (4023 views)
Permalink
What version of pacemaker is this?
Some very old versions wanted the agent to be installed on all nodes.

> On 26 Nov 2014, at 10:21 pm, Daniel Dehennin <daniel.dehennin@baby-gnu.org> wrote:
>
> Daniel Dehennin <daniel.dehennin@baby-gnu.org> writes:
>
>>> I'll try find how to make the change directly in XML.
>>
>> Ok, looking at git history this feature seems only available on master
>> branch and not yet released.
>
> I do not have that feature on my pacemaker version.
>
> Does it sounds normal, I have:
>
> - asymmetrical Opt-in cluster[1]
>
> - a group of resources with INFINITY location on a specific node
>
> And the nodes excluded are fenced because of many monitor errors about
> this resource.
>
> Regards.
>
> Footnotes:
> [1] http://clusterlabs.org/doc/en-US/Pacemaker/1.1-plugin/html/Pacemaker_Explained/_asymmetrical_opt_in_clusters.html
>
> --
> Daniel Dehennin
> Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
> Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org


_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: Avoid monitoring of resources on nodes [ In reply to ]
daniel.dehennin at baby-gnu
Dec 4, 2014, 12:52 AM
Post #10 of 12 (4023 views)
Permalink
Andrew Beekhof <andrew@beekhof.net> writes:

> What version of pacemaker is this?
> Some very old versions wanted the agent to be installed on all nodes.

It's 1.1.10+git20130802-1ubuntu2.1 on Trusty Tahr.

Regards.
--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF

Attached Files:

Re: Avoid monitoring of resources on nodes [ In reply to ]
andrew at beekhof
Jan 5, 2015, 10:18 PM
Post #11 of 12 (3868 views)
Permalink
> On 4 Dec 2014, at 7:52 pm, Daniel Dehennin <daniel.dehennin@baby-gnu.org> wrote:
>
> Andrew Beekhof <andrew@beekhof.net> writes:
>
>> What version of pacemaker is this?
>> Some very old versions wanted the agent to be installed on all nodes.
>
> It's 1.1.10+git20130802-1ubuntu2.1 on Trusty Tahr.

I'm reasonably sure you need:

+ Gao,Yan (1 year, 5 months ago) 767213e: Feature: pengine: Distinguish between the agent being missing and something the agent needs being missing

which first appeared in 1.1.11-rc1
_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
Re: Avoid monitoring of resources on nodes [ In reply to ]
daniel.dehennin at baby-gnu
Jan 6, 2015, 2:18 AM
Post #12 of 12 (3871 views)
Permalink
Andrew Beekhof <andrew@beekhof.net> writes:

> I'm reasonably sure you need:
>
> + Gao,Yan (1 year, 5 months ago) 767213e: Feature: pengine: Distinguish between the agent being missing and something the agent needs being missing
>
> which first appeared in 1.1.11-rc1

Thanks.

I changed my setup, only the bare metal nodes use GFS2.

I added an NFS resource with a dedicated IP resource to be used by the VMs.

Now my cluster is symmetrical.

Regards.
--
Daniel Dehennin
Récupérer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF

Attached Files:

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal