Mailing List Archive

Antw: RA Spec: A new API for utilization probe
Hi!

I'd still like to have "static" utilization as well for the following reasons:
You define a pool of (static) resources (in the sense of utilization capacities) for the nodes. These should be the initial values.
You define (static) resources (in the sense of utilization demands) for your cluster resources (here meaning the services the cluster manages).

The advantage of static values is this: If a system requires more and more resources over time (hopefully up to some limit), the initial placement of resources on the nodes will meet their maximum requirement (if configured correctly).

In the dynamic case, a resource might be placed on a node with insufficient capacities, causing unnecessary migrations if it turn out that the nodes capacities are running low.
I also can imagine that dynamic capacities (avoiding the word "utilization" here) can cause a kind of deadlock; did you consider that?

Regards,
Ulrich Windl

>>> "Xin Wei Hu" <xwhu@novell.com> schrieb am 15.07.2011 um 09:10 in Nachricht
<4E20748C020000C700023D0C@novprvlin0050.provo.novell.com>:
> Hi all,
>
> This is actually about the possible next step about utilization based
> resource allocation in pacemaker. As we know, current pacemaker has the
> ability to do resource allocation based on the utilization of nodes and
> resources. But all these have to be set prior running, and belong to the
> configuration of CIB.
>
> It should be a nature evolution for pacemaker to support dynamic
> utilization based resource reallocation. For that, I'd proposal
> following changes for you to review.
>
> - pacemaker to store the dynamic utilization value in the status
> section of CIB. A changing to the value in configuration triggers a new
> transaction, which is not optimal when trying to update the utilization
> of a pile of resources. Also, by storing it in status, we can leverage
> the attrd_updater with --dempan.
>
> - A new API for RA, for pacemaker to probe the utilization of a
> resource instance. I'd suggest a obvious name, like probe_utilization.
> This new API requires no parameter, and always return $OCF_SUCCESS. RA
> takes the responsibility to update the utilization value when called.
>
>
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
>




_______________________________________________
ha-wg-technical mailing list
ha-wg-technical@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
Re: Antw: RA Spec: A new API for utilization probe [ In reply to ]
在 2011-07-15五的 10:37 +0200,Ulrich Windl写道:
> Hi!
>
> I'd still like to have "static" utilization as well for the following reasons:
> You define a pool of (static) resources (in the sense of utilization capacities) for the nodes. These should be the initial values.
> You define (static) resources (in the sense of utilization demands) for your cluster resources (here meaning the services the cluster manages).
>
> The advantage of static values is this: If a system requires more and more resources over time (hopefully up to some limit), the initial placement of resources on the nodes will meet their maximum requirement (if configured correctly).

Agree. But then, adding the ability to handle dynamic thing doesn't mean to obsolete the static one.
It doesn't hurt to not update the dynamic utilization, so that the
static assigned value can always be used.

> In the dynamic case, a resource might be placed on a node with insufficient capacities, causing unnecessary migrations if it turn out that the nodes capacities are running low.

I think that's why I proposed to store it in the status section. So that
we can probe all resource instances before starting a new transaction to
actually migrate anything.

> I also can imagine that dynamic capacities (avoiding the word "utilization" here) can cause a kind of deadlock; did you consider that?

Not I'm aware of, yet. ;)

> Regards,
> Ulrich Windl
>
> >>> "Xin Wei Hu" <xwhu@novell.com> schrieb am 15.07.2011 um 09:10 in Nachricht
> <4E20748C020000C700023D0C@novprvlin0050.provo.novell.com>:
> > Hi all,
> >
> > This is actually about the possible next step about utilization based
> > resource allocation in pacemaker. As we know, current pacemaker has the
> > ability to do resource allocation based on the utilization of nodes and
> > resources. But all these have to be set prior running, and belong to the
> > configuration of CIB.
> >
> > It should be a nature evolution for pacemaker to support dynamic
> > utilization based resource reallocation. For that, I'd proposal
> > following changes for you to review.
> >
> > - pacemaker to store the dynamic utilization value in the status
> > section of CIB. A changing to the value in configuration triggers a new
> > transaction, which is not optimal when trying to update the utilization
> > of a pile of resources. Also, by storing it in status, we can leverage
> > the attrd_updater with --dempan.
> >
> > - A new API for RA, for pacemaker to probe the utilization of a
> > resource instance. I'd suggest a obvious name, like probe_utilization.
> > This new API requires no parameter, and always return $OCF_SUCCESS. RA
> > takes the responsibility to update the utilization value when called.
> >
> >
> > _______________________________________________
> > ha-wg-technical mailing list
> > ha-wg-technical@lists.linux-foundation.org
> > https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
> >
>
>
>
>


_______________________________________________
ha-wg-technical mailing list
ha-wg-technical@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
Re: Antw: RA Spec: A new API for utilization probe [ In reply to ]
>>> "Xin Wei Hu" <xwhu@novell.com> schrieb am 15.07.2011 um 11:09 in Nachricht
<4E209063020000C700023D22@novprvlin0050.provo.novell.com>:
> 在 2011-07-15五的 10:37 +0200,Ulrich Windl写道:
[...]
> > I also can imagine that dynamic capacities (avoiding the word
"utilization"
> here) can cause a kind of deadlock; did you consider that?
>
> Not I'm aware of, yet. ;)

Assume you have two nodes with a little capacity remaining, and each node
starts a resource. Both resources require more and more "utilization" until the
node's capacity is exceeded. Then none of the resources can be migrated
elsewhere. If you required the proper static amount of "utilization" at the
start, the resources would not start at all if insufficient capacity exists.

Ulrich


_______________________________________________
ha-wg-technical mailing list
ha-wg-technical@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
Re: Antw: RA Spec: A new API for utilization probe [ In reply to ]
Hi,

On 07/15/2011 02:49 PM, Ulrich Windl wrote:
>>>> "Xin Wei Hu"<xwhu@novell.com> schrieb am 15.07.2011 um 11:09 in Nachricht
> <4E209063020000C700023D22@novprvlin0050.provo.novell.com>:
>> 在 2011-07-15五的 10:37 +0200,Ulrich Windl写道:
> [...]
>>> I also can imagine that dynamic capacities (avoiding the word
> "utilization"
>> here) can cause a kind of deadlock; did you consider that?
>>
>> Not I'm aware of, yet. ;)
> Assume you have two nodes with a little capacity remaining, and each node
> starts a resource. Both resources require more and more "utilization" until the
> node's capacity is exceeded. Then none of the resources can be migrated
> elsewhere.
There are two different views on this:
- even with dynamic "utilization" you still start from a fixed value,
and that value, if not meeting the requirements based on existing
capacity, would lead to the resource not starting if insufficient
capacity exists.
- dynamic "utilization" does not mean start from x and always go to x++.
Percentages could go higher than default value and also below this value
(think assigning a resource agent managing an {insert resource here} 40%
of RAM, then during business hours that value would increase to 65%
dynamically based on utilization, then during nights and weekends actual
utilization would go to 30%).

A more pragmatic approach would be:
- don't oversubscribe resources (your second argument is an example of that)
- maintain a global score made of "available resource for utilization"
(whether (#CPU's x frequency) x nodes, RAM x nodes, etc.) and "currently
used utilization" (all running resources' cumulated utilization), thus
having a local check (can the resource start on this node) and a global
check (are there any resources left anywhere else in the allowed
"domain") that can be used to determine whether a resource should be
allowed to start.

My 2 cents.
> If you required the proper static amount of "utilization" at the
> start, the resources would not start at all if insufficient capacity exists.
>
> Ulrich
>
>
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
Re: Antw: RA Spec: A new API for utilization probe [ In reply to ]
在 2011-07-15五的 13:49 +0200,Ulrich Windl写道:
> >>> "Xin Wei Hu" <xwhu@novell.com> schrieb am 15.07.2011 um 11:09 in Nachricht
> <4E209063020000C700023D22@novprvlin0050.provo.novell.com>:
> > 在 2011-07-15五的 10:37 +0200,Ulrich Windl写道:
> [...]
> > > I also can imagine that dynamic capacities (avoiding the word
> "utilization"
> > here) can cause a kind of deadlock; did you consider that?
> >
> > Not I'm aware of, yet. ;)
>
> Assume you have two nodes with a little capacity remaining, and each node
> starts a resource. Both resources require more and more "utilization" until the
> node's capacity is exceeded. Then none of the resources can be migrated
> elsewhere. If you required the proper static amount of "utilization" at the
> start, the resources would not start at all if insufficient capacity exists.

I actually think this make prefect sense for a dynamic utilization.
If the resources' utilization exceed the capacity of the nodes,
pacemaker then can choose to stop some, depends on the priorities of the
resources we already have, to free capacities for others.

Of course, this also requires we to do the whole things more accurate ;)

> Ulrich
>
>


_______________________________________________
ha-wg-technical mailing list
ha-wg-technical@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
Re: Antw: RA Spec: A new API for utilization probe [ In reply to ]
On 2011-07-15T13:49:15, Ulrich Windl <Ulrich.Windl@rz.uni-regensburg.de> wrote:

> Assume you have two nodes with a little capacity remaining, and each
> node starts a resource. Both resources require more and more
> "utilization" until the node's capacity is exceeded. Then none of the
> resources can be migrated elsewhere.

What is, in your opinion, the desired solution here?

> If you required the proper static amount of "utilization" at the
> start, the resources would not start at all if insufficient capacity
> exists.

I can't follow this.


Regards,
Lars

--
Architect Storage/HA, OPS Engineering, Novell, Inc.
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical