Mailing List Archive

Greetings Toni,
Not sure I'm answering, but just as a quick note you can prioritize NUMA
and CPU-pinning by creating dedicated flavors [1] with something like:

openstack flavor set m1.largenuma --property hw:numa_cpus.0=0,1 --property
hw:numa_mem.0=2048

Usually NUMA cores are reserved or shared only for specific fixed ops, and
I'm not sure what you mean by "throwing off more cores" in case of need,
probably you need to look into something like Heat autoscale?

Cheers,
Fabrizio

[1] https://docs.openstack.org/nova/pike/admin/cpu-topologies.html

On Wed, Jul 4, 2018 at 5:19 PM Toni Mueller <toni.mueller@oeko.net> wrote:

>
> Hi,
>
> I am still trying to figure how to best utilise the small set of
> hardware, and discovered the NUMA configuration mechanism. It allows me
> to configure reserved cores for certain VMs, but it does not seem to
> allow me to say "you can share these cores, but VMs of, say, appropriate
> flavour take precedence and will throw you off these cores in case they
> need more power".
>
> How can I achieve that, dynamically?
>
> TIA!
>
>
> Thanks,
> Toni
>
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>

Hi Tony,

The short answer is that you cannot do that today. Today, each Nova
compute node is either "all in" for NUMA and CPU pinning or it's not.

This means that for resource-constrained environments like "The Edge!",
there are not very good ways to finely divide up a compute node and make
the most efficient use of its resources.

There is no current way to say "On this dual-Xeon compute node, put all
workloads that don't care about dedicated CPUs on this socket and all
workloads that DO care about dedicated CPUs on the other socket.".

That said, we have had lengthy discussions about tracking dedicated
guest CPU resources and dividing up the available logical host
processors into buckets for "shared CPU" and "dedicated CPU" workloads
on the following spec:

https://review.openstack.org/#/c/555081/

It is not going to land in Rocky. However, we should be able to make
good progress towards the goals in that spec in early Stein.

Best,
-jay

On 07/04/2018 11:08 AM, Toni Mueller wrote:
>
> Hi,
>
> I am still trying to figure how to best utilise the small set of
> hardware, and discovered the NUMA configuration mechanism. It allows me
> to configure reserved cores for certain VMs, but it does not seem to
> allow me to say "you can share these cores, but VMs of, say, appropriate
> flavour take precedence and will throw you off these cores in case they
> need more power".
>
> How can I achieve that, dynamically?
>
> TIA!
>
>
> Thanks,
> Toni
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Hi Fabrizio!

thank you for your answer!

On Thu, Jul 05, 2018 at 05:35:00PM +0200, Fabrizio Soppelsa wrote:
> and CPU-pinning by creating dedicated flavors [1] with something like:
>
> openstack flavor set m1.largenuma --property hw:numa_cpus.0=0,1 --property
> hw:numa_mem.0=2048

I had suggested this already, but this approach was rejected by the
manager because they want to overbook more. But I found something with
"cpu_weight", which seems to accomplish a similar task without having to
pin the CPU, and thus make it unusable for everyone else.


Kind regards,
Toni


_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Hi Jay,

On Fri, Jul 06, 2018 at 12:46:04PM -0400, Jay Pipes wrote:
> There is no current way to say "On this dual-Xeon compute node, put all
> workloads that don't care about dedicated CPUs on this socket and all
> workloads that DO care about dedicated CPUs on the other socket.".

it turned out that this is not what I should want to say. What I should
say instead is:

"Run all VMs on all cores, but if certain VMs suddenly spike, give them
all they ask for at the expense of everyone else, and also avoid moving
them around between cores, if possible."

The idea is that these high priority VMs are (probably) idle most of the
time, but at other times need high performance. It was thus deemed to be
a huge waste to reserve cores for them.

> https://review.openstack.org/#/c/555081/

Thank you for the pointer!


Thanks,
Toni


_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

On 07/16/2018 10:30 AM, Toni Mueller wrote:
>
> Hi Jay,
>
> On Fri, Jul 06, 2018 at 12:46:04PM -0400, Jay Pipes wrote:
>> There is no current way to say "On this dual-Xeon compute node, put all
>> workloads that don't care about dedicated CPUs on this socket and all
>> workloads that DO care about dedicated CPUs on the other socket.".
>
> it turned out that this is not what I should want to say. What I should
> say instead is:
>
> "Run all VMs on all cores, but if certain VMs suddenly spike, give them
> all they ask for at the expense of everyone else, and also avoid moving
> them around between cores, if possible."
>
> The idea is that these high priority VMs are (probably) idle most of the
> time, but at other times need high performance. It was thus deemed to be
> a huge waste to reserve cores for them.

You're looking for something like VMWare DRS, then:

https://www.vmware.com/products/vsphere/drs-dpm.html

This isn't something Nova is looking to implement.

Best,
-jay


_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack