Mailing List Archive: [nova]Capacity discrepancy between command line and MySQL query

[nova]Capacity discrepancy between command line and MySQL query

Aug 27, 2018, 6:40 AM

Post #1 of 3 (563 views)

Hi,

I tried to migrate a guest to another host but it failed with a
message saying there's not enough capacity on the target host even
though the server should me nearly empty. The guest I'm trying to
move needs 4 cores, 4 GB of memory and 50 GB of disk. Each compute
node should have 20 cores, 128 GB RAM & 260 GB HD space.

When I check it with "openstack host show compute1" I see that there's
plenty of free resources. However, when I check it directly in MariaDB
nova_api or using Placement API calls I see different results i.e. not
enough cores & disk.

Is there a safe way to make the different registries / databases to
match? Can I just overwrite it using the Placement API?

I'm using Pike.

BR,
Risto

PS
I did make a few attempts to resize the guest that now runs on
compute1 but for some reason they failed and by default the resize
tries to restart the resized guest on a different host (compute1).
In the end I was able to do the resize on the same host (compute2).
I was wondering if the resize attempts messed up the compute1 resource
management.

PPS
I wrote a question about this on ask.openstack.org in mid-July but the
message is still in moderation:
https://ask.openstack.org/en/question/115200/

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [nova]Capacity discrepancy between command line and MySQL query [ In reply to ]

jaypipes at gmail

Aug 28, 2018, 7:06 AM

Post #2 of 3 (561 views)

Permalink

On 08/27/2018 09:40 AM, Risto Vaaraniemi wrote:
> Hi,
>
> I tried to migrate a guest to another host but it failed with a
> message saying there's not enough capacity on the target host even
> though the server should me nearly empty. The guest I'm trying to
> move needs 4 cores, 4 GB of memory and 50 GB of disk. Each compute
> node should have 20 cores, 128 GB RAM & 260 GB HD space.
>
> When I check it with "openstack host show compute1" I see that there's
> plenty of free resources. However, when I check it directly in MariaDB
> nova_api or using Placement API calls I see different results i.e. not
> enough cores & disk.
>
> Is there a safe way to make the different registries / databases to
> match? Can I just overwrite it using the Placement API?
>
> I'm using Pike.
>
> BR,
> Risto
>
> PS
> I did make a few attempts to resize the guest that now runs on
> compute1 but for some reason they failed and by default the resize
> tries to restart the resized guest on a different host (compute1).
> In the end I was able to do the resize on the same host (compute2).
> I was wondering if the resize attempts messed up the compute1 resource
> management.

Very likely, yes.

It's tough to say what exact sequence of resize and migrate commands
have caused your inventory and allocation records in placement to become
corrupted.

Have you tried restarting the nova-compute services on both compute
nodes and seeing whether the placement service tries to adjust
allocations upon restart?

Also, please check the logs on the nova-compute workers looking for any
warnings or errors related to communication with placement.

Best,
-jay

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [nova]Capacity discrepancy between command line and MySQL query [ In reply to ]

ristovvtt at gmail

Sep 25, 2018, 6:29 AM

Post #3 of 3 (546 views)

Permalink

On Tue, Aug 28, 2018 at 5:12 PM Jay Pipes <jaypipes@gmail.com> wrote:
>
> On 08/27/2018 09:40 AM, Risto Vaaraniemi wrote:
> > Hi,
> >
> > I tried to migrate a guest to another host but it failed with a
> > message saying there's not enough capacity on the target host even
> > though the server should me nearly empty.
> >
> > I did make a few attempts to resize the guest that now runs on
> > compute1 but for some reason they failed and by default the resize
> > tries to restart the resized guest on a different host (compute1).
> > In the end I was able to do the resize on the same host (compute2).
> > I was wondering if the resize attempts messed up the compute1 resource
> > management.
>
> Very likely, yes.
>
> Have you tried restarting the nova-compute services on both compute
> nodes and seeing whether the placement service tries to adjust
> allocations upon restart?
>
> Also, please check the logs on the nova-compute workers looking for any
> warnings or errors related to communication with placement.

I tried restarting but that didn't help.

In the end, I was able to solve the situation by making a backup image
of the guest that caused the problem and re-creating the guest with the same
flavor. From the placement outputs I could see that it was the the guest
allocation that was wrong for some reason. The host did not have any extra
hanging allocations after that.

Thanks for your time, anyway. :)

BR,
Risto

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack