I created a nova bug [1] to track a request that came up in the upgrades
SIG room at the PTG today [2] and would like to see if there is any
feedback from other operators/developers that weren't part of the
discussion.
The basic problem is that failing to archive/purge deleted records* from
the database can make upgrades much slower during schema migrations.
Anecdotes from the room mentioned that it can be literally impossible to
complete upgrades for keystone and heat in certain scenarios if you
don't purge the database first.
The request was that a configurable limit gets added to each service
which is checked as part of the service's pre-upgrade check routine [3]
and warn if the number of records to purge is over that limit.
For example, the nova-status upgrade check could warn if there are over
100000 deleted records total across all cells databases. Maybe cinder
would have something similar for deleted volumes. Keystone could have
something for revoked tokens.
Another idea in the room was flagging on records over a certain age
limit. For example, if there are deleted instances in nova that were
deleted >1 year ago.
How do people feel about this? It seems pretty straight-forward to me.
If people are generally in favor of this, then the question is what
would be sane defaults - or should we not assume a default and force
operators to opt into this?
* nova delete doesn't actually delete the record from the instances
table, it flips a value to hide it - you have to archive/purge those
records to get them out of the main table.
[1] https://bugs.launchpad.net/nova/+bug/1791824
[2] https://etherpad.openstack.org/p/upgrade-sig-ptg-stein
[3] https://governance.openstack.org/tc/goals/stein/upgrade-checkers.html
--
Thanks,
Matt
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
SIG room at the PTG today [2] and would like to see if there is any
feedback from other operators/developers that weren't part of the
discussion.
The basic problem is that failing to archive/purge deleted records* from
the database can make upgrades much slower during schema migrations.
Anecdotes from the room mentioned that it can be literally impossible to
complete upgrades for keystone and heat in certain scenarios if you
don't purge the database first.
The request was that a configurable limit gets added to each service
which is checked as part of the service's pre-upgrade check routine [3]
and warn if the number of records to purge is over that limit.
For example, the nova-status upgrade check could warn if there are over
100000 deleted records total across all cells databases. Maybe cinder
would have something similar for deleted volumes. Keystone could have
something for revoked tokens.
Another idea in the room was flagging on records over a certain age
limit. For example, if there are deleted instances in nova that were
deleted >1 year ago.
How do people feel about this? It seems pretty straight-forward to me.
If people are generally in favor of this, then the question is what
would be sane defaults - or should we not assume a default and force
operators to opt into this?
* nova delete doesn't actually delete the record from the instances
table, it flips a value to hide it - you have to archive/purge those
records to get them out of the main table.
[1] https://bugs.launchpad.net/nova/+bug/1791824
[2] https://etherpad.openstack.org/p/upgrade-sig-ptg-stein
[3] https://governance.openstack.org/tc/goals/stein/upgrade-checkers.html
--
Thanks,
Matt
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators