settingsLogin | Registersettings

[openstack-dev] [openstack-ansible] When to purge the DB, and when not to purge the DB?

0 votes

Hi everyone,

In a recent conversation on the Operators list [1] there was a discussion about purging archived data in the database. It would seem to me an important step in maintaining an environment which should be done from time to time and perhaps at the very least prior to a major upgrade.

What’re the thoughts on how often this should be done? Should we include it as an opt-in step, or an opt-out step?

[1] http://lists.openstack.org/pipermail/openstack-operators/2016-June/010813.html


Jesse Pretorius
IRC: odyssey4me
http://lists.openstack.org/pipermail/openstack-operators/2016-June/010813.html


Rackspace Limited is a company registered in England & Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail message may contain confidential or privileged information intended for the recipient. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at abuse@rackspace.com and delete the original message. Your cooperation is appreciated.

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Jul 1, 2016 in openstack-dev by Jesse.Pretorius_at_r (2,260 points)   1 3
retagged Jan 26, 2017 by admin

6 Responses

0 votes

-----Original Message-----
From: Jesse Pretorius jesse.pretorius@rackspace.co.uk
Reply: OpenStack Development Mailing List (not for usage questions)

Date: July 1, 2016 at 04:45:17
To: OpenStack Development Mailing List (not for usage questions)

Subject:  [openstack-dev] [openstack-ansible] When to purge the DB,
and when not to purge the DB?

Hi everyone,

In a recent conversation on the Operators list [1] there was a discussion about purging
archived data in the database. It would seem to me an important step in maintaining an
environment which should be done from time to time and perhaps at the very least prior
to a major upgrade.

What’re the thoughts on how often this should be done? Should we include it as an opt-in
step, or an opt-out step?

[1] http://lists.openstack.org/pipermail/openstack-operators/2016-June/010813.html

Is OpenStack-Ansible now going to get into the minutae of operating
the entire cloud? I was under the impression that it left the easy
things to the operators (e.g., deciding when and how to purge the
database) while taking care of the things that are less obvious
(setting up OpenStack, and interacting with the database directly to
only set up things for the services).

--
Ian Cordasco


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 1, 2016 by sigmavirus24_at_gmai (8,720 points)   2 2 3
0 votes

On 7/1/16, 8:48 PM, "Ian Cordasco" sigmavirus24@gmail.com wrote:

-----Original Message-----
From: Jesse Pretorius jesse.pretorius@rackspace.co.uk

In a recent conversation on the Operators list [1] there was a discussion about purging
archived data in the database. It would seem to me an important step in maintaining an
environment which should be done from time to time and perhaps at the very least prior
to a major upgrade.

What’re the thoughts on how often this should be done? Should we include it as an opt-in
step, or an opt-out step?

[1] http://lists.openstack.org/pipermail/openstack-operators/2016-June/010813.html

Is OpenStack-Ansible now going to get into the minutae of operating
the entire cloud? I was under the impression that it left the easy
things to the operators (e.g., deciding when and how to purge the
database) while taking care of the things that are less obvious
(setting up OpenStack, and interacting with the database directly to
only set up things for the services).

That’s a good question which betrays the fact that I phrased my question poorly. :)

The question is whether we should do something like this:

1) As part of the normal execution of the service playbooks;
2) As part of the automated major upgrade (i.e. The step is not optional);
3) As part of the manual major upgrade (i.e. The step is optional);
4) Never.

If never, it might be useful for our community to curate a few bits of operations tooling to automated this sort of thing on demand. The tooling can be placed into the Ops repository [1] if this is done.

[1] https://github.com/openstack/openstack-ansible-ops


Rackspace Limited is a company registered in England & Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail message may contain confidential or privileged information intended for the recipient. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at abuse@rackspace.com and delete the original message. Your cooperation is appreciated.

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 2, 2016 by Jesse.Pretorius_at_r (2,260 points)   1 3
0 votes

+1 "As part of the normal execution of the service playbooks"

w/ user_variable to turn it on/off of course; whether it defaults to on or
off is really designers choice. Some production clouds will have the
playbooks ran against them frequently, others not so much.

On Sat, Jul 2, 2016 at 7:43 AM Jesse Pretorius <
Jesse.Pretorius@rackspace.co.uk> wrote:

On 7/1/16, 8:48 PM, "Ian Cordasco" sigmavirus24@gmail.com wrote:

-----Original Message-----
From: Jesse Pretorius jesse.pretorius@rackspace.co.uk

In a recent conversation on the Operators list [1] there was a
discussion about purging
archived data in the database. It would seem to me an important step in
maintaining an
environment which should be done from time to time and perhaps at the
very least prior
to a major upgrade.

What’re the thoughts on how often this should be done? Should we
include it as an opt-in
step, or an opt-out step?

[1]
http://lists.openstack.org/pipermail/openstack-operators/2016-June/010813.html

Is OpenStack-Ansible now going to get into the minutae of operating
the entire cloud? I was under the impression that it left the easy
things to the operators (e.g., deciding when and how to purge the
database) while taking care of the things that are less obvious
(setting up OpenStack, and interacting with the database directly to
only set up things for the services).

That’s a good question which betrays the fact that I phrased my question
poorly. :)

The question is whether we should do something like this:

1) As part of the normal execution of the service playbooks;
2) As part of the automated major upgrade (i.e. The step is not optional);
3) As part of the manual major upgrade (i.e. The step is optional);
4) Never.

If never, it might be useful for our community to curate a few bits of
operations tooling to automated this sort of thing on demand. The tooling
can be placed into the Ops repository [1] if this is done.

[1] https://github.com/openstack/openstack-ansible-ops


Rackspace Limited is a company registered in England & Wales (company
registered number 03897010) whose registered office is at 5 Millington
Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy
can be viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail
message may contain confidential or privileged information intended for the
recipient. Any dissemination, distribution or copying of the enclosed
material is prohibited. If you receive this transmission in error, please
notify us immediately by e-mail at abuse@rackspace.com and delete the
original message. Your cooperation is appreciated.

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 2, 2016 by Wade_Holler (500 points)  
0 votes

The question is whether we should do something like this:

1) As part of the normal execution of the service playbooks;
2) As part of the automated major upgrade (i.e. The step is not optional);
3) As part of the manual major upgrade (i.e. The step is optional);
4) Never.

I am not an operator, but I would think that #4 is the right thing to
do. If I want to purge the database, it's going to be based on billing
reasons (or lack thereof) and be tied to another archival, audit, etc
policy that the "business people" are involved with. Install and
configuration of my services shouldn't really ever touch my data other
than mechanical upgrade scripts and the like, IMHO.

Purging the database only during upgrades is not sufficient for large
installs, so why artificially tie it to that process? In Nova we don't
do data migrations as part of schema updates anymore, so it's not like a
purge is going to make the upgrade any faster...

--Dan


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 2, 2016 by Dan_Smith (9,860 points)   1 2 4
0 votes

On Jul 2, 2016 10:37 AM, "Dan Smith" dms@danplanet.com wrote:

The question is whether we should do something like this:

1) As part of the normal execution of the service playbooks;
2) As part of the automated major upgrade (i.e. The step is not
optional);
3) As part of the manual major upgrade (i.e. The step is optional);
4) Never.

I am not an operator, but I would think that #4 is the right thing to
do. If I want to purge the database, it's going to be based on billing
reasons (or lack thereof) and be tied to another archival, audit, etc
policy that the "business people" are involved with. Install and
configuration of my services shouldn't really ever touch my data other
than mechanical upgrade scripts and the like, IMHO.

Purging the database only during upgrades is not sufficient for large
installs, so why artificially tie it to that process? In Nova we don't
do data migrations as part of schema updates anymore, so it's not like a
purge is going to make the upgrade any faster...

I agree with this sentiment. If OSA feels like it must provide automation
for purging databases, it should be in the ops repo mentioned earlier.

I see no reason to over extend upgrades with something not inherently
necessary or appropriate for upgrades.

--
Ian


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 2, 2016 by sigmavirus24_at_gmai (8,720 points)   2 2 3
0 votes

+1 to #4 -- As an operator I have archive and audit requirements that
proactive automated DB pruning would likely get in the way of. If we
do produce pruning tools I think they should exist in the OPS repo
and, as a rule, should not be part of the general deployment/upgrade
framework.

On Sat, Jul 2, 2016 at 5:20 PM, Ian Cordasco sigmavirus24@gmail.com wrote:

On Jul 2, 2016 10:37 AM, "Dan Smith" dms@danplanet.com wrote:

The question is whether we should do something like this:

1) As part of the normal execution of the service playbooks;
2) As part of the automated major upgrade (i.e. The step is not
optional);
3) As part of the manual major upgrade (i.e. The step is optional);
4) Never.

I am not an operator, but I would think that #4 is the right thing to
do. If I want to purge the database, it's going to be based on billing
reasons (or lack thereof) and be tied to another archival, audit, etc
policy that the "business people" are involved with. Install and
configuration of my services shouldn't really ever touch my data other
than mechanical upgrade scripts and the like, IMHO.

Purging the database only during upgrades is not sufficient for large
installs, so why artificially tie it to that process? In Nova we don't
do data migrations as part of schema updates anymore, so it's not like a
purge is going to make the upgrade any faster...

I agree with this sentiment. If OSA feels like it must provide automation
for purging databases, it should be in the ops repo mentioned earlier.

I see no reason to over extend upgrades with something not inherently
necessary or appropriate for upgrades.

--
Ian


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 5, 2016 by Carter,_Kevin (580 points)   1
...