settingsLogin | Registersettings

Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing this wrong?

0 votes

Adding [kolla] tag.


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

From: "Kris G. Lindgren" klindgren@godaddy.com
Date: Friday, January 20, 2017 at 4:54 PM
To: "openstack-dev@lists.openstack.org" openstack-dev@lists.openstack.org
Cc: "openstack-operators@lists.openstack.org" openstack-operators@lists.openstack.org
Subject: Re: [kolla-ansible] Am I doing this wrong?

Poke. Bueller?


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

From: "Kris G. Lindgren" klindgren@godaddy.com
Date: Tuesday, January 10, 2017 at 5:34 PM
To: "openstack-dev@lists.openstack.org" openstack-dev@lists.openstack.org
Subject: [kolla-ansible] Am I doing this wrong?

Hello Kolla/Kolla-ansible peoples.

I have been trying to take kolla/kolla-ansible and use it to start moving our existing openstack deployment into containers. At the same time also trying to fix some of the problems that we created with our previous deployment work (everything was in puppet). Where we had puppet doing everything which eventually created a system that effectively performed actions at a distance. As we were never really 100% what puppet was going to do when we ran it. Even with NOOP mode enabled. So taking an example of building and deploying glance via kolla-ansible. I am running into some problems/concerns and wanted to reach out to make sure that I am not missing something.

Things that I am noticing:
* I need to define a number of servers in my inventory outside of the specific servers that I want to perform actions against. I need to define groups baremetal, rabbitmq, memcached, and control (IN addition to the glance specific groups) most of these seem to be gathering information for config? (Baremetal was needed soley to try to run the bootstrap play). Running a change specifically against "glance" causes fact gathering on a number of other servers not specifically where glance is running? My concern here is that I want to be able to run kola-ansible against a specific service and know that only those servers are being logged into.

  • I want to run a dry-run only, being able to see what will happen before it happens, not during; during makes it really hard to see what will happen until it happens. Also supporting ansible --diff would really help in understanding what will be changed (before it happens). Ideally, this wouldn’t be 100% needed. But the ability to figure out what a run would ACTUALLY do on a box is what I was hoping to see.

  • Database task are ran on every deploy and status of change DB permissions always reports as changed? Even when nothing happens, which makes you wonder "what changed"? Seems like this is because the task either reports a 0 or a 1, where it seems like there is 3 states, did nothing, updated something, failed to do what was required. Also, Can someone tell me why the DB stuff is done on a deployment task? Seems like the db checks/migration work should only be done on a upgrade or a bootstrap?

  • Database services (that at least we have) our not managed by our team, so don't want kolla-ansible touching those (since it won't be able to). No way to mark the DB as "externally managed"? IE we dont have permissions to create databases or add users. But we got all other permissions on the databases that are created, so normal db-manage tooling works.

  • Maintenance level operations; doesn't seem to be any built-in to say 'take a server out of a production state, deploy to it, test it, put it back into production' Seems like if kola-ansible is doing haproxy for API's, it should be managing this? Or an extension point to allow us to run our own maintenance/testing scripts?

  • Config must come from kolla-ansible and generated templates. I know we have a patch up for externally managed service configuration. But if we aren't suppose to use kolla-ansible for generating configs (see below), why cant we override this piece?

Hard to determine what kolla-ansible should be used for:

  • Certain parts of it are 'reference only' (the config tasks), some are not recommended
    to be used at all (bootstrap?); what is the expected parts of kolla-ansible people are
    actually using (and not just as a reference point); if parts of kolla-ansible are just
    reference only then might as well be really upfront about it and tell people how to
    disable/replace those reference pieces?

  • Seems like this will cause everyone who needs to make tweaks to fork or create an "overlay" to override playbooks/tasks with specific functions?

Other questions:

Is kolla-ansibles design philosophy that every deployment is an upgrade? Or every deployment should include all the base level boostrap tests?

Because it seems to me that you have a required set of tasks that should only be done once (boot strap). Another set of tasks that should be done for day to day care/feeding: service restarts, config changes, updates to code (new container deployments), package updates (new docker container deployment). And a final set of tasks for upgrades where you will need to do things like db migrations and other special upgrade things. It also seems like the day to day care and feeding tasks should be incredibly targeted/explicit. For example, deploying a new glance container (not in an upgrade scenario). I would expect it to login to the glance servers one at a time. Place the server in maintenance mode to ensure that actions are not performed against it. Downloaded the new container. Start the new container. Test the new container, if successful, place the new container into rotation. Stop the old container. Remove the server from maintenance mode. Move on to the next server. All of that would only need to involve login into the glance servers. In testing kola-ansible it does not seem like the act of deploying is that targeted?


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Jan 21, 2017 in openstack-dev by Kris_G._Lindgren (7,740 points)   1 5 10

7 Responses

0 votes

Kris,

Thanks for adding the kolla tag. As we reach milestone 3, every kolla dev is knee deep in development and probably not reading mails not directly tagged with [kolla].

IOutlook 2016 has a bug where I cannot respond inline. I am replying here so my gmail account picks up the email so that I can respond inline.

It will take me several hours to process your email and I’m not certain I can answer all of the questions to your satisfaction as kolla is a pretty large project and it is difficult for any one person to know all of the details. I will hopefully have a response sometime this evening for the things I can answer. Hopefully others in the community can fill in the gaps.

Regards
-steve

From: "Kris G. Lindgren" klindgren@godaddy.com
Reply-To: "OpenStack Development Mailing List (not for usage questions)" openstack-dev@lists.openstack.org
Date: Friday, January 20, 2017 at 6:03 PM
To: "openstack-dev@lists.openstack.org" openstack-dev@lists.openstack.org
Cc: "openstack-operators@lists.openstack.org" openstack-operators@lists.openstack.org
Subject: Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing this wrong?

Adding [kolla] tag.


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

From: "Kris G. Lindgren" klindgren@godaddy.com
Date: Friday, January 20, 2017 at 4:54 PM
To: "openstack-dev@lists.openstack.org" openstack-dev@lists.openstack.org
Cc: "openstack-operators@lists.openstack.org" openstack-operators@lists.openstack.org
Subject: Re: [kolla-ansible] Am I doing this wrong?

Poke. Bueller?


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

From: "Kris G. Lindgren" klindgren@godaddy.com
Date: Tuesday, January 10, 2017 at 5:34 PM
To: "openstack-dev@lists.openstack.org" openstack-dev@lists.openstack.org
Subject: [kolla-ansible] Am I doing this wrong?

Hello Kolla/Kolla-ansible peoples.

I have been trying to take kolla/kolla-ansible and use it to start moving our existing openstack deployment into containers. At the same time also trying to fix some of the problems that we created with our previous deployment work (everything was in puppet). Where we had puppet doing everything which eventually created a system that effectively performed actions at a distance. As we were never really 100% what puppet was going to do when we ran it. Even with NOOP mode enabled. So taking an example of building and deploying glance via kolla-ansible. I am running into some problems/concerns and wanted to reach out to make sure that I am not missing something.

Things that I am noticing:
* I need to define a number of servers in my inventory outside of the specific servers that I want to perform actions against. I need to define groups baremetal, rabbitmq, memcached, and control (IN addition to the glance specific groups) most of these seem to be gathering information for config? (Baremetal was needed soley to try to run the bootstrap play). Running a change specifically against "glance" causes fact gathering on a number of other servers not specifically where glance is running? My concern here is that I want to be able to run kola-ansible against a specific service and know that only those servers are being logged into.

  • I want to run a dry-run only, being able to see what will happen before it happens, not during; during makes it really hard to see what will happen until it happens. Also supporting ansible --diff would really help in understanding what will be changed (before it happens). Ideally, this wouldn’t be 100% needed. But the ability to figure out what a run would ACTUALLY do on a box is what I was hoping to see.

  • Database task are ran on every deploy and status of change DB permissions always reports as changed? Even when nothing happens, which makes you wonder "what changed"? Seems like this is because the task either reports a 0 or a 1, where it seems like there is 3 states, did nothing, updated something, failed to do what was required. Also, Can someone tell me why the DB stuff is done on a deployment task? Seems like the db checks/migration work should only be done on a upgrade or a bootstrap?

  • Database services (that at least we have) our not managed by our team, so don't want kolla-ansible touching those (since it won't be able to). No way to mark the DB as "externally managed"? IE we dont have permissions to create databases or add users. But we got all other permissions on the databases that are created, so normal db-manage tooling works.

  • Maintenance level operations; doesn't seem to be any built-in to say 'take a server out of a production state, deploy to it, test it, put it back into production' Seems like if kola-ansible is doing haproxy for API's, it should be managing this? Or an extension point to allow us to run our own maintenance/testing scripts?

  • Config must come from kolla-ansible and generated templates. I know we have a patch up for externally managed service configuration. But if we aren't suppose to use kolla-ansible for generating configs (see below), why cant we override this piece?

Hard to determine what kolla-ansible should be used for:

  • Certain parts of it are 'reference only' (the config tasks), some are not recommended
    to be used at all (bootstrap?); what is the expected parts of kolla-ansible people are
    actually using (and not just as a reference point); if parts of kolla-ansible are just
    reference only then might as well be really upfront about it and tell people how to
    disable/replace those reference pieces?

  • Seems like this will cause everyone who needs to make tweaks to fork or create an "overlay" to override playbooks/tasks with specific functions?

Other questions:

Is kolla-ansibles design philosophy that every deployment is an upgrade? Or every deployment should include all the base level boostrap tests?

Because it seems to me that you have a required set of tasks that should only be done once (boot strap). Another set of tasks that should be done for day to day care/feeding: service restarts, config changes, updates to code (new container deployments), package updates (new docker container deployment). And a final set of tasks for upgrades where you will need to do things like db migrations and other special upgrade things. It also seems like the day to day care and feeding tasks should be incredibly targeted/explicit. For example, deploying a new glance container (not in an upgrade scenario). I would expect it to login to the glance servers one at a time. Place the server in maintenance mode to ensure that actions are not performed against it. Downloaded the new container. Start the new container. Test the new container, if successful, place the new container into rotation. Stop the old container. Remove the server from maintenance mode. Move on to the next server. All of that would only need to involve login into the glance servers. In testing kola-ansible it does not seem like the act of deploying is that targeted?


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 23, 2017 by Steven_Dake_(stdake) (24,540 points)   2 10 23
0 votes

Hi Kris,

Thanks for the feedback, I think everyone involved in kolla-ansible
should take the time to read through it, as it definitely highlights
some areas that we need to improve.

There's a lot of questions here, so I haven't gone into too much detail
on any specific one; my hope is that I can clear up the majority of it
and then we can follow up on some of the topics that require more
discussion.

Hope it helps,
-Paul

 * I need to define a number of servers in my inventory outside of
the specific servers that I want to perform actions against.  I need
to define groups baremetal, rabbitmq, memcached, and control (IN
addition to the glance specific groups) most of these seem to be
gathering information for config? (Baremetal was needed soley to try
to run the bootstrap play)

You only need to define the top level groups, i.e. control, network,
storage, monitoring, etc. If you don't want or have dedicated nodes for
each of these groups it's fine to put the same node into multiple
groups. So for example, if you're not interested in monitoring right
now, you can just put your control node(s) under this and forget about
it. The groups marked with [*:children] (e.g. bootstrap) are "groups of
groups" and you shouldn't need to modify these at all.

Running a change specifically against
"glance" causes fact gathering on a number of other servers not
specifically where glance is running? My concern here is that I
want to be able to run kola-ansible against a specific service and
know that only those servers are being logged into.

The fact gathering on every server is a compromise taken by Kolla to
work around limitations in Ansible. It works well for the majority of
situations; for more detail and potential improvements on this please
have a read of this post:
http://lists.openstack.org/pipermail/openstack-dev/2016-November/107833.html

* I want to run a dry-run only, being able to see what will happen
before it happens, not during; during makes it really hard to see
what will happen until it happens. Also supporting  `ansible --diff`
would really help in understanding what will be changed (before it
happens).

Agree a dry run would be useful, I believe it came up during the
Barcelona design summit but has not yet been looked at. The ansible
--diff sounds like something we could easily do, if you could log a
blueprint at blueprints.launchpad.net/kolla-ansible I think that would help.

* Database task are ran on every deploy and status of change DB
permissions always reports as changed? Even when nothing happens,
which makes you wonder "what changed"?

This shouldn't be the case, I just double checked taking Glance as an
example, it reports "ok" (no change) for all runs after the initial
deploy. Perhaps you've come across a bug, if you think this is the case
please log one.

Also, Can someone tell me why the DB stuff is done on a
deployment task? Seems like the db checks/migration work should
only be done on a upgrade or a bootstrap?

Deploy includes bootstrap, but bootstrap is only done if the database is
not found (or on upgrade). Again it sounds like you're coming across
some unusual behavior here, suggest checking in with us on

openstack-kolla or filing a bug.

* Database services (that at least we have) our not managed by our
team, so don't want kolla-ansible touching those (since it won't be
able to). No way to mark the DB as "externally managed"?  IE we dont
have permissions to create databases or add users.  But we got all
other permissions on the databases that are created, so normal
db-manage tooling works.

This is definitely something we need - I'm pretty sure I saw something
around this in the review queue very recently. I can't find it off hand
so hopefully someone can chip in here on the status of this work.

* Maintenance level operations; doesn't seem to be any built-in to
say 'take a server out  of a production state, deploy to it, test
it, put it back into production'  Seems like if kola-ansible is
doing haproxy for API's, it should be managing this?  Or an
extension point to allow us to run our own maintenance/testing 

scripts?

Again, discussed, needs to happen, but not there as of yet.

* Config must come from kolla-ansible and generated templates.  I
know we have a patch up for externally managed service
configuration.  But if we aren't suppose to use kolla-ansible for
generating configs (see below), why cant we override this piece?

I'm not quite following you here, the config templates from
kolla-ansible are one of it's stronger pieces imo, they're reasonably
well tested and maintained. What leads you to believe they shouldn't be
used?

* Certain parts of it are 'reference only' (the config tasks),
are not recommended

This is untrue - kolla-ansible is designed to stand up a stable and
usable OpenStack 'out of the box'. There are definitely gaps in the
operator type tasks as you've highlighted, but I would not call it
'reference only'.

Is kolla-ansibles design philosophy that every deployment is an
upgrade?  Or every deployment should include all the base level
boostrap tests?

Every deployment should be idempotent. This is key to kolla-ansible's
philosophy. It sounds like you've come across some tasks that are
performing this way which is confusing things. Upgrades are only
performed if explicitly requested (i.e. kolla-ansible upgrade).

On 23/01/17 00:06, Steven Dake (stdake) wrote:
Kris,

Thanks for adding the kolla tag. As we reach milestone 3, every kolla
dev is knee deep in development and probably not reading mails not
directly tagged with [kolla].

IOutlook 2016 has a bug where I cannot respond inline. I am replying
here so my gmail account picks up the email so that I can respond inline.

It will take me several hours to process your email and I’m not certain
I can answer all of the questions to your satisfaction as kolla is a
pretty large project and it is difficult for any one person to know all
of the details. I will hopefully have a response sometime this evening
for the things I can answer. Hopefully others in the community can fill
in the gaps.

Regards

-steve

*From: *"Kris G. Lindgren" <klindgren@godaddy.com>
*Reply-To: *"OpenStack Development Mailing List (not for usage
questions)" <openstack-dev@lists.openstack.org>
*Date: *Friday, January 20, 2017 at 6:03 PM
*To: *"openstack-dev@lists.openstack.org"
<openstack-dev@lists.openstack.org>
*Cc: *"openstack-operators@lists.openstack.org"
<openstack-operators@lists.openstack.org>
*Subject: *Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing
this wrong?



Adding [kolla] tag.





___________________________________________________________________

Kris Lindgren

Senior Linux Systems Engineer

GoDaddy



*From: *"Kris G. Lindgren" <klindgren@godaddy.com>
*Date: *Friday, January 20, 2017 at 4:54 PM
*To: *"openstack-dev@lists.openstack.org"
<openstack-dev@lists.openstack.org>
*Cc: *"openstack-operators@lists.openstack.org"
<openstack-operators@lists.openstack.org>
*Subject: *Re: [kolla-ansible] Am I doing this wrong?



Poke.  Bueller?





___________________________________________________________________

Kris Lindgren

Senior Linux Systems Engineer

GoDaddy



*From: *"Kris G. Lindgren" <klindgren@godaddy.com>
*Date: *Tuesday, January 10, 2017 at 5:34 PM
*To: *"openstack-dev@lists.openstack.org"
<openstack-dev@lists.openstack.org>
*Subject: *[kolla-ansible] Am I doing this wrong?



Hello Kolla/Kolla-ansible peoples.



I have been trying to take kolla/kolla-ansible and use it to start
moving our existing openstack deployment into containers.  At the
same time also trying to fix some of the problems that we created
with our previous deployment work (everything was in puppet).  Where
we had puppet doing **everything** which eventually created a system
that effectively performed actions at a distance.  As we were never
really 100% what puppet was going to do when we ran it.  Even with
NOOP mode enabled.  So taking an example of building and deploying
glance via kolla-ansible. I am running into some problems/concerns
and wanted to reach out to make sure that I am not missing something.



Things that I am noticing:

 * I need to define a number of servers in my inventory outside of
the specific servers that I want to perform actions against.  I need
to define groups baremetal, rabbitmq, memcached, and control (IN
addition to the glance specific groups) most of these seem to be
gathering information for config? (Baremetal was needed soley to try
to run the bootstrap play).  Running a change specifically against
"glance" causes fact gathering on a number of other servers not
specifically where glance is running?  My concern here is that I
want to be able to run kola-ansible against a specific service and
know that only those servers are being logged into.



* I want to run a dry-run only, being able to see what will happen
before it happens, not during; during makes it really hard to see
what will happen until it happens. Also supporting  `ansible --diff`
would really help in understanding what will be changed (before it
happens).  Ideally, this wouldn’t be 100% needed.  But the ability
to figure out what a run would **ACTUALLY** do on a box is what I
was hoping to see.



* Database task are ran on every deploy and status of change DB
permissions always reports as changed? Even when nothing happens,
which makes you wonder "what changed"?  Seems like this is because
the task either reports a 0 or a 1, where it seems like there is 3
states, did nothing, updated something, failed to do what was
required.  Also, Can someone tell me why the DB stuff is done on a
deployment task?  Seems like the db checks/migration work should
only be done on a upgrade or a bootstrap?



* Database services (that at least we have) our not managed by our
team, so don't want kolla-ansible touching those (since it won't be
able to). No way to mark the DB as "externally managed"?  IE we dont
have permissions to create databases or add users.  But we got all
other permissions on the databases that are created, so normal
db-manage tooling works.



* Maintenance level operations; doesn't seem to be any built-in to
say 'take a server out  of a production state, deploy to it, test
it, put it back into production'  Seems like if kola-ansible is
doing haproxy for API's, it should be managing this?  Or an
extension point to allow us to run our own maintenance/testing scripts?



* Config must come from kolla-ansible and generated templates.  I
know we have a patch up for externally managed service
configuration.  But if we aren't suppose to use kolla-ansible for
generating configs (see below), why cant we override this piece?



Hard to determine what kolla-ansible *should* be used for:



* Certain parts of it are 'reference only' (the config tasks), some
are not recommended

  to be used at all (bootstrap?); what is the expected parts of
kolla-ansible people are

  actually using (and not just as a reference point); if parts of
kolla-ansible are just

  *reference only* then might as well be really upfront about it and
tell people how to

  disable/replace those reference pieces?



* Seems like this will cause everyone who needs to make tweaks to
fork or create an "overlay" to override playbooks/tasks with
specific functions?



Other questions:



Is kolla-ansibles design philosophy that every deployment is an
upgrade?  Or every deployment should include all the base level
boostrap tests?



Because it seems to me that you have a required set of tasks that
should only be done once (boot strap).  Another set of tasks that
should be done for day to day care/feeding: service restarts, config
changes, updates to code (new container deployments), package
updates (new docker container deployment).  And a final set of tasks
for upgrades where you will need to do things like db migrations and
other special upgrade things.  It also seems like the day to day
care and feeding tasks should be incredibly targeted/explicit. For
example, deploying a new glance container (not in an upgrade
scenario).  I would expect it to login to the glance servers one at
a time.  Place the server in maintenance mode to ensure that actions
are not performed against it.  Downloaded the new container.  Start
the new container.  Test the new container, if successful, place the
new container into rotation.  Stop the old container.  Remove the
server from maintenance mode.  Move on to the next server.  All of
that would only need to involve login into the glance servers.  In
testing kola-ansible it does not seem like the act of deploying is
that targeted?





___________________________________________________________________

Kris Lindgren

Senior Linux Systems Engineer

GoDaddy


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 23, 2017 by Paul_Bourke (3,840 points)   1 3
0 votes

Ah, I think you may be misreading what Sean is saying there. What he
means is kolla-ansible provides the bare minimum config templates to
make the service work. To template every possible config option would be
too much of a maintenance burden on the project.

Of course, users will want to customise these. But instead of modifying
the templates directly, we recommend you use the "config override"
mechanism [0]

This has a number of benefits, the main one being that you can pick up
new releases of Kolla and not get stuck in merge hell, Ansible will pick
up the Kolla base templates and merge them with user provided overrides.

Wrt to the fact gathering, I understand your concern, we essentially
have the same problem in our team. It can be raised again for further
discussion, I'm sure there's other ways it can be solved.

[0]
http://docs.openstack.org/developer/kolla-ansible/advanced-configuration.html#openstack-service-configuration-in-kolla

-Paul

On 23/01/17 18:03, Kris G. Lindgren wrote:
Hi Paul,

Thanks for responding.

The fact gathering on every server is a compromise taken by Kolla to

work around limitations in Ansible. It works well for the majority of

situations; for more detail and potential improvements on this please

have a read of this post:

http://lists.openstack.org/pipermail/openstack-dev/2016-November/107833.html

So my problem with this is the logging in to the compute nodes. While
this may be fine for a smaller deployment. Logging into thousands, even
hundreds, of nodes via ansible to gather facts, just to do a deployment
against 2 or 3 of them is not tenable. Additionally, in our higher
audited environments (pki/pci) will cause our auditors heartburn.

I'm not quite following you here, the config templates from

kolla-ansible are one of it's stronger pieces imo, they're reasonably

well tested and maintained. What leads you to believe they shouldn't be

used?

* Certain parts of it are 'reference only' (the config tasks),

are not recommended

This is untrue - kolla-ansible is designed to stand up a stable and

usable OpenStack 'out of the box'. There are definitely gaps in the

operator type tasks as you've highlighted, but I would not call it

‘reference only'.

http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-01-09.log.html#t2017-01-09T21:33:15

This is where we were told the config stuff was “reference only”?


Kris Lindgren

Senior Linux Systems Engineer

GoDaddy


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 24, 2017 by Paul_Bourke (3,840 points)   1 3
0 votes

-----Original Message-----
From: Paul Bourke [mailto:paul.bourke@oracle.com]
Sent: Tuesday, January 24, 2017 11:49 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing this wrong?

Ah, I think you may be misreading what Sean is saying there. What he means is
kolla-ansible provides the bare minimum config templates to make the service
work. To template every possible config option would be too much of a
maintenance burden on the project.

Of course, users will want to customise these. But instead of modifying the
templates directly, we recommend you use the "config override"
mechanism [0]

This has a number of benefits, the main one being that you can pick up new
releases of Kolla and not get stuck in merge hell, Ansible will pick up the Kolla base
templates and merge them with user provided overrides.
[Mooney, Sean K] paul is correct here, I did not intend to suggest that kolla-ansible should not
Be used to generate and manage config files. I simply wanted to point out that where
Customization is required to a config, it is preferable to use the config override mechanism
When possible vs modifying the ansible templates directly.

Wrt to the fact gathering, I understand your concern, we essentially have the same
problem in our team. It can be raised again for further discussion, I'm sure there's
other ways it can be solved.
[Mooney, Sean K] I belive you are intended to be able to use the ansible --limit and --tags flags,
To restrict the plays executed and node processed by a deploy and upgrade command.
I have used the --tags flags successfully in the past, I have had less success with the --limit flag.
In theory with the right combination of --limit and --tag you should be able to constrain the node
On which facts are gathered to just those that would be modified e.g. 2-3 instead of hundreds.

[0]
http://docs.openstack.org/developer/kolla-ansible/advanced-
configuration.html#openstack-service-configuration-in-kolla

-Paul

On 23/01/17 18:03, Kris G. Lindgren wrote:

Hi Paul,

Thanks for responding.

The fact gathering on every server is a compromise taken by Kolla to

work around limitations in Ansible. It works well for the majority of

situations; for more detail and potential improvements on this please

have a read of this post:

http://lists.openstack.org/pipermail/openstack-dev/2016-November/1078
33.html

So my problem with this is the logging in to the compute nodes. While
this may be fine for a smaller deployment. Logging into thousands,
even hundreds, of nodes via ansible to gather facts, just to do a
deployment against 2 or 3 of them is not tenable. Additionally, in
our higher audited environments (pki/pci) will cause our auditors heartburn.

I'm not quite following you here, the config templates from

kolla-ansible are one of it's stronger pieces imo, they're reasonably

well tested and maintained. What leads you to believe they shouldn't
be

used?

* Certain parts of it are 'reference only' (the config tasks),

are not recommended

This is untrue - kolla-ansible is designed to stand up a stable and

usable OpenStack 'out of the box'. There are definitely gaps in the

operator type tasks as you've highlighted, but I would not call it

'reference only'.

http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack
-kolla.2017-01-09.log.html#t2017-01-09T21:33:15

This is where we were told the config stuff was "reference only"?


Kris Lindgren

Senior Linux Systems Engineer

GoDaddy


__

____ OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 25, 2017 by Mooney,_Sean_K (3,580 points)   3 9
0 votes

[Mooney, Sean K] I belive you are intended to be able to use the
ansible --limit and --tags flags,
To restrict the plays executed and node processed by a deploy and
upgrade command.
I have used the --tags flags successfully in the past, I have had
less success with the --limit flag.
In theory with the right combination of --limit and --tag you should
be able to constrain the node
On which facts are gathered to just those that would be modified e.g.
2-3 instead of hundreds.

Unfortunately it's not that simple, --limit by default will restrict
fact gathering to that node only, resulting in missing facts for
templates that require info about other nodes. Hence we have plays in to
explicity gather facts when limit is used. See
https://github.com/openstack/kolla-ansible/blob/master/ansible/site.yml#L15-L32
for info on this also. Perhaps we're overcomplicating it, might be worth
reaching out to people from Ansible for more info.

On 25/01/17 00:01, Mooney, Sean K wrote:

-----Original Message-----
From: Paul Bourke [mailto:paul.bourke@oracle.com]
Sent: Tuesday, January 24, 2017 11:49 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing this wrong?

Ah, I think you may be misreading what Sean is saying there. What he means is
kolla-ansible provides the bare minimum config templates to make the service
work. To template every possible config option would be too much of a
maintenance burden on the project.

Of course, users will want to customise these. But instead of modifying the
templates directly, we recommend you use the "config override"
mechanism [0]

This has a number of benefits, the main one being that you can pick up new
releases of Kolla and not get stuck in merge hell, Ansible will pick up the Kolla base
templates and merge them with user provided overrides.
[Mooney, Sean K] paul is correct here, I did not intend to suggest that kolla-ansible should not
Be used to generate and manage config files. I simply wanted to point out that where
Customization is required to a config, it is preferable to use the config override mechanism
When possible vs modifying the ansible templates directly.

Wrt to the fact gathering, I understand your concern, we essentially have the same
problem in our team. It can be raised again for further discussion, I'm sure there's
other ways it can be solved.
[Mooney, Sean K] I belive you are intended to be able to use the ansible --limit and --tags flags,
To restrict the plays executed and node processed by a deploy and upgrade command.
I have used the --tags flags successfully in the past, I have had less success with the --limit flag.
In theory with the right combination of --limit and --tag you should be able to constrain the node
On which facts are gathered to just those that would be modified e.g. 2-3 instead of hundreds.

[0]
http://docs.openstack.org/developer/kolla-ansible/advanced-
configuration.html#openstack-service-configuration-in-kolla

-Paul

On 23/01/17 18:03, Kris G. Lindgren wrote:

Hi Paul,

Thanks for responding.

The fact gathering on every server is a compromise taken by Kolla to

work around limitations in Ansible. It works well for the majority of

situations; for more detail and potential improvements on this please

have a read of this post:

http://lists.openstack.org/pipermail/openstack-dev/2016-November/1078
33.html

So my problem with this is the logging in to the compute nodes. While
this may be fine for a smaller deployment. Logging into thousands,
even hundreds, of nodes via ansible to gather facts, just to do a
deployment against 2 or 3 of them is not tenable. Additionally, in
our higher audited environments (pki/pci) will cause our auditors heartburn.

I'm not quite following you here, the config templates from

kolla-ansible are one of it's stronger pieces imo, they're reasonably

well tested and maintained. What leads you to believe they shouldn't
be

used?

* Certain parts of it are 'reference only' (the config tasks),

are not recommended

This is untrue - kolla-ansible is designed to stand up a stable and

usable OpenStack 'out of the box'. There are definitely gaps in the

operator type tasks as you've highlighted, but I would not call it

'reference only'.

http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack
-kolla.2017-01-09.log.html#t2017-01-09T21:33:15

This is where we were told the config stuff was "reference only"?


Kris Lindgren

Senior Linux Systems Engineer

GoDaddy


__

____ OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 25, 2017 by Paul_Bourke (3,840 points)   1 3
0 votes

Hi Paul,

Thanks for responding.

The fact gathering on every server is a compromise taken by Kolla to
work around limitations in Ansible. It works well for the majority of
situations; for more detail and potential improvements on this please
have a read of this post:
http://lists.openstack.org/pipermail/openstack-dev/2016-November/107833.html

So my problem with this is the logging in to the compute nodes. While this may be fine for a smaller deployment. Logging into thousands, even hundreds, of nodes via ansible to gather facts, just to do a deployment against 2 or 3 of them is not tenable. Additionally, in our higher audited environments (pki/pci) will cause our auditors heartburn.

I'm not quite following you here, the config templates from
kolla-ansible are one of it's stronger pieces imo, they're reasonably
well tested and maintained. What leads you to believe they shouldn't be
used?

* Certain parts of it are 'reference only' (the config tasks),
are not recommended

This is untrue - kolla-ansible is designed to stand up a stable and
usable OpenStack 'out of the box'. There are definitely gaps in the
operator type tasks as you've highlighted, but I would not call it
‘reference only'.

http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-01-09.log.html#t2017-01-09T21:33:15

This is where we were told the config stuff was “reference only”?


Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 25, 2017 by Kris_G._Lindgren (7,740 points)   1 5 10
0 votes

Thanks peeps for responding to Kris. Kris, I had offered a response – do you need further information answered? It looks to me like all the questions have been answered by others in the community. If not, feel free to respond and I’ll answer the remainders.

Sean when your around and I am please ping me – it looks like you have the same outlook problem as I do and have sort of figured out how to solve it. I’d like to know how so I don’t have to top post or use gmail for the ml.

Regards
-steve

-----Original Message-----
From: "sean.k.mooney@intel.com" sean.k.mooney@intel.com
Reply-To: "OpenStack Development Mailing List (not for usage questions)" openstack-dev@lists.openstack.org
Date: Tuesday, January 24, 2017 at 5:01 PM
To: "OpenStack Development Mailing List (not for usage questions)" openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing this wrong?

> -----Original Message-----
> From: Paul Bourke [mailto:paul.bourke@oracle.com]
> Sent: Tuesday, January 24, 2017 11:49 AM
> To: OpenStack Development Mailing List (not for usage questions) <openstack-
> dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing this wrong?
> 
> Ah, I think you may be misreading what Sean is saying there. What he means is
> kolla-ansible provides the bare minimum config templates to make the service
> work. To template every possible config option would be too much of a
> maintenance burden on the project.
> 
> Of course, users will want to customise these. But instead of modifying the
> templates directly, we recommend you use the "config override"
> mechanism [0]
> 
> This has a number of benefits, the main one being that you can pick up new
> releases of Kolla and not get stuck in merge hell, Ansible will pick up the Kolla base
> templates and merge them with user provided overrides.
[Mooney, Sean K] paul is correct here, I did not intend to suggest that kolla-ansible should not
Be used to generate and manage config files. I simply wanted to point out that where
Customization is required to a config, it is preferable to use the config override mechanism
When possible vs modifying the ansible templates directly.
> 
> Wrt to the fact gathering, I understand your concern, we essentially have the same
> problem in our team. It can be raised again for further discussion, I'm sure there's
> other ways it can be solved.
[Mooney, Sean K] I belive you are intended to be able to use the ansible --limit and --tags flags,
To restrict the plays executed and node processed by a deploy and upgrade command.
I have used the --tags flags successfully in the past, I have had less success with the --limit flag.
In theory with the right combination of --limit and --tag you should be able to constrain  the node
On which facts are gathered to just those that would be modified e.g. 2-3 instead of hundreds. 
> 
> [0]
> http://docs.openstack.org/developer/kolla-ansible/advanced-
> configuration.html#openstack-service-configuration-in-kolla
> 
> -Paul
> 
> On 23/01/17 18:03, Kris G. Lindgren wrote:
> > Hi Paul,
> >
> >
> >
> > Thanks for responding.
> >
> >
> >
> >> The fact gathering on every server is a compromise taken by Kolla to
> >
> >> work around limitations in Ansible. It works well for the majority of
> >
> >> situations; for more detail and potential improvements on this please
> >
> >> have a read of this post:
> >
> >> http://lists.openstack.org/pipermail/openstack-dev/2016-November/1078
> >> 33.html
> >
> >
> >
> > So my problem with this is the logging in to the compute nodes.  While
> > this may be fine for a smaller deployment.  Logging into thousands,
> > even hundreds, of nodes via ansible to gather facts, just to do a
> > deployment against 2 or 3 of them is not tenable.  Additionally, in
> > our higher audited environments (pki/pci) will cause our auditors heartburn.
> >
> >
> >
> >> I'm not quite following you here, the config templates from
> >
> >> kolla-ansible are one of it's stronger pieces imo, they're reasonably
> >
> >> well tested and maintained. What leads you to believe they shouldn't
> >> be
> >
> >> used?
> >
> >>
> >
> >> >     * Certain parts of it are 'reference only' (the config tasks),
> >
> >>  >     are not recommended
> >
> >>
> >
> >> This is untrue - kolla-ansible is designed to stand up a stable and
> >
> >> usable OpenStack 'out of the box'. There are definitely gaps in the
> >
> >> operator type tasks as you've highlighted, but I would not call it
> >
> >> 'reference only'.
> >
> >
> >
> > http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack
> > -kolla.2017-01-09.log.html#t2017-01-09T21:33:15
> >
> >
> >
> >
> > This is where we were told the config stuff was "reference only"?
> >
> >
> >
> >
> ___________________________________________________________________
> >
> > Kris Lindgren
> >
> > Senior Linux Systems Engineer
> >
> > GoDaddy
> >
> >
> >
> >
> ____________________________________________________________________
> __
> > ____ OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> ____________________________________________________________________
> ______
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 25, 2017 by Steven_Dake_(stdake) (24,540 points)   2 10 23
...