settingsLogin | Registersettings

[Openstack-operators] OpenStack services and ca certificate config entries

0 votes

We're facing a bit of a frustration. In some of our environments, we're
using a self-signed certificate for our ssl termination (haproxy). We have
our various services pointing at the haproxy for service cross-talk, such
as nova to neutron or nova to glance or nova to cinder or neutron to nova
or cinder to glance or all the things to keystone. When using a self-signed
certificate, these services have trouble validating the cert when they
attempt to talk to each other. This problem can be solved in a few ways,
such as adding the CA to the system bundle (of your platform has such a
thing), adding the CA to the bundle python requests uses (because
hilariously it doesn't always use the system bundle), or the more direct
way of telling nova, neutron, et al the direct path to the CA file.

This last choice is the way we went forward, more explicit, and didn't
depend on knowledge if python-requests was using its own bundle or the
operating system's bundle. To configure this there are a few places that
need to be touched.

nova.conf:
[keystone_authtoken]
cafile =

[neutron]
cacertificatesfile =

[cinder]
cacertificatesfile =

(nothing for glance hilariously)

neutron.conf
[DEFAULT]
novacacertificates_file =

[keystone_authtoken]
cafile =

glance-api.conf and glance-registry.conf
[keystone_authtoken]
cafile =

cinder.conf
[DEFAULT]
glancecacertificates_file =

[keystone_authtoken]
cafile =

heat.conf
[clients]
ca_file =

[clients]
ca
file =

As you can see, there are a lot of places where one would have to define
the path, and the frustrating part is that the config name and section
varies across the services. Does anybody think this is a good thing? Can
anybody think of a good way forward to come to some sort of agreement on
config names? It does seem like heat is a winner here, it has a default
that can be defined for all clients, and then each client could potentially
point to a different path, but every config entry is named the same. Can we
do that across all the other services?

I chatted a bit on twitter last night with some nova folks, they suggested
starting a thread here on ops list and potentially turning it into a
hallway session or real session at the Vancouver design summit (which
operators are officially part of).

asked Mar 25, 2015 in openstack-operators by jlk_at_bluebox.net (2,760 points)   2 3

7 Responses

0 votes

I faced this very issue in the past. We solved the problem by adding the CA to the system bundle (as you stated). We also ran into problems where python would still not validate the CA. However, this turned out to be a permissions error with cacerts.txt[1] when httplib2 was installed through pip. Nowadays openstack uses requests which I don’t believe utilizes httplib2.

[1] https://code.google.com/p/httplib2/issues/detail?id=292&q=certificate

On Wednesday, March 25, 2015 at 11:13 AM, Jesse Keating wrote:

We're facing a bit of a frustration. In some of our environments, we're using a self-signed certificate for our ssl termination (haproxy). We have our various services pointing at the haproxy for service cross-talk, such as nova to neutron or nova to glance or nova to cinder or neutron to nova or cinder to glance or all the things to keystone. When using a self-signed certificate, these services have trouble validating the cert when they attempt to talk to each other. This problem can be solved in a few ways, such as adding the CA to the system bundle (of your platform has such a thing), adding the CA to the bundle python requests uses (because hilariously it doesn't always use the system bundle), or the more direct way of telling nova, neutron, et al the direct path to the CA file.

This last choice is the way we went forward, more explicit, and didn't depend on knowledge if python-requests was using its own bundle or the operating system's bundle. To configure this there are a few places that need to be touched.

nova.conf:
[keystoneauthtoken]
cafile =

[neutron]
ca
certificatesfile =

[cinder]
ca
certificatesfile =

(nothing for glance hilariously)


neutron.conf
[DEFAULT]

nova
cacertificatesfile =

[keystoneauthtoken]
cafile =

glance-api.conf and glance-registry.conf
[keystone
authtoken]
cafile =

cinder.conf
[DEFAULT]
glancecacertificatesfile =

[keystone
authtoken]
cafile =

heat.conf
[clients]
cafile =

[clients
]
ca_file =


As you can see, there are a lot of places where one would have to define the path, and the frustrating part is that the config name and section varies across the services. Does anybody think this is a good thing? Can anybody think of a good way forward to come to some sort of agreement on config names? It does seem like heat is a winner here, it has a default that can be defined for all clients, and then each client could potentially point to a different path, but every config entry is named the same. Can we do that across all the other services?

I chatted a bit on twitter last night with some nova folks, they suggested starting a thread here on ops list and potentially turning it into a hallway session or real session at the Vancouver design summit (which operators are officially part of).

- jlk
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org (mailto:OpenStack-operators@lists.openstack.org)
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Mar 25, 2015 by John_Dewey (1,320 points)   2
0 votes

This sounds like something we can bake into the session object to make it easier / more consistent.

--Morgan

Sent via mobile

On Mar 25, 2015, at 14:03, John Dewey john@dewey.ws wrote:

I faced this very issue in the past. We solved the problem by adding the CA to the system bundle (as you stated). We also ran into problems where python would still not validate the CA. However, this turned out to be a permissions error with cacerts.txt[1] when httplib2 was installed through pip. Nowadays openstack uses requests which I don’t believe utilizes httplib2.

[1] https://code.google.com/p/httplib2/issues/detail?id=292&q=certificate

On Wednesday, March 25, 2015 at 11:13 AM, Jesse Keating wrote:

We're facing a bit of a frustration. In some of our environments, we're using a self-signed certificate for our ssl termination (haproxy). We have our various services pointing at the haproxy for service cross-talk, such as nova to neutron or nova to glance or nova to cinder or neutron to nova or cinder to glance or all the things to keystone. When using a self-signed certificate, these services have trouble validating the cert when they attempt to talk to each other. This problem can be solved in a few ways, such as adding the CA to the system bundle (of your platform has such a thing), adding the CA to the bundle python requests uses (because hilariously it doesn't always use the system bundle), or the more direct way of telling nova, neutron, et al the direct path to the CA file.

This last choice is the way we went forward, more explicit, and didn't depend on knowledge if python-requests was using its own bundle or the operating system's bundle. To configure this there are a few places that need to be touched.

nova.conf:
[keystone_authtoken]
cafile =

[neutron]
cacertificatesfile =

[cinder]
cacertificatesfile =

(nothing for glance hilariously)

neutron.conf
[DEFAULT]
novacacertificates_file =

[keystone_authtoken]
cafile =

glance-api.conf and glance-registry.conf
[keystone_authtoken]
cafile =

cinder.conf
[DEFAULT]
glancecacertificates_file =

[keystone_authtoken]
cafile =

heat.conf
[clients]
ca_file =

[clients]
ca
file =

As you can see, there are a lot of places where one would have to define the path, and the frustrating part is that the config name and section varies across the services. Does anybody think this is a good thing? Can anybody think of a good way forward to come to some sort of agreement on config names? It does seem like heat is a winner here, it has a default that can be defined for all clients, and then each client could potentially point to a different path, but every config entry is named the same. Can we do that across all the other services?

I chatted a bit on twitter last night with some nova folks, they suggested starting a thread here on ops list and potentially turning it into a hallway session or real session at the Vancouver design summit (which operators are officially part of).


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Mar 25, 2015 by Morgan_Fainberg (17,320 points)   2 4 9
0 votes

We were adding our CA to the bundle, we are using python-requests.
python-requests is getting installed via pip in a venv, and it includes
it's own CA bundle. So we were creating a symlink from the system bundle to
the venv requests bundle. Then something upgraded python-requests in the
venv, which overwrote the symlink, which actually overwrote the system
bundle, for maximum lols.

At that point we decided to go with the openstack config file route :)

  • jlk

On Wed, Mar 25, 2015 at 2:03 PM, John Dewey john@dewey.ws wrote:

I faced this very issue in the past. We solved the problem by adding the
CA to the system bundle (as you stated). We also ran into problems where
python would still not validate the CA. However, this turned out to be a
permissions error with cacerts.txt[1] when httplib2 was installed through
pip. Nowadays openstack uses requests which I don’t believe utilizes
httplib2.

[1] https://code.google.com/p/httplib2/issues/detail?id=292&q=certificate

On Wednesday, March 25, 2015 at 11:13 AM, Jesse Keating wrote:

We're facing a bit of a frustration. In some of our environments, we're
using a self-signed certificate for our ssl termination (haproxy). We have
our various services pointing at the haproxy for service cross-talk, such
as nova to neutron or nova to glance or nova to cinder or neutron to nova
or cinder to glance or all the things to keystone. When using a self-signed
certificate, these services have trouble validating the cert when they
attempt to talk to each other. This problem can be solved in a few ways,
such as adding the CA to the system bundle (of your platform has such a
thing), adding the CA to the bundle python requests uses (because
hilariously it doesn't always use the system bundle), or the more direct
way of telling nova, neutron, et al the direct path to the CA file.

This last choice is the way we went forward, more explicit, and didn't
depend on knowledge if python-requests was using its own bundle or the
operating system's bundle. To configure this there are a few places that
need to be touched.

nova.conf:
[keystone_authtoken]
cafile =

[neutron]
cacertificatesfile =

[cinder]
cacertificatesfile =

(nothing for glance hilariously)

neutron.conf
[DEFAULT]
novacacertificates_file =

[keystone_authtoken]
cafile =

glance-api.conf and glance-registry.conf
[keystone_authtoken]
cafile =

cinder.conf
[DEFAULT]
glancecacertificates_file =

[keystone_authtoken]
cafile =

heat.conf
[clients]
ca_file =

[clients]
ca
file =

As you can see, there are a lot of places where one would have to define
the path, and the frustrating part is that the config name and section
varies across the services. Does anybody think this is a good thing? Can
anybody think of a good way forward to come to some sort of agreement on
config names? It does seem like heat is a winner here, it has a default
that can be defined for all clients, and then each client could potentially
point to a different path, but every config entry is named the same. Can we
do that across all the other services?

I chatted a bit on twitter last night with some nova folks, they suggested
starting a thread here on ops list and potentially turning it into a
hallway session or real session at the Vancouver design summit (which
operators are officially part of).


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Mar 25, 2015 by jlk_at_bluebox.net (2,760 points)   2 3
0 votes

Thanks for starting this thread Jesse. I agree that heat looks like a
good model for other projects to model themselves on here.

Can anyone think of a use case for having a per client / driver CA
file? I can't, but perhaps I'm missing something.

Michael

On Thu, Mar 26, 2015 at 5:13 AM, Jesse Keating jlk@bluebox.net wrote:
We're facing a bit of a frustration. In some of our environments, we're
using a self-signed certificate for our ssl termination (haproxy). We have
our various services pointing at the haproxy for service cross-talk, such as
nova to neutron or nova to glance or nova to cinder or neutron to nova or
cinder to glance or all the things to keystone. When using a self-signed
certificate, these services have trouble validating the cert when they
attempt to talk to each other. This problem can be solved in a few ways,
such as adding the CA to the system bundle (of your platform has such a
thing), adding the CA to the bundle python requests uses (because
hilariously it doesn't always use the system bundle), or the more direct way
of telling nova, neutron, et al the direct path to the CA file.

This last choice is the way we went forward, more explicit, and didn't
depend on knowledge if python-requests was using its own bundle or the
operating system's bundle. To configure this there are a few places that
need to be touched.

nova.conf:
[keystone_authtoken]
cafile =

[neutron]
cacertificatesfile =

[cinder]
cacertificatesfile =

(nothing for glance hilariously)

neutron.conf
[DEFAULT]
novacacertificates_file =

[keystone_authtoken]
cafile =

glance-api.conf and glance-registry.conf
[keystone_authtoken]
cafile =

cinder.conf
[DEFAULT]
glancecacertificates_file =

[keystone_authtoken]
cafile =

heat.conf
[clients]
ca_file =

[clients]
ca
file =

As you can see, there are a lot of places where one would have to define the
path, and the frustrating part is that the config name and section varies
across the services. Does anybody think this is a good thing? Can anybody
think of a good way forward to come to some sort of agreement on config
names? It does seem like heat is a winner here, it has a default that can be
defined for all clients, and then each client could potentially point to a
different path, but every config entry is named the same. Can we do that
across all the other services?

I chatted a bit on twitter last night with some nova folks, they suggested
starting a thread here on ops list and potentially turning it into a hallway
session or real session at the Vancouver design summit (which operators are
officially part of).

  • jlk


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Rackspace Australia


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Mar 25, 2015 by Michael_Still (16,180 points)   3 5 13
0 votes

I'll start by saying I went the system bundle route also and have thus far
had no issues with it. I'll also say that I'm using RDO packages still and
not doing anything with venvs or pip installed stuff.

On Wed, Mar 25, 2015 at 6:33 PM, Michael Still mikal@stillhq.com wrote:

Thanks for starting this thread Jesse. I agree that heat looks like a
good model for other projects to model themselves on here.

Can anyone think of a use case for having a per client / driver CA
file? I can't, but perhaps I'm missing something.

There could potentially be instances where one service would be running
certificates issued off of one internal CA and others on another, but
really I don't see the point of splitting them out when you can concatenate
the CA certificates together and feed it in as a bundle that covers
everything. This one section from Heat would cover everything I would
think.

[clients]
ca_file =

Michael

On Thu, Mar 26, 2015 at 5:13 AM, Jesse Keating jlk@bluebox.net wrote:

We're facing a bit of a frustration. In some of our environments, we're
using a self-signed certificate for our ssl termination (haproxy). We
have
our various services pointing at the haproxy for service cross-talk,
such as
nova to neutron or nova to glance or nova to cinder or neutron to nova or
cinder to glance or all the things to keystone. When using a self-signed
certificate, these services have trouble validating the cert when they
attempt to talk to each other. This problem can be solved in a few ways,
such as adding the CA to the system bundle (of your platform has such a
thing), adding the CA to the bundle python requests uses (because
hilariously it doesn't always use the system bundle), or the more direct
way
of telling nova, neutron, et al the direct path to the CA file.

This last choice is the way we went forward, more explicit, and didn't
depend on knowledge if python-requests was using its own bundle or the
operating system's bundle. To configure this there are a few places that
need to be touched.

nova.conf:
[keystone_authtoken]
cafile =

[neutron]
cacertificatesfile =

[cinder]
cacertificatesfile =

(nothing for glance hilariously)

neutron.conf
[DEFAULT]
novacacertificates_file =

[keystone_authtoken]
cafile =

glance-api.conf and glance-registry.conf
[keystone_authtoken]
cafile =

cinder.conf
[DEFAULT]
glancecacertificates_file =

[keystone_authtoken]
cafile =

heat.conf
[clients]
ca_file =

[clients]
ca
file =

As you can see, there are a lot of places where one would have to define
the
path, and the frustrating part is that the config name and section varies
across the services. Does anybody think this is a good thing? Can anybody
think of a good way forward to come to some sort of agreement on config
names? It does seem like heat is a winner here, it has a default that
can be
defined for all clients, and then each client could potentially point to
a
different path, but every config entry is named the same. Can we do that
across all the other services?

I chatted a bit on twitter last night with some nova folks, they
suggested
starting a thread here on ops list and potentially turning it into a
hallway
session or real session at the Vancouver design summit (which operators
are
officially part of).

  • jlk


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Rackspace Australia


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-Erik


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Mar 26, 2015 by Erik_McCormick (3,880 points)   2 4
0 votes

I have struggled with this stuff in the past, too. And have had situations where certs come from different CAs, so I can see some value in an ability to override the global defaults for specific services.

However, for the small fraction of use cases that would cover, I think the cert concatenation solution/work-around is sufficient. I can’t really think of why one couldn’t/wouldn’t do that.

But definitely +1 on consistent config option naming, at a minimum.

Mike

From: Erik McCormick
Date: Wednesday, March 25, 2015 at 9:36 PM
To: Michael Still
Cc: Jesse Keating, OpenStack Operators
Subject: Re: [Openstack-operators] OpenStack services and ca certificate config entries

I'll start by saying I went the system bundle route also and have thus far had no issues with it. I'll also say that I'm using RDO packages still and not doing anything with venvs or pip installed stuff.

On Wed, Mar 25, 2015 at 6:33 PM, Michael Still mikal@stillhq.com wrote:
Thanks for starting this thread Jesse. I agree that heat looks like a
good model for other projects to model themselves on here.

Can anyone think of a use case for having a per client / driver CA
file? I can't, but perhaps I'm missing something.

There could potentially be instances where one service would be running certificates issued off of one internal CA and others on another, but really I don't see the point of splitting them out when you can concatenate the CA certificates together and feed it in as a bundle that covers everything. This one section from Heat would cover everything I would think.

[clients]
ca_file =

Michael

On Thu, Mar 26, 2015 at 5:13 AM, Jesse Keating jlk@bluebox.net wrote:
We're facing a bit of a frustration. In some of our environments, we're
using a self-signed certificate for our ssl termination (haproxy). We have
our various services pointing at the haproxy for service cross-talk, such as
nova to neutron or nova to glance or nova to cinder or neutron to nova or
cinder to glance or all the things to keystone. When using a self-signed
certificate, these services have trouble validating the cert when they
attempt to talk to each other. This problem can be solved in a few ways,
such as adding the CA to the system bundle (of your platform has such a
thing), adding the CA to the bundle python requests uses (because
hilariously it doesn't always use the system bundle), or the more direct way
of telling nova, neutron, et al the direct path to the CA file.

This last choice is the way we went forward, more explicit, and didn't
depend on knowledge if python-requests was using its own bundle or the
operating system's bundle. To configure this there are a few places that
need to be touched.

nova.conf:
[keystone_authtoken]
cafile =

[neutron]
cacertificatesfile =

[cinder]
cacertificatesfile =

(nothing for glance hilariously)

neutron.conf
[DEFAULT]
novacacertificates_file =

[keystone_authtoken]
cafile =

glance-api.conf and glance-registry.conf
[keystone_authtoken]
cafile =

cinder.conf
[DEFAULT]
glancecacertificates_file =

[keystone_authtoken]
cafile =

heat.conf
[clients]
ca_file =

[clients]
ca
file =

As you can see, there are a lot of places where one would have to define the
path, and the frustrating part is that the config name and section varies
across the services. Does anybody think this is a good thing? Can anybody
think of a good way forward to come to some sort of agreement on config
names? It does seem like heat is a winner here, it has a default that can be
defined for all clients, and then each client could potentially point to a
different path, but every config entry is named the same. Can we do that
across all the other services?

I chatted a bit on twitter last night with some nova folks, they suggested
starting a thread here on ops list and potentially turning it into a hallway
session or real session at the Vancouver design summit (which operators are
officially part of).

  • jlk


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Rackspace Australia


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-Erik


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Mar 26, 2015 by Michael_Dorman (4,160 points)   5 11
0 votes

FYI -
http://lists.openstack.org/pipermail/openstack-dev/2014-June/037238.html
happened over the summer, which at least normalizes the names. I felt
like bits of this conversation were familiar. That's all in projects in
Kilo.

I do agree that single CA seems like a more sensible default than
currently needing multiple sections in the nova.conf (as an example).
That's basically a raw export of the complexity of the underlying code
and should be addressed.

On 03/26/2015 12:52 AM, Mike Dorman wrote:
I have struggled with this stuff in the past, too. And have had
situations where certs come from different CAs, so I can see some value
in an ability to override the global defaults for specific services.

However, for the small fraction of use cases that would cover, I think
the cert concatenation solution/work-around is sufficient. I can’t
really think of why one couldn’t/wouldn’t do that.

But definitely +1 on consistent config option naming, at a minimum.

Mike

From: Erik McCormick
Date: Wednesday, March 25, 2015 at 9:36 PM
To: Michael Still
Cc: Jesse Keating, OpenStack Operators
Subject: Re: [Openstack-operators] OpenStack services and ca certificate
config entries

I'll start by saying I went the system bundle route also and have thus
far had no issues with it. I'll also say that I'm using RDO packages
still and not doing anything with venvs or pip installed stuff.

On Wed, Mar 25, 2015 at 6:33 PM, Michael Still <mikal@stillhq.com
mikal@stillhq.com> wrote:

Thanks for starting this thread Jesse. I agree that heat looks like a
good model for other projects to model themselves on here.

Can anyone think of a use case for having a per client / driver CA
file? I can't, but perhaps I'm missing something.

There could potentially be instances where one service would be running
certificates issued off of one internal CA and others on another, but
really I don't see the point of splitting them out when you can
concatenate the CA certificates together and feed it in as a bundle that
covers everything. This one section from Heat would cover everything I
would think.

[clients]
ca_file =

Michael

On Thu, Mar 26, 2015 at 5:13 AM, Jesse Keating <jlk@bluebox.net
<mailto:jlk@bluebox.net>> wrote:
> We're facing a bit of a frustration. In some of our environments, we're
> using a self-signed certificate for our ssl termination (haproxy). We have
> our various services pointing at the haproxy for service cross-talk, such as
> nova to neutron or nova to glance or nova to cinder or neutron to nova or
> cinder to glance or all the things to keystone. When using a self-signed
> certificate, these services have trouble validating the cert when they
> attempt to talk to each other. This problem can be solved in a few ways,
> such as adding the CA to the system bundle (of your platform has such a
> thing), adding the CA to the bundle python requests uses (because
> hilariously it doesn't always use the system bundle), or the more direct way
> of telling nova, neutron, et al the direct path to the CA file.
>
> This last choice is the way we went forward, more explicit, and didn't
> depend on knowledge if python-requests was using its own bundle or the
> operating system's bundle. To configure this there are a few places that
> need to be touched.
>
> nova.conf:
> [keystone_authtoken]
> cafile = <path>
>
> [neutron]
> ca_certificates_file = <path>
>
> [cinder]
> ca_certificates_file = <path>
>
> (nothing for glance hilariously)
>
>
> neutron.conf
> [DEFAULT]
> nova_ca_certificates_file = <path>
>
> [keystone_authtoken]
> cafile = <path>
>
> glance-api.conf and glance-registry.conf
> [keystone_authtoken]
> cafile = <path>
>
> cinder.conf
> [DEFAULT]
> glance_ca_certificates_file = <path>
>
> [keystone_authtoken]
> cafile = <path>
>
> heat.conf
> [clients]
> ca_file = <path>
>
> [clients_<whatever>]
> ca_file = <path>
>
>
> As you can see, there are a lot of places where one would have to define the
> path, and the frustrating part is that the config name and section varies
> across the services. Does anybody think this is a good thing? Can anybody
> think of a good way forward to come to some sort of agreement on config
> names? It does seem like heat is a winner here, it has a default that can be
> defined for all clients, and then each client could potentially point to a
> different path, but every config entry is named the same. Can we do that
> across all the other services?
>
> I chatted a bit on twitter last night with some nova folks, they suggested
> starting a thread here on ops list and potentially turning it into a hallway
> session or real session at the Vancouver design summit (which operators are
> officially part of).
>
> - jlk
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators@lists.openstack.org
<mailto:OpenStack-operators@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>



--
Rackspace Australia

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
<mailto:OpenStack-operators@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-Erik


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Sean Dague
http://dague.net


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

responded Mar 26, 2015 by Sean_Dague (66,200 points)   4 8 14
...