settingsLogin | Registersettings

[openstack-dev] [oslo][nova] Anyone interested in writing a policy generator sphinx extension?

0 votes

Nova has policy defaults in code now and we can generate the sample
using oslopolicy-sample-generator but we'd like to get the default
policy sample in the Nova developer documentation also, like we have for
nova.conf.sample.

I see we use the sphinxconfiggen extension for building the
nova.conf.sample in our docs, but I don't see anything like that for
generating docs for a sample policy file.

Has anyone already started working on that, or is interested in working
on that? I've never written a sphinx extension before but I'm guessing
it could be borrowed a bit from how sphinxconfiggen was written in
oslo.config.

--

Thanks,

Matt Riedemann


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Sep 21, 2016 in openstack-dev by Matt_Riedemann (48,320 points)   3 10 25
retagged Jan 26, 2017 by admin

15 Responses

0 votes

Hi,

On Wed, Sep 21, 2016 at 10:49 AM, Matt Riedemann
mriedem@linux.vnet.ibm.com wrote:
Nova has policy defaults in code now and we can generate the sample using
oslopolicy-sample-generator but we'd like to get the default policy sample
in the Nova developer documentation also, like we have for nova.conf.sample.

I see we use the sphinxconfiggen extension for building the nova.conf.sample
in our docs, but I don't see anything like that for generating docs for a
sample policy file.

Has anyone already started working on that, or is interested in working on
that? I've never written a sphinx extension before but I'm guessing it could
be borrowed a bit from how sphinxconfiggen was written in oslo.config.

I'm not a developer. I'm one of the operator that asked about
documentation now that policy defaults got moved in code.
I'm happy to see a proper follow up on this request, I really appreciate it. +1
I hope this thread can stay on topic so a proper documentation
solution is found.

Thanks


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 22, 2016 by mgagne_at_calavera.c (1,700 points)   2 3
0 votes

Andrew,

the idea is to shift existing RBAC implementation:
currently policy is enforced in the service (Nova, for instance)
against the result of token validation, which is, in general, an access
check;
I'm thinking about performing policy enforcement along with access check
in a single operation and only if necessary -
not every operation is protected and requires token validation,
though now keystone middleware validates a token on every request.

AFAIK Nova is using some custom logic to change local policies at run-time,
so I assume there may be a benefit in dynamic centralized storage
managed via API,
so that Horizon can even provide a UI for that.

There are many questions in the matter, and my main is:
if we do RBAC in OpenStack the best way?

On 21.09.2016 20:16, Andrew Laski wrote:

On Wed, Sep 21, 2016, at 12:02 PM, Joshua Harlow wrote:

Andrew Laski wrote:

However, I have asked twice now on the review what the benefit of doing
this is and haven't received a response so I'll ask here. The proposal
would add additional latency to nearly every API operation in a service
and in return what do they get? Now that it's possible to register sane
policy defaults within a project most operators do not even need to
think about policy for projects that do that. And any policy changes
that are necessary are easily handled by a config management system.

I would expect to see a pretty significant benefit in exchange for
moving policy control out of Nova, and so far it's not clear to me what
that would be.
One way to do this is to setup something like etc.d or zookeeper and
have policy files be placed into certain 'keys' in there by keystone,
then consuming projects would 'watch' those keys for being changed (and
get notified when they are changed); the project would then reload its
policy when the other service (keystone) write a new key/policy.

https://coreos.com/etcd/docs/latest/api.html#waiting-for-a-change

or
https://zookeeper.apache.org/doc/r3.4.5/zookeeperProgrammers.html#ch_zkWatches

or (pretty sure consul has something similar),

This is pretty standard stuff folks :-/ and it's how afaik things like
https://github.com/skynetservices/skydns work (and more), and it would
avoid that 'additional latency' (unless the other service is adjusting
the policy key every millisecond, which seems sorta unreasonable).
Sure. Or have Keystone be a frontend for ansible/puppet/chef/.... What's
not clear to me in any of this is what's the benefit to having Keystone
as a fronted to policy configuration/changes, or be involved in any real
way with authorization decisions? What issue is being solved by getting
Keystone involved?

-Josh


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 22, 2016 by Alexander_V_Makarov (900 points)   1 2
0 votes

On Thu, Sep 22, 2016, at 01:34 PM, Alexander Makarov wrote:
Andrew,

the idea is to shift existing RBAC implementation:
currently policy is enforced in the service (Nova, for instance)
against the result of token validation, which is, in general, an access
check;
I'm thinking about performing policy enforcement along with access check
in a single operation and only if necessary -
not every operation is protected and requires token validation,
though now keystone middleware validates a token on every request.

There's a lot of information necessary to determine which policy checks
to run, and even more information necessary in order to run the policy
check.

For example: When booting an instance there are policies that are run
based on the data included in the POST body of the request. Should
Keystone be passed that POST body and contain logic to determine which
checks to run? Or would Nova pass a list of checks to Keystone and say
"do these policies pass?" I don't see an advantage to either of those.

Also, a lot of policy checks rely on data that must be looked up from
the Nova db. Checking policy for deletion of an instance requires
looking up that instance to check the context userid/projectid against
the instance userid/projectid. So Nova would need to do that lookup
and pass that information to Keystone to use for a policy check. Which
also means that Keystone can't do the check at the time of token
validation because at that point it doesn't have enough data for the
policy check.

AFAIK Nova is using some custom logic to change local policies at
run-time,
so I assume there may be a benefit in dynamic centralized storage
managed via API,
so that Horizon can even provide a UI for that.

There is no special mechanism in Nova for changing policies at run time.
Policy defaults are now embedded in the Nova code so it's possible to
run without a policy file, and it's possible to override those policies
with a policy file. But once Nova is running the only way to update
policies is to update the policy file. That works the same in every
project because the reload mechanism is handled automatically in
oslo.policy.

Furthermore, what's more interesting to Horizon I think is the question
of what is a user allowed to do with the API. And that question requires
more information than just policy to answer. An example is that it's
possible to resize an instance to a smaller size if the instance is on a
Xen hypervisor. So in order for Horizon to allow/disallow that option
for a user it needs information that Nova is in possession of, and
Keystone never will be. That's why the Nova community has been
discussing the idea of exposing "capabilities" in the Nova API which be
let a user know what they can do based on policy settings and many other
factors.

There are many questions in the matter, and my main is:
if we do RBAC in OpenStack the best way?

I'm sure we don't do RBAC in the best way. However, just like quota
enforcement, I think it's something that is going to need to be handled
in each project individually. But unlike quota administration I don't
see the benefit of centralizing policy administration. It's fairly
static data, and on the occasions that it needs to change it can be
updated with a configuration management solution and the live policy
reloading that oslo.policy has.

On 21.09.2016 20:16, Andrew Laski wrote:

On Wed, Sep 21, 2016, at 12:02 PM, Joshua Harlow wrote:

Andrew Laski wrote:

However, I have asked twice now on the review what the benefit of doing
this is and haven't received a response so I'll ask here. The proposal
would add additional latency to nearly every API operation in a service
and in return what do they get? Now that it's possible to register sane
policy defaults within a project most operators do not even need to
think about policy for projects that do that. And any policy changes
that are necessary are easily handled by a config management system.

I would expect to see a pretty significant benefit in exchange for
moving policy control out of Nova, and so far it's not clear to me what
that would be.
One way to do this is to setup something like etc.d or zookeeper and
have policy files be placed into certain 'keys' in there by keystone,
then consuming projects would 'watch' those keys for being changed (and
get notified when they are changed); the project would then reload its
policy when the other service (keystone) write a new key/policy.

https://coreos.com/etcd/docs/latest/api.html#waiting-for-a-change

or
https://zookeeper.apache.org/doc/r3.4.5/zookeeperProgrammers.html#ch_zkWatches

or (pretty sure consul has something similar),

This is pretty standard stuff folks :-/ and it's how afaik things like
https://github.com/skynetservices/skydns work (and more), and it would
avoid that 'additional latency' (unless the other service is adjusting
the policy key every millisecond, which seems sorta unreasonable).
Sure. Or have Keystone be a frontend for ansible/puppet/chef/.... What's
not clear to me in any of this is what's the benefit to having Keystone
as a fronted to policy configuration/changes, or be involved in any real
way with authorization decisions? What issue is being solved by getting
Keystone involved?

-Josh


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 23, 2016 by andrew_at_lascii.com (6,820 points)   1 2 5
0 votes

On Wed, 2016-09-21 at 11:16 -0400, Doug Hellmann wrote:
Excerpts from Matt Riedemann's message of 2016-09-21 09:49:29 -0500:

Nova has policy defaults in code now and we can generate the
sample 
using oslopolicy-sample-generator but we'd like to get the default 
policy sample in the Nova developer documentation also, like we
have for 
nova.conf.sample.

I see we use the sphinxconfiggen extension for building the 
nova.conf.sample in our docs, but I don't see anything like that
for 
generating docs for a sample policy file.

Has anyone already started working on that, or is interested in
working 
on that? I've never written a sphinx extension before but I'm
guessing 
it could be borrowed a bit from how sphinxconfiggen was written in 
oslo.config.

I don't have time to do it myself, but I can help get someone else
started and work with them on code reviews in oslo.policy.

I can take a look at this, though I'll only persue a "sphinxpolicygen"
module for now (we don't use oslo_config's "sphinxext" module yet).

Stephen


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 26, 2016 by Stephen_Finucane (1,620 points)   5
0 votes

On 9/26/2016 8:48 AM, Stephen Finucane wrote:
On Wed, 2016-09-21 at 11:16 -0400, Doug Hellmann wrote:

Excerpts from Matt Riedemann's message of 2016-09-21 09:49:29 -0500:

Nova has policy defaults in code now and we can generate the
sample
using oslopolicy-sample-generator but we'd like to get the default
policy sample in the Nova developer documentation also, like we
have for
nova.conf.sample.

I see we use the sphinxconfiggen extension for building the
nova.conf.sample in our docs, but I don't see anything like that
for
generating docs for a sample policy file.

Has anyone already started working on that, or is interested in
working
on that? I've never written a sphinx extension before but I'm
guessing
it could be borrowed a bit from how sphinxconfiggen was written in
oslo.config.

I don't have time to do it myself, but I can help get someone else
started and work with them on code reviews in oslo.policy.

I can take a look at this, though I'll only persue a "sphinxpolicygen"
module for now (we don't use oslo_config's "sphinxext" module yet).

Stephen


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Just to circle back on progress here, Stephen got things done and
results can be seen here:

http://docs.openstack.org/developer/nova/sample_policy.html

--

Thanks,

Matt Riedemann


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 23, 2016 by Matt_Riedemann (48,320 points)   3 10 25
...