settingsLogin | Registersettings

[openstack-dev] [python-openstackclient][python-openstacksdk][neutron][nova] supporting resource extensions with our CLI

0 votes

I think we have a gap in our OSC CLI for non-stadium plugin/driver
projects (neutron plugin projects, nova driver projects) that implement
RESTful resource API attribute extensions.

For details, go directly to [1].
For a summary read on...

For OpenStack APIs that support extensions (ex [2]), the classic
python-client CLIs worked "out of the box" for extensions
adding attributes to existing RESTful resources.

For example, your project has a neutron plugin that adds a 'my_bool'
boolean attribute to 'network' resources that can be set via POST/PUT
and is returned with GET. This just works with the python-neutronclient
CLI without any client-side code changes.

However, with OSC resource attributes must be added directly/statically
to the sdk's resource and then consumed in the client; the support does
not come "for free" in the CLI. While this is fine for stadium projects
(they can contribute directly to the sdk/client), non-stadium projects
have no viable option to plugin/extend the CLI today for this type of
API extension mechanism.

With the phasing out of the python clients, a number of our users will
be left without a CLI to interface with these extensions.

I'd like to try and close this gap in Queens and welcome discussion in [1].

Thanks

[1] https://bugs.launchpad.net/python-openstacksdk/+bug/1705755
[2] https://wiki.openstack.org/wiki/NeutronDevelopment#API_Extensions


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Aug 10, 2017 in openstack-dev by Boden_Russell (1,780 points)   2 5

8 Responses

0 votes

On 8/3/2017 1:39 PM, Boden Russell wrote:
I think we have a gap in our OSC CLI for non-stadium plugin/driver
projects (neutron plugin projects, nova driver projects) that implement
RESTful resource API attribute extensions.

For details, go directly to [1].
For a summary read on...

For OpenStack APIs that support extensions (ex [2]), the classic
python-client CLIs worked "out of the box" for extensions
adding attributes to existing RESTful resources.

For example, your project has a neutron plugin that adds a 'my_bool'
boolean attribute to 'network' resources that can be set via POST/PUT
and is returned with GET. This just works with the python-neutronclient
CLI without any client-side code changes.

However, with OSC resource attributes must be added directly/statically
to the sdk's resource and then consumed in the client; the support does
not come "for free" in the CLI. While this is fine for stadium projects
(they can contribute directly to the sdk/client), non-stadium projects
have no viable option to plugin/extend the CLI today for this type of
API extension mechanism.

With the phasing out of the python clients, a number of our users will
be left without a CLI to interface with these extensions.

I'd like to try and close this gap in Queens and welcome discussion in [1].

Thanks

[1] https://bugs.launchpad.net/python-openstacksdk/+bug/1705755
[2] https://wiki.openstack.org/wiki/NeutronDevelopment#API_Extensions


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

There is nothing like this for Nova so I'm not sure why Nova should be
involved here. We dropped all support for extending the API via
stevedore extension loading in Pike [1]. The virt drivers don't extend
the REST API either.

[1] https://blueprints.launchpad.net/nova/+spec/api-no-more-extensions-pike

--

Thanks,

Matt


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 7, 2017 by mriedemos_at_gmail.c (15,720 points)   2 4 5
0 votes

Excerpts from Matt Riedemann's message of 2017-08-07 14:33:22 -0500:

On 8/3/2017 1:39 PM, Boden Russell wrote:

I think we have a gap in our OSC CLI for non-stadium plugin/driver
projects (neutron plugin projects, nova driver projects) that implement
RESTful resource API attribute extensions.

For details, go directly to [1].
For a summary read on...

For OpenStack APIs that support extensions (ex [2]), the classic
python-client CLIs worked "out of the box" for extensions
adding attributes to existing RESTful resources.

For example, your project has a neutron plugin that adds a 'my_bool'
boolean attribute to 'network' resources that can be set via POST/PUT
and is returned with GET. This just works with the python-neutronclient
CLI without any client-side code changes.

However, with OSC resource attributes must be added directly/statically
to the sdk's resource and then consumed in the client; the support does
not come "for free" in the CLI. While this is fine for stadium projects
(they can contribute directly to the sdk/client), non-stadium projects
have no viable option to plugin/extend the CLI today for this type of
API extension mechanism.

With the phasing out of the python clients, a number of our users will
be left without a CLI to interface with these extensions.

I'd like to try and close this gap in Queens and welcome discussion in [1].

Thanks

[1] https://bugs.launchpad.net/python-openstacksdk/+bug/1705755
[2] https://wiki.openstack.org/wiki/NeutronDevelopment#API_Extensions


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

There is nothing like this for Nova so I'm not sure why Nova should be
involved here. We dropped all support for extending the API via
stevedore extension loading in Pike [1]. The virt drivers don't extend
the REST API either.

[1] https://blueprints.launchpad.net/nova/+spec/api-no-more-extensions-pike

And there was much rejoicing.

If the thing you're doing doesn't fit in the mainline API, then what
you're doing is making a new API. Extensions just bypass the important
part where that API gets designed and thought through.


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2017 by Clint_Byrum (40,940 points)   4 5 9
0 votes

On 08/08/2017 12:39 AM, Clint Byrum wrote:
Excerpts from Matt Riedemann's message of 2017-08-07 14:33:22 -0500:

There is nothing like this for Nova so I'm not sure why Nova should be
involved here. We dropped all support for extending the API via
stevedore extension loading in Pike [1]. The virt drivers don't extend
the REST API either.

[1] https://blueprints.launchpad.net/nova/+spec/api-no-more-extensions-pike

And there was much rejoicing.

If the thing you're doing doesn't fit in the mainline API, then what
you're doing is making a new API. Extensions just bypass the important
part where that API gets designed and thought through.

100 times this.

-jay


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2017 by Jay_Pipes (59,760 points)   3 10 13
0 votes

On 8/7/17 10:39 PM, Clint Byrum wrote:
If the thing you're doing doesn't fit in the mainline API, then what
you're doing is making a new API. Extensions just bypass the important
part where that API gets designed and thought through.

Irrespective of opinions as to if API extensions are good or not, the
fact of the matter is we support them in neutron today and as a result
we have users that rely on them as well as the ability to interface
(CLI) with such extensions via python-neutronclient. That said, I think
this concern has been heard [1] and we will work to address it
short-to-mid term.

As to neutron's longer-term goals W/R/T API extensions; I can't speak to
that.

[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-August/120759.html


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2017 by Boden_Russell (1,780 points)   2 5
0 votes

2017-08-08 22:41 GMT+09:00 Boden Russell bodenvmw@gmail.com:

On 8/7/17 10:39 PM, Clint Byrum wrote:

If the thing you're doing doesn't fit in the mainline API, then what
you're doing is making a new API. Extensions just bypass the important
part where that API gets designed and thought through.

Irrespective of opinions as to if API extensions are good or not, the
fact of the matter is we support them in neutron today and as a result
we have users that rely on them as well as the ability to interface
(CLI) with such extensions via python-neutronclient. That said, I think
this concern has been heard [1] and we will work to address it
short-to-mid term.

I totally agree with other comments that all API features should be upstreamed.

Replying to Boden's question on the short-term solution.
If you need CLI support, you can implement OSC plugin to support your
API extensions
rather than extending OSC or OSC plugin provided by python-neutronclient.
If you need SDK support, you can provide your own python bindings
(perhaps it will be most stable)
or continue to use the python-neutronclient CLI extension mechanism
(which extends methods
based on "neutronclient.extension" entry points.

Does it answer to you, Boden?

Akihiro

As to neutron's longer-term goals W/R/T API extensions; I can't speak to
that.

[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-August/120759.html


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2017 by Akihiro_Motoki (8,520 points)   2 3 4
0 votes

2017-08-08 23:12 GMT+09:00 Akihiro Motoki amotoki@gmail.com:

2017-08-08 22:41 GMT+09:00 Boden Russell bodenvmw@gmail.com:

On 8/7/17 10:39 PM, Clint Byrum wrote:

If the thing you're doing doesn't fit in the mainline API, then what
you're doing is making a new API. Extensions just bypass the important
part where that API gets designed and thought through.

Irrespective of opinions as to if API extensions are good or not, the
fact of the matter is we support them in neutron today and as a result
we have users that rely on them as well as the ability to interface
(CLI) with such extensions via python-neutronclient. That said, I think
this concern has been heard [1] and we will work to address it
short-to-mid term.

I totally agree with other comments that all API features should be upstreamed.

Replying to Boden's question on the short-term solution.
If you need CLI support, you can implement OSC plugin to support your
API extensions
rather than extending OSC or OSC plugin provided by python-neutronclient.
If you need SDK support, you can provide your own python bindings
(perhaps it will be most stable)
or continue to use the python-neutronclient CLI extension mechanism
(which extends methods
based on "neutronclient.extension" entry points.

Does it answer to you, Boden?

My above reply does not answer the original question.
The subject of the thread says "resource extension" but this is
actually on attribute extension.
My reply applies to 'resource' extension but does not apply to
'attribute extension'

Akihiro

As to neutron's longer-term goals W/R/T API extensions; I can't speak to
that.

[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-August/120759.html


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2017 by Akihiro_Motoki (8,520 points)   2 3 4
0 votes

On 8/8/17 10:29 AM, Akihiro Motoki wrote:
My reply applies to 'resource' extension but does not apply to
'attribute extension'

My apologies for using confusing terminology; as you pointed out we
don't currently have a good solution for attribute extensions with OSC
and I've tried to outline the technicals in the RFE style bug [1] where
I hope we can continue this discussion.

python-neutronclient does support these attribute extensions, but as of
today neutronclient gives deprecation warnings when used; so users are
"concerned" when they see this and we don't have a complete story for
OSC. Perhaps we can suppress those deprecations until we figure out a
path forward?

Thanks

[1] https://bugs.launchpad.net/neutron/+bug/1705755


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2017 by Boden_Russell (1,780 points)   2 5
0 votes

I think this discussion has several aspects from general direction to
implementation details.

TL;DR
My suggestion as a main maintainer of python-neutronclient is to
upstream all out-of-tree APIs, support all features in OSC and push
away 'neutron' CLI.

The long version;

In my basic understanding, OpenStack in general has chosen the way
that we don't accept vendor extensions in API from the POV of inter
operability. neutron still supports to load out-of-tree API
definitions but I think neutron is just behind it. Nova drops
out-of-tree extension support in the API in Pike as Matt mentioned,
and if we move the (micro-)versioning forward out-of-tree API
definition will have more difficulties. IMHO I would like to see all
APIs are upstreamed as others commented. I believe this point is most
important and we all needs to be in a same page. Hopefully all
out-of-tree APIs will be upstreamed.

My understanding is the current OSC and openstacksdk (in addition to
shade) adopts this position.

On the other hand, we also need to take care of existing users on the
'neutron' CLI side.
I think there are several options on python-neutronclient.
(a) keep the current deprecation and recommend out-of-tree API to be upstreamed
(b) no deprecation warning from neutron CLI but keep the deprecation
policy on 'neutron' CLI (which means no new feature and positive
maintenance is provided)
(c) changing the current deprecation policy and continue to add new
features to 'neutron' CLI

I think (a) or (b) is realistic.
(c) needs more developers and reviewers. (I personally supports OSC,
openstacksdk and shade.)

The deprecation means no features will be coming for 'neutron' CLI and
all new features are suggested to go to OSC. I believe this is the
future we agreed. We can keep 'neutron' CLI only for existing
features, even though no neutron (and stadium projects) need to
implement new features in 'neutron' CLI. neutron CLI will be gone even
if there is no deprecation notice. At some point, we can split out
CLI part of neutronclient to a hosted project if someone wants to
keep 'neutron' CLI.

There is another background which makes neutron CLI difficult to keep
backward compatibility. The neutron option parsing is ugly enough and
it is hard to be maintained. It breaks backward compatibility so
easily and prevents from changing option formats or introducing new
options which are considered better. Thus, as a main maintainer of
python-neutronclient, I would like to push away this feature
completely, but perhaps this is the feature which out-of-tree API
would like to leverage. Does anyone want to support the extra argument
mechanism in neutron CLI ([1] and [2] for more detail)?

[1] https://docs.openstack.org/python-neutronclient/latest/cli/neutron.html#extra-arguments-for-create-update-operation
[2] https://docs.openstack.org/python-neutronclient/latest/contributor/cli_option_guideline.html

Thanks,
Akihiro

2017-08-09 2:46 GMT+09:00 Boden Russell bodenvmw@gmail.com:

On 8/8/17 10:29 AM, Akihiro Motoki wrote:

My reply applies to 'resource' extension but does not apply to
'attribute extension'

My apologies for using confusing terminology; as you pointed out we
don't currently have a good solution for attribute extensions with OSC
and I've tried to outline the technicals in the RFE style bug [1] where
I hope we can continue this discussion.

python-neutronclient does support these attribute extensions, but as of
today neutronclient gives deprecation warnings when used; so users are
"concerned" when they see this and we don't have a complete story for
OSC. Perhaps we can suppress those deprecations until we figure out a
path forward?

Thanks

[1] https://bugs.launchpad.net/neutron/+bug/1705755


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 10, 2017 by Akihiro_Motoki (8,520 points)   2 3 4
...