settingsLogin | Registersettings

[Openstack-operators] [keystone] Removing functionality that was deprecated in Kilo and upcoming deprecated functionality in Mitaka

0 votes

This post is being sent again to the operators mailing list, and i
apologize if it's duplicated for some folks. The original thread is here:
http://lists.openstack.org/pipermail/openstack-dev/2015-November/080816.html

In the Mitaka release, the keystone team will be removing functionality
that was marked for deprecation in Kilo, and marking certain functions as
deprecated in Mitaka (that may be removed in at least 2 cycles).

removing deprecated functionality
=====================

This is not a full list, but these are by and large the most contentious
topics.

  • Eventlet support: This was marked as deprecated back in Kilo and is
    currently scheduled to be removed in Mitaka in favor of running keystone in
    a WSGI server. This is currently how we test keystone in the gate, and
    based on the feedback we received at the summit, a lot of folks have moved
    to running keystone under Apache since we’ve announced this change.
    OpenStack's CI is configured to mainly test using this deployment model.
    See [0] for when we started to issue warnings.

  • Using LDAP to store assignment data: Like eventlet support, this feature
    was also deprecated in Kilo and scheduled to be removed in Mitaka. To store
    assignment data (role assignments) we suggest using an SQL based backend
    rather than LDAP. See [1] for when we started to issue warnings.

  • Using LDAP to store project and domain data: The same as above, see [2]
    for when we started to issue warnings.

  • for a complete list:
    https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka

functions deprecated as of mitaka
=====================

The following will adhere to the TC’s new standard on deprecating
functionality [3].

  • LDAP write support for identity: We suggest simply not writing to LDAP
    for users and groups, this effectively makes create, delete and update of
    LDAP users and groups a no-op. It will be removed in the O release.

  • PKI tokens: We suggest using UUID or fernet tokens instead. The PKI token
    format has had issues with security and causes problems with both horizon
    and swift when the token contains an excessively large service catalog. It
    will be removed in the O release.

  • v2.0 of our API: Lastly, the keystone team recommends using v3 of our
    Identity API. We have had the intention of deprecating v2.0 for a while
    (since Juno actually), and have finally decided to formally deprecate v2.0.
    OpenStack’s CI runs successful v3 only jobs, there is complete feature
    parity with v2.0, and we feel the CLI exposed via openstackclient is mature
    enough to say with certainty that we can deprecate v2.0. It will be around
    for at least FOUR releases, with the authentication routes
    (POST /auth/tokens) potentially sticking around for longer.

  • for a complete list:
    https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka

If you have ANY concern about the following, please speak up now and let us
know!

Thanks!

Steve Martinelli
OpenStack Keystone Project Team Lead

[0]
https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80
[1]
https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34
[2]
https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39

[3]
http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
asked Nov 30, 2015 in openstack-operators by Steve_Martinelli (6,500 points)   1 3 5

19 Responses

0 votes

I have an objection to eventlet going away. We have problems with running
Apache and modwsgi with multiple python virtual environments. In some of
our stacks we're running both Horizon and Keystone. Each get their own
virtual environment. Apache mod
wsgi doesn't really work that way, so we'd
have to do some ugly hacks to expose the python environments of both to
Apache at the same time.

I believe we spoke about this at Summit. Have you had time to look into
this scenario and have suggestions?

  • jlk

On Mon, Nov 30, 2015 at 10:26 AM, Steve Martinelli stevemar@ca.ibm.com
wrote:

This post is being sent again to the operators mailing list, and i
apologize if it's duplicated for some folks. The original thread is here:
http://lists.openstack.org/pipermail/openstack-dev/2015-November/080816.html

In the Mitaka release, the keystone team will be removing functionality
that was marked for deprecation in Kilo, and marking certain functions as
deprecated in Mitaka (that may be removed in at least 2 cycles).

removing deprecated functionality
=====================

This is not a full list, but these are by and large the most contentious
topics.

  • Eventlet support: This was marked as deprecated back in Kilo and is
    currently scheduled to be removed in Mitaka in favor of running keystone in
    a WSGI server. This is currently how we test keystone in the gate, and
    based on the feedback we received at the summit, a lot of folks have moved
    to running keystone under Apache since we’ve announced this change.
    OpenStack's CI is configured to mainly test using this deployment model.
    See [0] for when we started to issue warnings.

  • Using LDAP to store assignment data: Like eventlet support, this feature
    was also deprecated in Kilo and scheduled to be removed in Mitaka. To store
    assignment data (role assignments) we suggest using an SQL based backend
    rather than LDAP. See [1] for when we started to issue warnings.

  • Using LDAP to store project and domain data: The same as above, see [2]
    for when we started to issue warnings.

  • for a complete list:
    https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka

functions deprecated as of mitaka
=====================

The following will adhere to the TC’s new standard on deprecating
functionality [3].

  • LDAP write support for identity: We suggest simply not writing to LDAP
    for users and groups, this effectively makes create, delete and update of
    LDAP users and groups a no-op. It will be removed in the O release.

  • PKI tokens: We suggest using UUID or fernet tokens instead. The PKI
    token format has had issues with security and causes problems with both
    horizon and swift when the token contains an excessively large service
    catalog. It will be removed in the O release.

  • v2.0 of our API: Lastly, the keystone team recommends using v3 of our
    Identity API. We have had the intention of deprecating v2.0 for a while
    (since Juno actually), and have finally decided to formally deprecate v2.0.
    OpenStack’s CI runs successful v3 only jobs, there is complete feature
    parity with v2.0, and we feel the CLI exposed via openstackclient is mature
    enough to say with certainty that we can deprecate v2.0. It will be around
    for at least FOUR releases, with the authentication routes (POST
    /auth/tokens) potentially sticking around for longer.

  • for a complete list:
    https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka

If you have ANY concern about the following, please speak up now and let
us know!

Thanks!

Steve Martinelli
OpenStack Keystone Project Team Lead

[0]
https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80
[1]
https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34
[2]
https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39
[3]
http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 30, 2015 by jlk_at_bluebox.net (2,760 points)   2 3
0 votes

On 1 December 2015 at 09:36, Jesse Keating jlk@bluebox.net wrote:
I have an objection to eventlet going away. We have problems with running
Apache and modwsgi with multiple python virtual environments. In some of
our stacks we're running both Horizon and Keystone. Each get their own
virtual environment. Apache mod
wsgi doesn't really work that way, so we'd
have to do some ugly hacks to expose the python environments of both to
Apache at the same time.

I believe we spoke about this at Summit. Have you had time to look into this
scenario and have suggestions?

modwsgi with separate process definitions should work - Graham
Dumpleton, the mod
wsgi maintainer, works for Red Hat - I'm fairly
sure they'd be amenable to us roping him in to help you sort things
out :)

-Rob


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 30, 2015 by Robert_Collins (27,200 points)   4 6 11
0 votes

I don't have a problem with eventlet itself going away, but I do feel that
keystone should pick a python based web server capable of running WSGI apps
( such as uWSGI ) for the reference implementation rather than Apache which
can be declared appropriately in the requirements.txt of the project. I
feel it is important to allow the operator to make choices based on their
organization's skill sets ( i.e. apache vs nginx ) to help keep complexity
low.

I understand there are some newer features that rely on Apache (
federation, etc ) but we should allow the need for those features inform
the operators choice of web server rather than force it for everybody.

Having a default implementation using uWSGI is also more inline with the 12
factor way of writing applications and will run a lot more comfortably in
[application] containers than apache would which is probably an important
consideration given how many people are focused on being able to run
openstack projects inside containers.

On Mon, Nov 30, 2015 at 2:36 PM, Jesse Keating jlk@bluebox.net wrote:

I have an objection to eventlet going away. We have problems with running
Apache and modwsgi with multiple python virtual environments. In some of
our stacks we're running both Horizon and Keystone. Each get their own
virtual environment. Apache mod
wsgi doesn't really work that way, so we'd
have to do some ugly hacks to expose the python environments of both to
Apache at the same time.

I believe we spoke about this at Summit. Have you had time to look into
this scenario and have suggestions?

  • jlk

On Mon, Nov 30, 2015 at 10:26 AM, Steve Martinelli stevemar@ca.ibm.com
wrote:

This post is being sent again to the operators mailing list, and i
apologize if it's duplicated for some folks. The original thread is here:
http://lists.openstack.org/pipermail/openstack-dev/2015-November/080816.html

In the Mitaka release, the keystone team will be removing functionality
that was marked for deprecation in Kilo, and marking certain functions as
deprecated in Mitaka (that may be removed in at least 2 cycles).

removing deprecated functionality
=====================

This is not a full list, but these are by and large the most contentious
topics.

  • Eventlet support: This was marked as deprecated back in Kilo and is
    currently scheduled to be removed in Mitaka in favor of running keystone in
    a WSGI server. This is currently how we test keystone in the gate, and
    based on the feedback we received at the summit, a lot of folks have moved
    to running keystone under Apache since we’ve announced this change.
    OpenStack's CI is configured to mainly test using this deployment model.
    See [0] for when we started to issue warnings.

  • Using LDAP to store assignment data: Like eventlet support, this
    feature was also deprecated in Kilo and scheduled to be removed in Mitaka.
    To store assignment data (role assignments) we suggest using an SQL based
    backend rather than LDAP. See [1] for when we started to issue warnings.

  • Using LDAP to store project and domain data: The same as above, see [2]
    for when we started to issue warnings.

  • for a complete list:
    https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka

functions deprecated as of mitaka
=====================

The following will adhere to the TC’s new standard on deprecating
functionality [3].

  • LDAP write support for identity: We suggest simply not writing to LDAP
    for users and groups, this effectively makes create, delete and update of
    LDAP users and groups a no-op. It will be removed in the O release.

  • PKI tokens: We suggest using UUID or fernet tokens instead. The PKI
    token format has had issues with security and causes problems with both
    horizon and swift when the token contains an excessively large service
    catalog. It will be removed in the O release.

  • v2.0 of our API: Lastly, the keystone team recommends using v3 of our
    Identity API. We have had the intention of deprecating v2.0 for a while
    (since Juno actually), and have finally decided to formally deprecate v2.0.
    OpenStack’s CI runs successful v3 only jobs, there is complete feature
    parity with v2.0, and we feel the CLI exposed via openstackclient is mature
    enough to say with certainty that we can deprecate v2.0. It will be around
    for at least FOUR releases, with the authentication routes (POST
    /auth/tokens) potentially sticking around for longer.

  • for a complete list:
    https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka

If you have ANY concern about the following, please speak up now and let
us know!

Thanks!

Steve Martinelli
OpenStack Keystone Project Team Lead

[0]
https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80
[1]
https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34
[2]
https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39
[3]
http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 30, 2015 by pczarkowski_plus_ope (300 points)   1
0 votes

modwsgi is compiled to a specific python for Apache. It's a single process
(with multiple forks). It looks like the upstream solution to running
different pythons with mod
wsgi is to either use modwsgi-express to launch
a new apache+mod
wsgi per-application, or to modify wsgi script (
https://github.com/openstack/keystone/blob/master/keystone/server/wsgi.py )
to load a virtualenv at the top with execfile()

Neither of these are very good options.

  • jlk

On Mon, Nov 30, 2015 at 12:41 PM, Robert Collins robertc@robertcollins.net
wrote:

On 1 December 2015 at 09:36, Jesse Keating jlk@bluebox.net wrote:

I have an objection to eventlet going away. We have problems with running
Apache and modwsgi with multiple python virtual environments. In some of
our stacks we're running both Horizon and Keystone. Each get their own
virtual environment. Apache mod
wsgi doesn't really work that way, so
we'd
have to do some ugly hacks to expose the python environments of both to
Apache at the same time.

I believe we spoke about this at Summit. Have you had time to look into
this
scenario and have suggestions?

modwsgi with separate process definitions should work - Graham
Dumpleton, the mod
wsgi maintainer, works for Red Hat - I'm fairly
sure they'd be amenable to us roping him in to help you sort things
out :)

-Rob


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 30, 2015 by jlk_at_bluebox.net (2,760 points)   2 3
0 votes

100% agree.

We should look at uwsgi as the reference architecture.  Nginx/Apache/etc should be interchangeable, and up to the operator which they choose to use.  Hell, with tcp load balancing now in opensource Nginx, I could get rid of Apache and HAProxy by utilizing uwsgi.

John
On November 30, 2015 at 1:05:26 PM, Paul Czarkowski (pczarkowski+openstackops@bluebox.net) wrote:

I don't have a problem with eventlet itself going away, but I do feel that keystone should pick a python based web server capable of running WSGI apps ( such as uWSGI ) for the reference implementation rather than Apache which can be declared appropriately in the requirements.txt of the project.   I feel it is important to allow the operator to make choices based on their organization's skill sets ( i.e. apache vs nginx ) to help keep complexity low.

I understand there are some newer features that rely on Apache ( federation, etc )  but we should allow the need for those features inform the operators choice of web server rather than force it for everybody.

Having a default implementation using uWSGI is also more inline with the 12 factor way of writing applications and will run a lot more comfortably in [application] containers than apache would which is probably an important consideration given how many people are focused on being able to run openstack projects inside containers.

On Mon, Nov 30, 2015 at 2:36 PM, Jesse Keating jlk@bluebox.net wrote:
I have an objection to eventlet going away. We have problems with running Apache and modwsgi with multiple python virtual environments. In some of our stacks we're running both Horizon and Keystone. Each get their own virtual environment. Apache modwsgi doesn't really work that way, so we'd have to do some ugly hacks to expose the python environments of both to Apache at the same time.

I believe we spoke about this at Summit. Have you had time to look into this scenario and have suggestions?

  • jlk

On Mon, Nov 30, 2015 at 10:26 AM, Steve Martinelli stevemar@ca.ibm.com wrote:
This post is being sent again to the operators mailing list, and i apologize if it's duplicated for some folks. The original thread is here: http://lists.openstack.org/pipermail/openstack-dev/2015-November/080816.html

In the Mitaka release, the keystone team will be removing functionality that was marked for deprecation in Kilo, and marking certain functions as deprecated in Mitaka (that may be removed in at least 2 cycles).

removing deprecated functionality
=====================

This is not a full list, but these are by and large the most contentious topics.

  • Eventlet support: This was marked as deprecated back in Kilo and is currently scheduled to be removed in Mitaka in favor of running keystone in a WSGI server. This is currently how we test keystone in the gate, and based on the feedback we received at the summit, a lot of folks have moved to running keystone under Apache since we’ve announced this change. OpenStack's CI is configured to mainly test using this deployment model. See [0] for when we started to issue warnings.

  • Using LDAP to store assignment data: Like eventlet support, this feature was also deprecated in Kilo and scheduled to be removed in Mitaka. To store assignment data (role assignments) we suggest using an SQL based backend rather than LDAP. See [1] for when we started to issue warnings.

  • Using LDAP to store project and domain data: The same as above, see [2] for when we started to issue warnings.

  • for a complete list: https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka

functions deprecated as of mitaka
=====================

The following will adhere to the TC’s new standard on deprecating functionality [3].

  • LDAP write support for identity: We suggest simply not writing to LDAP for users and groups, this effectively makes create, delete and update of LDAP users and groups a no-op. It will be removed in the O release.

  • PKI tokens: We suggest using UUID or fernet tokens instead. The PKI token format has had issues with security and causes problems with both horizon and swift when the token contains an excessively large service catalog. It will be removed in the O release.

  • v2.0 of our API: Lastly, the keystone team recommends using v3 of our Identity API. We have had the intention of deprecating v2.0 for a while (since Juno actually), and have finally decided to formally deprecate v2.0. OpenStack’s CI runs successful v3 only jobs, there is complete feature parity with v2.0, and we feel the CLI exposed via openstackclient is mature enough to say with certainty that we can deprecate v2.0. It will be around for at least FOUR releases, with the authentication routes (POST /auth/tokens) potentially sticking around for longer.

  • for a complete list: https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka

If you have ANY concern about the following, please speak up now and let us know!

Thanks!

Steve Martinelli
OpenStack Keystone Project Team Lead

[0] https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80
[1] https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34
[2] https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39
[3] http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Dec 1, 2015 by John_Dewey (1,320 points)   2
0 votes

Trying to summarize here...

  • There isn't much interest in keeping eventlet around.
  • Folks are OK with running keystone in a WSGI server, but feel they are
    constrained by Apache.
  • uWSGI could help to support multiple web servers.

My opinion:

  • Adding support for uWSGI definitely sounds like it's worth
    investigating, but not achievable in this release (unless someone already
    has something cooked up).
  • I'm tempted to let eventlet stick around another release, since it's
    causing pain on some of our operators.
  • Other folks have managed to run keystone in a web server (and hopefully
    not feel pain when doing so!), so it might be worth getting technical
    details on just how it was accomplished. If we get an OK from the operator
    community later on in mitaka, I'd still be OK with removing eventlet, but I
    don't want to break folks.

stevemar

From: John Dewey john@dewey.ws
100% agree.

We should look at uwsgi as the reference architecture. Nginx/Apache/etc
should be interchangeable, and up to the operator which they choose to use.
Hell, with tcp load balancing now in opensource Nginx, I could get rid of
Apache and HAProxy by utilizing uwsgi.

John

On November 30, 2015 at 1:05:26 PM, Paul Czarkowski (pczarkowski
+openstackops@bluebox.net) wrote:

  I don't have a problem with eventlet itself going away, but I do feel
  that keystone should pick a python based web server capable of
  running WSGI apps ( such as uWSGI ) for the reference implementation
  rather than Apache which can be declared appropriately in the
  requirements.txt of the project.   I feel it is important to allow
  the operator to make choices based on their organization's skill sets
  ( i.e. apache vs nginx ) to help keep complexity low.

  I understand there are some newer features that rely on Apache
  ( federation, etc )  but we should allow the need for those features
  inform the operators choice of web server rather than force it for
  everybody.

  Having a default implementation using uWSGI is also more inline with
  the 12 factor way of writing applications and will run a lot more
  comfortably in [application] containers than apache would which is
  probably an important consideration given how many people are focused
  on being able to run openstack projects inside containers.

  On Mon, Nov 30, 2015 at 2:36 PM, Jesse Keating <jlk@bluebox.net>
  wrote:
    I have an objection to eventlet going away. We have problems with
    running Apache and mod_wsgi with multiple python virtual
    environments. In some of our stacks we're running both Horizon and
    Keystone. Each get their own virtual environment. Apache mod_wsgi
    doesn't really work that way, so we'd have to do some ugly hacks to
    expose the python environments of both to Apache at the same time.

    I believe we spoke about this at Summit. Have you had time to look
    into this scenario and have suggestions?

    - jlk

    On Mon, Nov 30, 2015 at 10:26 AM, Steve Martinelli <
    stevemar@ca.ibm.com> wrote:
     This post is being sent again to the operators mailing list, and i
     apologize if it's duplicated for some folks. The original thread
     is here:
     http://lists.openstack.org/pipermail/openstack-dev/2015-November/080816.html

     In the Mitaka release, the keystone team will be removing
     functionality that was marked for deprecation in Kilo, and marking
     certain functions as deprecated in Mitaka (that may be removed in
     at least 2 cycles).

     removing deprecated functionality
     =====================

     This is not a full list, but these are by and large the most
     contentious topics.

     * Eventlet support: This was marked as deprecated back in Kilo and
     is currently scheduled to be removed in Mitaka in favor of running
     keystone in a WSGI server. This is currently how we test keystone
     in the gate, and based on the feedback we received at the summit,
     a lot of folks have moved to running keystone under Apache since
     we’ve announced this change. OpenStack's CI is configured to
     mainly test using this deployment model. See [0] for when we
     started to issue warnings.

     * Using LDAP to store assignment data: Like eventlet support, this
     feature was also deprecated in Kilo and scheduled to be removed in
     Mitaka. To store assignment data (role assignments) we suggest
     using an SQL based backend rather than LDAP. See [1] for when we
     started to issue warnings.

     * Using LDAP to store project and domain data: The same as above,
     see [2] for when we started to issue warnings.

     * for a complete list:
     https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka

     functions deprecated as of mitaka
     =====================

     The following will adhere to the TC’s new standard on deprecating
     functionality [3].

     * LDAP write support for identity: We suggest simply not writing
     to LDAP for users and groups, this effectively makes create,
     delete and update of LDAP users and groups a no-op. It will be
     removed in the O release.

     * PKI tokens: We suggest using UUID or fernet tokens instead. The
     PKI token format has had issues with security and causes problems
     with both horizon and swift when the token contains an excessively
     large service catalog. It will be removed in the O release.

     * v2.0 of our API: Lastly, the keystone team recommends using v3
     of our Identity API. We have had the intention of deprecating v2.0
     for a while (since Juno actually), and have finally decided to
     formally deprecate v2.0. OpenStack’s CI runs successful v3 only
     jobs, there is complete feature parity with v2.0, and we feel the
     CLI exposed via openstackclient is mature enough to say with
     certainty that we can deprecate v2.0. It will be around for at
     least FOUR releases, with the authentication routes
     (POST /auth/tokens) potentially sticking around for longer.

     * for a complete list:
     https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka

     If you have ANY concern about the following, please speak up now
     and let us know!

     Thanks!

     Steve Martinelli
     OpenStack Keystone Project Team Lead

     [0]
     https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80

     [1]
     https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34

     [2]
     https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39

     [3]
     http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html

     _______________________________________________
     OpenStack-operators mailing list
     OpenStack-operators@lists.openstack.org
     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

    _______________________________________________
    OpenStack-operators mailing list
    OpenStack-operators@lists.openstack.org
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

  _______________________________________________
  OpenStack-operators mailing list
  OpenStack-operators@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
   _______________________________________________
  OpenStack-operators mailing list
  OpenStack-operators@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Dec 1, 2015 by Steve_Martinelli (6,500 points)   1 3 5
0 votes

Trying to summarize here...

  • There isn't much interest in keeping eventlet around.
  • Folks are OK with running keystone in a WSGI server, but feel they are
    constrained by Apache.
  • uWSGI could help to support multiple web servers.

My opinion:

  • Adding support for uWSGI definitely sounds like it's worth
    investigating, but not achievable in this release (unless someone already
    has something cooked up).
  • I'm tempted to let eventlet stick around another release, since it's
    causing pain on some of our operators.
  • Other folks have managed to run keystone in a web server (and hopefully
    not feel pain when doing so!), so it might be worth getting technical
    details on just how it was accomplished. If we get an OK from the operator
    community later on in mitaka, I'd still be OK with removing eventlet, but I
    don't want to break folks.

stevemar

From: John Dewey john@dewey.ws
100% agree.

We should look at uwsgi as the reference architecture. Nginx/Apache/etc
should be interchangeable, and up to the operator which they choose to use.
Hell, with tcp load balancing now in opensource Nginx, I could get rid of
Apache and HAProxy by utilizing uwsgi.

John

On November 30, 2015 at 1:05:26 PM, Paul Czarkowski (pczarkowski
+openstackops@bluebox.net) wrote:

  I don't have a problem with eventlet itself going away, but I do feel
  that keystone should pick a python based web server capable of
  running WSGI apps ( such as uWSGI ) for the reference implementation
  rather than Apache which can be declared appropriately in the
  requirements.txt of the project.   I feel it is important to allow
  the operator to make choices based on their organization's skill sets
  ( i.e. apache vs nginx ) to help keep complexity low.

  I understand there are some newer features that rely on Apache
  ( federation, etc )  but we should allow the need for those features
  inform the operators choice of web server rather than force it for
  everybody.

  Having a default implementation using uWSGI is also more inline with
  the 12 factor way of writing applications and will run a lot more
  comfortably in [application] containers than apache would which is
  probably an important consideration given how many people are focused
  on being able to run openstack projects inside containers.

  On Mon, Nov 30, 2015 at 2:36 PM, Jesse Keating <jlk@bluebox.net>
  wrote:
    I have an objection to eventlet going away. We have problems with
    running Apache and mod_wsgi with multiple python virtual
    environments. In some of our stacks we're running both Horizon and
    Keystone. Each get their own virtual environment. Apache mod_wsgi
    doesn't really work that way, so we'd have to do some ugly hacks to
    expose the python environments of both to Apache at the same time.

    I believe we spoke about this at Summit. Have you had time to look
    into this scenario and have suggestions?

    - jlk

    On Mon, Nov 30, 2015 at 10:26 AM, Steve Martinelli <
    stevemar@ca.ibm.com> wrote:
     This post is being sent again to the operators mailing list, and i
     apologize if it's duplicated for some folks. The original thread
     is here:
     http://lists.openstack.org/pipermail/openstack-dev/2015-November/080816.html

     In the Mitaka release, the keystone team will be removing
     functionality that was marked for deprecation in Kilo, and marking
     certain functions as deprecated in Mitaka (that may be removed in
     at least 2 cycles).

     removing deprecated functionality
     =====================

     This is not a full list, but these are by and large the most
     contentious topics.

     * Eventlet support: This was marked as deprecated back in Kilo and
     is currently scheduled to be removed in Mitaka in favor of running
     keystone in a WSGI server. This is currently how we test keystone
     in the gate, and based on the feedback we received at the summit,
     a lot of folks have moved to running keystone under Apache since
     we’ve announced this change. OpenStack's CI is configured to
     mainly test using this deployment model. See [0] for when we
     started to issue warnings.

     * Using LDAP to store assignment data: Like eventlet support, this
     feature was also deprecated in Kilo and scheduled to be removed in
     Mitaka. To store assignment data (role assignments) we suggest
     using an SQL based backend rather than LDAP. See [1] for when we
     started to issue warnings.

     * Using LDAP to store project and domain data: The same as above,
     see [2] for when we started to issue warnings.

     * for a complete list:
     https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka

     functions deprecated as of mitaka
     =====================

     The following will adhere to the TC’s new standard on deprecating
     functionality [3].

     * LDAP write support for identity: We suggest simply not writing
     to LDAP for users and groups, this effectively makes create,
     delete and update of LDAP users and groups a no-op. It will be
     removed in the O release.

     * PKI tokens: We suggest using UUID or fernet tokens instead. The
     PKI token format has had issues with security and causes problems
     with both horizon and swift when the token contains an excessively
     large service catalog. It will be removed in the O release.

     * v2.0 of our API: Lastly, the keystone team recommends using v3
     of our Identity API. We have had the intention of deprecating v2.0
     for a while (since Juno actually), and have finally decided to
     formally deprecate v2.0. OpenStack’s CI runs successful v3 only
     jobs, there is complete feature parity with v2.0, and we feel the
     CLI exposed via openstackclient is mature enough to say with
     certainty that we can deprecate v2.0. It will be around for at
     least FOUR releases, with the authentication routes
     (POST /auth/tokens) potentially sticking around for longer.

     * for a complete list:
     https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka

     If you have ANY concern about the following, please speak up now
     and let us know!

     Thanks!

     Steve Martinelli
     OpenStack Keystone Project Team Lead

     [0]
     https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80

     [1]
     https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34

     [2]
     https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39

     [3]
     http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html

     _______________________________________________
     OpenStack-operators mailing list
     OpenStack-operators@lists.openstack.org
     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

    _______________________________________________
    OpenStack-operators mailing list
    OpenStack-operators@lists.openstack.org
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

  _______________________________________________
  OpenStack-operators mailing list
  OpenStack-operators@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
   _______________________________________________
  OpenStack-operators mailing list
  OpenStack-operators@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Dec 1, 2015 by Steve_Martinelli (6,500 points)   1 3 5
0 votes

Sounds like a solid set of takeaways. I would be okay with eventlet
sticking around until uWSGI ( or a similar thing ) is available and
functions in a similar fashion for those of us who want to choose our web
server.

On Tue, Dec 1, 2015 at 12:57 AM, Steve Martinelli stevemar@ca.ibm.com
wrote:

Trying to summarize here...

  • There isn't much interest in keeping eventlet around.
  • Folks are OK with running keystone in a WSGI server, but feel they are
    constrained by Apache.
  • uWSGI could help to support multiple web servers.

My opinion:

  • Adding support for uWSGI definitely sounds like it's worth
    investigating, but not achievable in this release (unless someone already
    has something cooked up).
  • I'm tempted to let eventlet stick around another release, since it's
    causing pain on some of our operators.
  • Other folks have managed to run keystone in a web server (and hopefully
    not feel pain when doing so!), so it might be worth getting technical
    details on just how it was accomplished. If we get an OK from the operator
    community later on in mitaka, I'd still be OK with removing eventlet, but I
    don't want to break folks.

stevemar

From: John Dewey john@dewey.ws

100% agree.

We should look at uwsgi as the reference architecture. Nginx/Apache/etc
should be interchangeable, and up to the operator which they choose to use.
Hell, with tcp load balancing now in opensource Nginx, I could get rid of
Apache and HAProxy by utilizing uwsgi.

John

On November 30, 2015 at 1:05:26 PM, Paul Czarkowski (
pczarkowski+openstackops@bluebox.net
pczarkowski+openstackops@bluebox.net) wrote:

I don't have a problem with eventlet itself going away, but I do feel
that keystone should pick a python based web server capable of running WSGI
apps ( such as uWSGI ) for the reference implementation rather than Apache
which can be declared appropriately in the requirements.txt of the project.
I feel it is important to allow the operator to make choices based on their
organization's skill sets ( i.e. apache vs nginx ) to help keep complexity
low.

  I understand there are some newer features that rely on Apache (
  federation, etc ) but we should allow the need for those features inform
  the operators choice of web server rather than force it for everybody.

  Having a default implementation using uWSGI is also more inline
  with the 12 factor way of writing applications and will run a lot more
  comfortably in [application] containers than apache would which is probably
  an important consideration given how many people are focused on being able
  to run openstack projects inside containers.

  On Mon, Nov 30, 2015 at 2:36 PM, Jesse Keating <*jlk@bluebox.net*
  <jlk@bluebox.net>> wrote:
     I have an objection to eventlet going away. We have problems
     with running Apache and mod_wsgi with multiple python virtual environments.
     In some of our stacks we're running both Horizon and Keystone. Each get
     their own virtual environment. Apache mod_wsgi doesn't really work that
     way, so we'd have to do some ugly hacks to expose the python environments
     of both to Apache at the same time.

     I believe we spoke about this at Summit. Have you had time to
     look into this scenario and have suggestions?


     - jlk

     On Mon, Nov 30, 2015 at 10:26 AM, Steve Martinelli <
     *stevemar@ca.ibm.com* <stevemar@ca.ibm.com>> wrote:
     This post is being sent again to the operators mailing list, and
     i apologize if it's duplicated for some folks. The original thread is here:
     *http://lists.openstack.org/pipermail/openstack-dev/2015-November/080816.html*
     

     In the Mitaka release, the keystone team will be removing
     functionality that was marked for deprecation in Kilo, and marking certain
     functions as deprecated in Mitaka (that may be removed in at least 2
     cycles).

     removing deprecated functionality
     =====================

     This is not a full list, but these are by and large the most
     contentious topics.

     * Eventlet support: This was marked as deprecated back in Kilo
     and is currently scheduled to be removed in Mitaka in favor of running
     keystone in a WSGI server. This is currently how we test keystone in the
     gate, and based on the feedback we received at the summit, a lot of folks
     have moved to running keystone under Apache since we’ve announced this
     change. OpenStack's CI is configured to mainly test using this deployment
     model. See [0] for when we started to issue warnings.

     * Using LDAP to store assignment data: Like eventlet support,
     this feature was also deprecated in Kilo and scheduled to be removed in
     Mitaka. To store assignment data (role assignments) we suggest using an SQL
     based backend rather than LDAP. See [1] for when we started to issue
     warnings.

     * Using LDAP to store project and domain data: The same as
     above, see [2] for when we started to issue warnings.

     * for a complete list:
     *https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka*
     <https://blueprints.launchpad.net/keystone/+spec/removed-as-of-mitaka>

     functions deprecated as of mitaka
     =====================

     The following will adhere to the TC’s new standard on
     deprecating functionality [3].

     * LDAP write support for identity: We suggest simply not writing
     to LDAP for users and groups, this effectively makes create, delete and
     update of LDAP users and groups a no-op. It will be removed in the O
     release.

     * PKI tokens: We suggest using UUID or fernet tokens instead.
     The PKI token format has had issues with security and causes problems with
     both horizon and swift when the token contains an excessively large service
     catalog. It will be removed in the O release.

     * v2.0 of our API: Lastly, the keystone team recommends using v3
     of our Identity API. We have had the intention of deprecating v2.0 for a
     while (since Juno actually), and have finally decided to formally deprecate
     v2.0. OpenStack’s CI runs successful v3 only jobs, there is complete
     feature parity with v2.0, and we feel the CLI exposed via openstackclient
     is mature enough to say with certainty that we can deprecate v2.0. It will
     be around for at least FOUR releases, with the authentication routes (POST
     /auth/tokens) potentially sticking around for longer.

     * for a complete list:
     *https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka*
     <https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-mitaka>


     If you have ANY concern about the following, please speak up now
     and let us know!


     Thanks!

     Steve Martinelli
     OpenStack Keystone Project Team Lead


     [0]
     *https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80*
     <https://github.com/openstack/keystone/blob/b475040636ccc954949e6372a60dd86845644611/keystone/server/eventlet.py#L77-L80>
     [1]
     *https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34*
     <https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/assignment/backends/ldap.py#L34>
     [2]
     *https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39*
     <https://github.com/openstack/keystone/blob/28a30f53a6c0d4e84d60795e08f137e8194abbe9/keystone/resource/backends/ldap.py#L36-L39>
     [3]
     *http://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html*
     



     _______________________________________________
     OpenStack-operators mailing list

OpenStack-operators@lists.openstack.org
OpenStack-operators@lists.openstack.org
*http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators*

     _______________________________________________
     OpenStack-operators mailing list

OpenStack-operators@lists.openstack.org
OpenStack-operators@lists.openstack.org
*http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators*

  _______________________________________________
  OpenStack-operators mailing list
  OpenStack-operators@lists.openstack.org

  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
  _______________________________________________
  OpenStack-operators mailing list
  OpenStack-operators@lists.openstack.org

  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Dec 2, 2015 by pczarkowski_plus_ope (300 points)   1
0 votes

... re-adding the operators mailing list.

sounds like we should document how to do this, with the assertion that it is not tested with our CI.

with that said, we should try to have a job that sets up keystone with nginx that is run periodically (similar to our eventlet job at the moment).

stevemar

Brant Knudson ---2015/12/07 05:52:20 PM---On Tue, Dec 1, 2015 at 12:57 AM, Steve Martinelli stevemar@ca.ibm.com wrote:

From: Brant Knudson blk@acm.org
To: "OpenStack Development Mailing List (not for usage questions)" openstack-dev@lists.openstack.org
Date: 2015/12/07 05:52 PM
Subject: Re: [openstack-dev] [Openstack-operators] [keystone] Removing functionality that was deprecated in Kilo and upcoming deprecated functionality in Mitaka

On Tue, Dec 1, 2015 at 12:57 AM, Steve Martinelli stevemar@ca.ibm.com wrote:Trying to summarize here...

  • There isn't much interest in keeping eventlet around.
  • Folks are OK with running keystone in a WSGI server, but feel they are constrained by Apache.
  • uWSGI could help to support multiple web servers.

My opinion:

  • Adding support for uWSGI definitely sounds like it's worth investigating, but not achievable in this release (unless someone already has something cooked up).

What needs to change to support uWSGI? You can already run keystone in python uwsgi and then front it with nginx:

 $ uwsgi --socket 127.0.0.1:5001 --wsgi-file $(which keystone-wsgi-public) --honour-stdin --enable-threads --workers 6
 $ uwsgi --socket 127.0.0.1:35358 --wsgi-file $(which keystone-wsgi-admin) --honour-stdin --enable-threads --workers 6

 $ sudo vi /etc/nginx/sites-available/keystone

server {
  listen 5000 defaultserver;
  server
name localhost;
  location / {
    include uwsgiparams;
    uwsgi
pass 127.0.0.1:5001;
    uwsgiparam SCRIPTNAME /;
  }
}
server {
  listen 35357 defaultserver;
  server
name localhost;
  location / {
    include uwsgiparams;
    uwsgi
pass 127.0.0.1:35358;
    uwsgiparam SCRIPTNAME /;
  }
}

 $ sudo ln -x /etc/nginx/sites-available/keystone /etc/nginx/sites-enabled/

 $ sudo nginx

and then you can make your regular curl calls.

Also, you can run keystone with regular http in python uwsgi (uwsgi --http) and then just do normal reverse proxy (from Apache or nginx or whatever), which I think would be adequate for keystone.

We don't do anything in keystone to stop deployments in web servers other than Apache. Keystone is just a regular wsgi app. We document Apache since it's popular and it provides mod_shib, which is the only saml2 module for web servers that I know of. Keystone can work with other saml2 modules and in different servers, it just takes the environment variables that the module sets and runs it through some mapping code. The mapping code has been shown to work alternative authentication modules (for ldap and kerberos).


responded Dec 7, 2015 by Steve_Martinelli (6,500 points)   1 3 5
0 votes

On Mon, Dec 07, 2015 at 06:18:04PM -0500, Steve Martinelli wrote:

... re-adding the operators mailing list.

sounds like we should document how to do this, with the assertion that it
is not tested with our CI.

with that said, we should try to have a job that sets up keystone with
nginx that is run periodically (similar to our eventlet job at the moment).

So, we actually run keystone with eventlet on every tempest-dsvm-postgres-full
job. It runs way more than periodically:

http://status.openstack.org/openstack-health/#/job/gate-tempest-dsvm-postgres-full

That's just a 24 hr window in the gate queue, including check it's much more.

This has been long standing behavior ever since keystone under mod_wsgi support
was added to devstack:

https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L1395-L1429

It's 1 of 3 things that are different that make the postgres job different. I've
always viewed that job config overloading as a bug, for this exact reason.

-Matt Treinish

From: Brant Knudson blk@acm.org
To: "OpenStack Development Mailing List (not for usage questions)"
openstack-dev@lists.openstack.org
Date: 2015/12/07 05:52 PM
Subject: Re: [openstack-dev] [Openstack-operators] [keystone] Removing
functionality that was deprecated in Kilo and upcoming
deprecated functionality in Mitaka

On Tue, Dec 1, 2015 at 12:57 AM, Steve Martinelli stevemar@ca.ibm.com
wrote:
Trying to summarize here...

  • There isn't much interest in keeping eventlet around.
  • Folks are OK with running keystone in a WSGI server, but feel they are
    constrained by Apache.
  • uWSGI could help to support multiple web servers.

    My opinion:

  • Adding support for uWSGI definitely sounds like it's worth
    investigating, but not achievable in this release (unless someone already
    has something cooked up).

What needs to change to support uWSGI? You can already run keystone in
python uwsgi and then front it with nginx:

 $ uwsgi --socket 127.0.0.1:5001 --wsgi-file $(which keystone-wsgi-public)
--honour-stdin --enable-threads --workers 6
 $ uwsgi --socket 127.0.0.1:35358 --wsgi-file $(which keystone-wsgi-admin)
--honour-stdin --enable-threads --workers 6

 $ sudo vi /etc/nginx/sites-available/keystone

server {
  listen 5000 defaultserver;
  server
name localhost;
  location / {
    include uwsgiparams;
    uwsgi
pass 127.0.0.1:5001;
    uwsgiparam SCRIPTNAME /;
  }
}
server {
  listen 35357 defaultserver;
  server
name localhost;
  location / {
    include uwsgiparams;
    uwsgi
pass 127.0.0.1:35358;
    uwsgiparam SCRIPTNAME /;
  }
}

 $ sudo ln -x /etc/nginx/sites-available/keystone /etc/nginx/sites-enabled/

 $ sudo nginx

and then you can make your regular curl calls.

Also, you can run keystone with regular http in python uwsgi (uwsgi --http)
and then just do normal reverse proxy (from Apache or nginx or whatever),
which I think would be adequate for keystone.

We don't do anything in keystone to stop deployments in web servers other
than Apache. Keystone is just a regular wsgi app. We document Apache since
it's popular and it provides mod_shib, which is the only saml2 module for
web servers that I know of. Keystone can work with other saml2 modules and
in different servers, it just takes the environment variables that the
module sets and runs it through some mapping code. The mapping code has
been shown to work alternative authentication modules (for ldap and
kerberos).


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

responded Dec 7, 2015 by Matthew_Treinish (11,200 points)   2 5 5
...