settingsLogin | Registersettings

[openstack-dev] Apache2 vs uWSGI vs ...

0 votes

In the Fuel project, we recently ran into few issues with Apache2 +
mod_wsgi as we switched Keystone to run in that environment and we started
a large battery of tests and ended up seeing these issues. Please see [1]
and [2] for examples.

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Thanks,
Dims

PS: I will leave issues with memcached + keystone for another email later :)

[1] https://bugs.launchpad.net/mos/+bug/1491576
[2] https://bugs.launchpad.net/fuel/+bug/1493372
[3] https://bugs.launchpad.net/fuel/+bug/1493372/comments/35
--
Davanum Srinivas :: https://twitter.com/dims


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Sep 17, 2015 in openstack-dev by Davanum_Srinivas (35,920 points)   2 5 9

23 Responses

0 votes

In the fuel project, we recently ran into a couple of issues with Apache2 +
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Thanks,
Dims

[1] https://bugs.launchpad.net/mos/+bug/1491576
[2] https://bugs.launchpad.net/fuel/+bug/1493372
[3] https://bugs.launchpad.net/fuel/+bug/1493372/comments/35
--
Davanum Srinivas :: https://twitter.com/dims


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 17, 2015 by Davanum_Srinivas (35,920 points)   2 5 9
0 votes

On 09/17/2015 06:48 PM, Davanum Srinivas wrote:
In the fuel project, we recently ran into a couple of issues with
Apache2 + mod_wsgi as we switched Keystone to run . Please see [1] and
[2].

Looking deep into Apache2 issues specifically around "apache2ctl
graceful" and module loading/unloading and the hooks used by modwsgi
[3]. I started wondering if Apache2 + mod
wsgi is the "right" solution
and if there was something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Thanks,
Dims

I'd be surprised if switching web servers fixed more problems than it
causes. The issues with Apache seem to be issues that are solvable; is
there any reason to think that they are not?

[1] https://bugs.launchpad.net/mos/+bug/1491576
[2] https://bugs.launchpad.net/fuel/+bug/1493372
[3] https://bugs.launchpad.net/fuel/+bug/1493372/comments/35
--
Davanum Srinivas :: https://twitter.com/dims


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Adam_Young (19,940 points)   2 7 12
0 votes

On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:
In the fuel project, we recently ran into a couple of issues with Apache2 +
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Disclaimer: it's been a while since I've cared about performance with a
web server in front of a Python app.

IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
on again. In general, I seem to remember it being thought of as a bit
old and crusty, but mostly working.

At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
and saw a significant performance increase. This was a Django app. uwsgi
is fairly straightforward to operate and comes loaded with a myriad of
options[1] to help folks make the most of it. I've played with Ironic
behind uwsgi and it seemed to work fine, though I haven't done any sort
of load testing. I'd encourage folks to give it a shot. :)

Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

gunicorn[2] is another good option that may be worth investigating; I
personally don't have any experience with it, but I seem to remember
hearing it has good eventlet support.

// jim

[0] https://uwsgi-docs.readthedocs.org/en/latest/
[1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
[2] http://gunicorn.org/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Jim_Rollenhagen (12,800 points)   2 3 3
0 votes

On 09/17/2015 10:04 PM, Jim Rollenhagen wrote:
On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:

In the fuel project, we recently ran into a couple of issues with Apache2 +
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?
Disclaimer: it's been a while since I've cared about performance with a
web server in front of a Python app.

IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
on again. In general, I seem to remember it being thought of as a bit
old and crusty, but mostly working.

I am not aware of that. It has been the workhorse of the Python/wsgi
world for a while, and we use it heavily.

At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
and saw a significant performance increase. This was a Django app. uwsgi
is fairly straightforward to operate and comes loaded with a myriad of
options[1] to help folks make the most of it. I've played with Ironic
behind uwsgi and it seemed to work fine, though I haven't done any sort
of load testing. I'd encourage folks to give it a shot. :)

Again, switching web servers is as likely to introduce as to solve
problems. If there are performance issues:

  1. Idenitfy what causes them
  2. Change configuration settings to deal with them
  3. Fix upstream bugs in the underlying system.

Keystone is not about performance. Keystone is about security. The
cloud is designed to scale horizontally first. Before advocating
switching to a difference web server, make sure it supports the
technologies required.

  1. TLS at the latest level
  2. Kerberos/GSSAPI/SPNEGO
  3. X509 Client cert validation
  4. SAML

OpenID connect would be a good one to add to the list; Its been
requested for a while.

If Keystone is having performance issues, it is most likely at the
database layer, not the web server.

"Programmers waste enormous amounts of time thinking about, or worrying
about, the speed of noncritical parts of their programs, and these
attempts at efficiency actually have a strong negative impact when
debugging and maintenance are considered. We /should/ forget about small
efficiencies, say about 97% of the time: premature optimization is the
root of all evil.
Yet we should not pass up our opportunities in that
critical 3%." --Donald Knuth

Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

gunicorn[2] is another good option that may be worth investigating; I
personally don't have any experience with it, but I seem to remember
hearing it has good eventlet support.

// jim

[0] https://uwsgi-docs.readthedocs.org/en/latest/
[1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
[2] http://gunicorn.org/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Adam_Young (19,940 points)   2 7 12
0 votes

There is and has been desire to support uWSGI and other alternatives to modwsgi. There are a variety of operational reasons to consider uWSGI and/or gunicorn behind apache most notably to facilitate easier management of the processes independently of the webserver itself. With modwsgi the processes are directly tied to the apache server where as with uWSGI and gunicorn you can manage the various services independently and/or with differing VENVs more easily.

There are potential other concerns that must be weighed when considering which method of deployment to use. I hope we have clear documentation within the next cycle (and possible choices for the gate) for utilizing uWSGI and/or gunicorn.

--Morgan

Sent via mobile

On Sep 18, 2015, at 06:12, Adam Young ayoung@redhat.com wrote:

On 09/17/2015 10:04 PM, Jim Rollenhagen wrote:

On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:
In the fuel project, we recently ran into a couple of issues with Apache2 +
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?
Disclaimer: it's been a while since I've cared about performance with a
web server in front of a Python app.

IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
on again. In general, I seem to remember it being thought of as a bit
old and crusty, but mostly working.

I am not aware of that. It has been the workhorse of the Python/wsgi world for a while, and we use it heavily.

At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
and saw a significant performance increase. This was a Django app. uwsgi
is fairly straightforward to operate and comes loaded with a myriad of
options[1] to help folks make the most of it. I've played with Ironic
behind uwsgi and it seemed to work fine, though I haven't done any sort
of load testing. I'd encourage folks to give it a shot. :)

Again, switching web servers is as likely to introduce as to solve problems. If there are performance issues:

  1. Idenitfy what causes them
  2. Change configuration settings to deal with them
  3. Fix upstream bugs in the underlying system.

Keystone is not about performance. Keystone is about security. The cloud is designed to scale horizontally first. Before advocating switching to a difference web server, make sure it supports the technologies required.

  1. TLS at the latest level
  2. Kerberos/GSSAPI/SPNEGO
  3. X509 Client cert validation
  4. SAML

OpenID connect would be a good one to add to the list; Its been requested for a while.

If Keystone is having performance issues, it is most likely at the database layer, not the web server.

"Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." --Donald Knuth

Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

gunicorn[2] is another good option that may be worth investigating; I
personally don't have any experience with it, but I seem to remember
hearing it has good eventlet support.

// jim

[0] https://uwsgi-docs.readthedocs.org/en/latest/
[1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
[2] http://gunicorn.org/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Morgan_Fainberg (17,320 points)   2 6 9
0 votes

Folks

I think we do not need to switch to nginx-only or consider any kind of war
between nginx and apache adherents. Everyone should be able to use
web-server he or she needs without being pinned to the unwanted one. It is
like Postgres vs MySQL war. Why not support both?

May be someone does not need something that apache supports and nginx not
and needs nginx features which apache does not support. Let's let our users
decide what they want.

And the first step should be simple here - support for uwsgi. It will allow
for usage of any web-server that can work with uwsgi. It will allow also us
to check for the support of all apache-like bindings like SPNEGO or
whatever and provide our users with enough info on making decisions. I did
not personally test nginx modules for SAML and SPNEGO, but I am pretty
confident about TLS/SSL parts of nginx.

Moreover, nginx will allow you to do things you cannot do with apache, e.g.
do smart load balancing, which may be crucial for high-loaded installations.

On Fri, Sep 18, 2015 at 4:12 PM, Adam Young ayoung@redhat.com wrote:

On 09/17/2015 10:04 PM, Jim Rollenhagen wrote:

On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:

In the fuel project, we recently ran into a couple of issues with Apache2 +
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Disclaimer: it's been a while since I've cared about performance with a
web server in front of a Python app.

IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
on again. In general, I seem to remember it being thought of as a bit
old and crusty, but mostly working.

I am not aware of that. It has been the workhorse of the Python/wsgi
world for a while, and we use it heavily.

At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
and saw a significant performance increase. This was a Django app. uwsgi
is fairly straightforward to operate and comes loaded with a myriad of
options[1] to help folks make the most of it. I've played with Ironic
behind uwsgi and it seemed to work fine, though I haven't done any sort
of load testing. I'd encourage folks to give it a shot. :)

Again, switching web servers is as likely to introduce as to solve
problems. If there are performance issues:

  1. Idenitfy what causes them
  2. Change configuration settings to deal with them
  3. Fix upstream bugs in the underlying system.

Keystone is not about performance. Keystone is about security. The cloud
is designed to scale horizontally first. Before advocating switching to a
difference web server, make sure it supports the technologies required.

  1. TLS at the latest level
  2. Kerberos/GSSAPI/SPNEGO
  3. X509 Client cert validation
  4. SAML

OpenID connect would be a good one to add to the list; Its been requested
for a while.

If Keystone is having performance issues, it is most likely at the
database layer, not the web server.

"Programmers waste enormous amounts of time thinking about, or worrying
about, the speed of noncritical parts of their programs, and these attempts
at efficiency actually have a strong negative impact when debugging and
maintenance are considered. We should forget about small efficiencies,
say about 97% of the time: premature optimization is the root of all
evil.
Yet we should not pass up our opportunities in that critical
3%." --Donald Knuth

Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

gunicorn[2] is another good option that may be worth investigating; I
personally don't have any experience with it, but I seem to remember
hearing it has good eventlet support.

// jim

[0] https://uwsgi-docs.readthedocs.org/en/latest/
[1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
[2] http://gunicorn.org/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
35bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com
www.mirantis.ru
vkuklin@mirantis.com


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Vladimir_Kuklin (7,320 points)   1 3 4
0 votes

Please consider that we use some apache mods - does
nginx/uwsgi/gunicorn have oauth, shibboleth & openid support?

On Fri, Sep 18, 2015 at 4:54 PM, Vladimir Kuklin vkuklin@mirantis.com wrote:
Folks

I think we do not need to switch to nginx-only or consider any kind of war
between nginx and apache adherents. Everyone should be able to use
web-server he or she needs without being pinned to the unwanted one. It is
like Postgres vs MySQL war. Why not support both?

May be someone does not need something that apache supports and nginx not
and needs nginx features which apache does not support. Let's let our users
decide what they want.

And the first step should be simple here - support for uwsgi. It will allow
for usage of any web-server that can work with uwsgi. It will allow also us
to check for the support of all apache-like bindings like SPNEGO or whatever
and provide our users with enough info on making decisions. I did not
personally test nginx modules for SAML and SPNEGO, but I am pretty confident
about TLS/SSL parts of nginx.

Moreover, nginx will allow you to do things you cannot do with apache, e.g.
do smart load balancing, which may be crucial for high-loaded installations.

On Fri, Sep 18, 2015 at 4:12 PM, Adam Young ayoung@redhat.com wrote:

On 09/17/2015 10:04 PM, Jim Rollenhagen wrote:

On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:

In the fuel project, we recently ran into a couple of issues with Apache2
+
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Disclaimer: it's been a while since I've cared about performance with a
web server in front of a Python app.

IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
on again. In general, I seem to remember it being thought of as a bit
old and crusty, but mostly working.

I am not aware of that. It has been the workhorse of the Python/wsgi
world for a while, and we use it heavily.

At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
and saw a significant performance increase. This was a Django app. uwsgi
is fairly straightforward to operate and comes loaded with a myriad of
options[1] to help folks make the most of it. I've played with Ironic
behind uwsgi and it seemed to work fine, though I haven't done any sort
of load testing. I'd encourage folks to give it a shot. :)

Again, switching web servers is as likely to introduce as to solve
problems. If there are performance issues:

  1. Idenitfy what causes them
  2. Change configuration settings to deal with them
  3. Fix upstream bugs in the underlying system.

Keystone is not about performance. Keystone is about security. The cloud
is designed to scale horizontally first. Before advocating switching to a
difference web server, make sure it supports the technologies required.

  1. TLS at the latest level
  2. Kerberos/GSSAPI/SPNEGO
  3. X509 Client cert validation
  4. SAML

OpenID connect would be a good one to add to the list; Its been requested
for a while.

If Keystone is having performance issues, it is most likely at the
database layer, not the web server.

"Programmers waste enormous amounts of time thinking about, or worrying
about, the speed of noncritical parts of their programs, and these attempts
at efficiency actually have a strong negative impact when debugging and
maintenance are considered. We should forget about small efficiencies, say
about 97% of the time: premature optimization is the root of all evil. Yet
we should not pass up our opportunities in that critical 3%." --Donald
Knuth

Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

gunicorn[2] is another good option that may be worth investigating; I
personally don't have any experience with it, but I seem to remember
hearing it has good eventlet support.

// jim

[0] https://uwsgi-docs.readthedocs.org/en/latest/
[1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
[2] http://gunicorn.org/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
35bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com
www.mirantis.ru
vkuklin@mirantis.com


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Kind Regards,
Alexander Makarov,
Senior Software Developer,

Mirantis, Inc.
35b/3, Vorontsovskaya St., 109147, Moscow, Russia

Tel.: +7 (495) 640-49-04
Tel.: +7 (926) 204-50-60

Skype: MAKAPOB.AJIEKCAHDP


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Alexander_V_Makarov (900 points)   1 2
0 votes

On 18/09/15 06:44 -0700, Morgan Fainberg wrote:
There is and has been desire to support uWSGI and other alternatives to
modwsgi. There are a variety of operational reasons to consider uWSGI and/or
gunicorn behind apache most notably to facilitate easier management of the
processes independently of the webserver itself. With mod
wsgi the processes
are directly tied to the apache server where as with uWSGI and gunicorn you can
manage the various services independently and/or with differing VENVs more
easily.

There are potential other concerns that must be weighed when considering which
method of deployment to use. I hope we have clear documentation within the next
cycle (and possible choices for the gate) for utilizing uWSGI and/or gunicorn.

+1

FWIW, Zaqar has always been shipped as a wsgi app and the container
the team has recommended ever since it was put in production for the
first time has been uWSGI. uWSGI is already used by Zaqar in the gate
but it's being installed independently.

Flavio

--Morgan

Sent via mobile

On Sep 18, 2015, at 06:12, Adam Young ayoung@redhat.com wrote:

On 09/17/2015 10:04 PM, Jim Rollenhagen wrote:

   On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:

       In the fuel project, we recently ran into a couple of issues with Apache2 +
       mod_wsgi as we switched Keystone to run . Please see [1] and [2].

       Looking deep into Apache2 issues specifically around "apache2ctl graceful"
       and module loading/unloading and the hooks used by mod_wsgi [3]. I started
       wondering if Apache2 + mod_wsgi is the "right" solution and if there was
       something else better that people are already using.

       One data point that keeps coming up is, all the CI jobs use Apache2 +
       mod_wsgi so it must be the best solution....Is it? If not, what is?

   Disclaimer: it's been a while since I've cared about performance with a
   web server in front of a Python app.

   IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
   on again. In general, I seem to remember it being thought of as a bit
   old and crusty, but mostly working.

I am not aware of that. It has been the workhorse of the Python/wsgi world
for a while, and we use it heavily.

   At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
   and saw a significant performance increase. This was a Django app. uwsgi
   is fairly straightforward to operate and comes loaded with a myriad of
   options[1] to help folks make the most of it. I've played with Ironic
   behind uwsgi and it seemed to work fine, though I haven't done any sort
   of load testing. I'd encourage folks to give it a shot. :)

Again, switching web servers is as likely to introduce as to solve
problems. If there are performance issues:

  1. Idenitfy what causes them
  2. Change configuration settings to deal with them
  3. Fix upstream bugs in the underlying system.

    Keystone is not about performance. Keystone is about security. The cloud
    is designed to scale horizontally first. Before advocating switching to a
    difference web server, make sure it supports the technologies required.

  4. TLS at the latest level

  5. Kerberos/GSSAPI/SPNEGO
  6. X509 Client cert validation
  7. SAML

    OpenID connect would be a good one to add to the list; Its been requested
    for a while.

    If Keystone is having performance issues, it is most likely at the database
    layer, not the web server.

    "Programmers waste enormous amounts of time thinking about, or worrying
    about, the speed of noncritical parts of their programs, and these attempts
    at efficiency actually have a strong negative impact when debugging and
    maintenance are considered. We should forget about small efficiencies, say
    about 97% of the time: premature optimization is the root of all evil. Yet
    we should not pass up our opportunities in that critical 3%." --Donald
    Knuth

    Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

    gunicorn[2] is another good option that may be worth investigating; I
    personally don't have any experience with it, but I seem to remember
    hearing it has good eventlet support.

    // jim

    [0] https://uwsgi-docs.readthedocs.org/en/latest/
    [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
    [2] http://gunicorn.org/


    OpenStack Development Mailing List (not for usage questions)
    Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
@flaper87
Flavio Percoco


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Sep 18, 2015 by Flavio_Percoco (36,960 points)   3 8 13
0 votes

There are 2 dimensions this discussion should happen in: web server and
application server. Now we use apache2 as web server and mod_wsgi as app
server.

I don't have a specific opinion on the app server (mod_wsgi vs uwsgi) and I
don't really care.

Regarding apache2 vs nginx. I don't see any reasons for the switch. Apache2 is
well known to deployers and sysadmins. It is very rich for modules. I wonder
if there are customer-written modules.

On Friday 18 September 2015 16:54:02 Vladimir Kuklin wrote:
Folks

I think we do not need to switch to nginx-only or consider any kind of war
between nginx and apache adherents. Everyone should be able to use
web-server he or she needs without being pinned to the unwanted one. It is
like Postgres vs MySQL war. Why not support both?

Why nginx? Why not lighttpd? OpenLitespeed? Litespeed? ?

What do you understand by "support both"? I understand it as "both are tested
in devstack". Apache2 is supported because you can set up devstack and
everything works.

There are things in keystone that work under apache. They are not tested. They
were written to work under apache because it's the simplest and the most
standard way to do. Making them work in nginx means forcing developers write
some code. You're ready to do that?

May be someone does not need something that apache supports and nginx not
and needs nginx features which apache does not support. Let's let our users
decide what they want.

And the first step should be simple here - support for uwsgi.

Why uwsgi? Why not gunicorn? Cherrypy? Twisted?

It will allow
for usage of any web-server that can work with uwsgi. It will allow also us
to check for the support of all apache-like bindings like SPNEGO or
whatever and provide our users with enough info on making decisions. I did
not personally test nginx modules for SAML and SPNEGO, but I am pretty
confident about TLS/SSL parts of nginx.

Moreover, nginx will allow you to do things you cannot do with apache, e.g.
do smart load balancing, which may be crucial for high-loaded installations.
On Fri, Sep 18, 2015 at 4:12 PM, Adam Young ayoung@redhat.com wrote:

On 09/17/2015 10:04 PM, Jim Rollenhagen wrote:

On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:

In the fuel project, we recently ran into a couple of issues with Apache2
+
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Disclaimer: it's been a while since I've cared about performance with a
web server in front of a Python app.

IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
on again. In general, I seem to remember it being thought of as a bit
old and crusty, but mostly working.

I am not aware of that. It has been the workhorse of the Python/wsgi
world for a while, and we use it heavily.

At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
and saw a significant performance increase. This was a Django app. uwsgi
is fairly straightforward to operate and comes loaded with a myriad of
options[1] to help folks make the most of it. I've played with Ironic
behind uwsgi and it seemed to work fine, though I haven't done any sort
of load testing. I'd encourage folks to give it a shot. :)

Again, switching web servers is as likely to introduce as to solve
problems. If there are performance issues:

  1. Idenitfy what causes them
  2. Change configuration settings to deal with them
  3. Fix upstream bugs in the underlying system.

Keystone is not about performance. Keystone is about security. The cloud
is designed to scale horizontally first. Before advocating switching to a
difference web server, make sure it supports the technologies required.

  1. TLS at the latest level
  2. Kerberos/GSSAPI/SPNEGO
  3. X509 Client cert validation
  4. SAML

OpenID connect would be a good one to add to the list; Its been requested
for a while.

If Keystone is having performance issues, it is most likely at the
database layer, not the web server.

"Programmers waste enormous amounts of time thinking about, or worrying
about, the speed of noncritical parts of their programs, and these
attempts
at efficiency actually have a strong negative impact when debugging and
maintenance are considered. We should forget about small efficiencies,
say about 97% of the time: premature optimization is the root of all
evil.
Yet we should not pass up our opportunities in that critical
3%." --Donald Knuth

Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

gunicorn[2] is another good option that may be worth investigating; I
personally don't have any experience with it, but I seem to remember
hearing it has good eventlet support.

// jim

[0] https://uwsgi-docs.readthedocs.org/en/latest/
[1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
[2] http://gunicorn.org/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribehttp://lists
.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
С наилучшими пожеланиями,
Boris


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Boris_Bobrov (1,720 points)   1 3
0 votes

Part of the reason to use Apache though is the diverse set of authentication mechanisms it supports. Operators have the desire to plugin Keystone into their existing authentication systems and Apache tends to be easier to do that then others.

Thanks,
Kevin


From: Jim Rollenhagen [jim@jimrollenhagen.com]
Sent: Thursday, September 17, 2015 7:04 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] Apache2 vs uWSGI vs ...

On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote:
In the fuel project, we recently ran into a couple of issues with Apache2 +
mod_wsgi as we switched Keystone to run . Please see [1] and [2].

Looking deep into Apache2 issues specifically around "apache2ctl graceful"
and module loading/unloading and the hooks used by modwsgi [3]. I started
wondering if Apache2 + mod
wsgi is the "right" solution and if there was
something else better that people are already using.

One data point that keeps coming up is, all the CI jobs use Apache2 +
mod_wsgi so it must be the best solution....Is it? If not, what is?

Disclaimer: it's been a while since I've cared about performance with a
web server in front of a Python app.

IIRC, mod_wsgi was abandoned for a while, but I think it's being worked
on again. In general, I seem to remember it being thought of as a bit
old and crusty, but mostly working.

At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0]
and saw a significant performance increase. This was a Django app. uwsgi
is fairly straightforward to operate and comes loaded with a myriad of
options[1] to help folks make the most of it. I've played with Ironic
behind uwsgi and it seemed to work fine, though I haven't done any sort
of load testing. I'd encourage folks to give it a shot. :)

Of course, uwsgi can also be ran behind Apache2, if you'd prefer.

gunicorn[2] is another good option that may be worth investigating; I
personally don't have any experience with it, but I seem to remember
hearing it has good eventlet support.

// jim

[0] https://uwsgi-docs.readthedocs.org/en/latest/
[1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html
[2] http://gunicorn.org/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 18, 2015 by Fox,_Kevin_M (29,360 points)   1 3 4
...