settingsLogin | Registersettings

[openstack-dev] [horizon] Fetch resources in parallel

0 votes

Hi team,

We all know that unfortunately Horizon's performance not good enough in
some cases. That's why we try to fix it on both sides: client-side and
server-side.

I would like to talk mostly about server-side now. On some views, we use
'futurist' library [1] to fetch resources from API's in parallel. I've
filed a blueprint for this effort [2]. You can find some work in progress
patches in the Gerrit.

For now, we use futurist.ThreadPoolExecutor in Horizon to fetch resources
from APIs. It requires some specific Apache configuration changes to allow
WSGI app to create threads. It means, our code execution depends on Apache
mod_wsgi configuration.

To avoid this, we can use futurist.GreenThreadPoolExecutor. It
uses eventlet green threads which can be used to speed-up I/O operations
like 'call external API'.

I did very simple performance testing [3] with proposed patch [4] for
volumes views. My tests are not ideal but you can see how it's going with
green thread. I propose to switch to the futurist.GreenThreadPoolExecutor
and add 'eventlet.monkey_patch()' call to the wsgi app for Horizon. It will
speed up parallel API calls and make Horizon independent on Apache or other
web server configuration.

I added this topic to the next weekly meeting agenda [5] so we can discuss
it there too.

[1] https://github.com/openstack/futurist/
[2]
https://blueprints.launchpad.net/horizon/+spec/fetch-resources-in-parallel
[3]
https://docs.google.com/spreadsheets/d/14zDpdkPUfGDR_ZycaGUoT64kHsGi1gL9JOha8n5PVeg/edit?usp=sharing
[4] https://review.openstack.org/#/c/426493/
[5] https://wiki.openstack.org/wiki/Meetings/Horizon#Agenda_for_Next_Meeting

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Oct 31, 2017 in openstack-dev by Ivan_Kolodyazhny (3,680 points)   5 6

1 Response

0 votes

Just forgot to mention one other option: we can use celery [6] too but I
didn't do any performance testing with it. This solution will be more
complex.

[6] http://www.celeryproject.org/

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Tue, Oct 31, 2017 at 7:15 PM, Ivan Kolodyazhny e0ne@e0ne.info wrote:

Hi team,

We all know that unfortunately Horizon's performance not good enough in
some cases. That's why we try to fix it on both sides: client-side and
server-side.

I would like to talk mostly about server-side now. On some views, we use
'futurist' library [1] to fetch resources from API's in parallel. I've
filed a blueprint for this effort [2]. You can find some work in progress
patches in the Gerrit.

For now, we use futurist.ThreadPoolExecutor in Horizon to fetch resources
from APIs. It requires some specific Apache configuration changes to allow
WSGI app to create threads. It means, our code execution depends on Apache
mod_wsgi configuration.

To avoid this, we can use futurist.GreenThreadPoolExecutor. It
uses eventlet green threads which can be used to speed-up I/O operations
like 'call external API'.

I did very simple performance testing [3] with proposed patch [4] for
volumes views. My tests are not ideal but you can see how it's going with
green thread. I propose to switch to the futurist.GreenThreadPoolExecutor
and add 'eventlet.monkey_patch()' call to the wsgi app for Horizon. It will
speed up parallel API calls and make Horizon independent on Apache or other
web server configuration.

I added this topic to the next weekly meeting agenda [5] so we can discuss
it there too.

[1] https://github.com/openstack/futurist/
[2] https://blueprints.launchpad.net/horizon/+spec/
fetch-resources-in-parallel
[3] https://docs.google.com/spreadsheets/d/14zDpdkPUfGDR_
ZycaGUoT64kHsGi1gL9JOha8n5PVeg/edit?usp=sharing
[4] https://review.openstack.org/#/c/426493/
[5] https://wiki.openstack.org/wiki/Meetings/Horizon#
AgendaforNext_Meeting

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 31, 2017 by Ivan_Kolodyazhny (3,680 points)   5 6
...