settingsLogin | Registersettings

[Openstack-operators] CentOS upgrade killed my Liberty system

0 votes

Hi all;

About 18 months ago I did a piece by piece install of Liberty (following
instructions on the Openstack page) and had been using it until somewhat
recently when it broke after a CentOS 7 update ran. I think Python/http is
what changed / broke (sessionless connections no longer supported?).
Basics like nova list still work, but Horizon won't run.

How can I "downgrade" my CentOS box?
Or alternatively, how painful is it to update to the most current
Openstack?

Also, I noticed that the CeotOS cloud SIG repo moved.

Please help. :-)

BTW, for installs I wrote a tool to help with setting up the many Openstack
config files. If interested, you can find it here.
http://chrishull.com/career/openstack/index.html
https://github.com/chrishull/github-openstack

Several months ago I tried to check this into the Openstack Operators GIT
repo, but had difficulty.

Anyway. Please advise on how I can fix my Liberty system if you can.

Thanks;
-Chris

asked Oct 10, 2017 in openstack-operators by Christopher_Hull (2,220 points)   2 5 7

5 Responses

0 votes

Can you paste out the horizon related error logs, may be caused by horizon
dependency issue.

On Tue, Oct 10, 2017 at 2:12 AM, Christopher Hull chrishull42@gmail.com
wrote:

Hi all;

About 18 months ago I did a piece by piece install of Liberty (following
instructions on the Openstack page) and had been using it until somewhat
recently when it broke after a CentOS 7 update ran. I think Python/http is
what changed / broke (sessionless connections no longer supported?).
Basics like nova list still work, but Horizon won't run.

How can I "downgrade" my CentOS box?
Or alternatively, how painful is it to update to the most current
Openstack?

Also, I noticed that the CeotOS cloud SIG repo moved.

Please help. :-)

BTW, for installs I wrote a tool to help with setting up the many
Openstack config files. If interested, you can find it here.
http://chrishull.com/career/openstack/index.html
https://github.com/chrishull/github-openstack

Several months ago I tried to check this into the Openstack Operators GIT
repo, but had difficulty.

Anyway. Please advise on how I can fix my Liberty system if you can.

Thanks;
-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865 <(415)%20385-4865>
    chrishull42@gmail.com
    http://chrishull.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Tang Yaguang


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 10, 2017 by Yaguang_Tang (1,180 points)   1 2 4
0 votes

Hello Yaguang;

Thanks very much for the assist. Curiously, no errors in horizon. But an
error upstream in httpd...

Browser hangs for a while then shows "Internal Server Error"

/var/log/httpd/error shows this...

[Tue Oct 10 09:32:53.860393 2017] [core:error] [pid 27391] [client
172.22.10.130:53239] End of script output before headers: django.wsgi

Hope this looks familiar to someone.

Thanks all;

-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865
    chrishull42@gmail.com
    http://chrishull.com

On Tue, Oct 10, 2017 at 5:47 AM, Yaguang Tang heut2008@gmail.com wrote:

Can you paste out the horizon related error logs, may be caused by horizon
dependency issue.

On Tue, Oct 10, 2017 at 2:12 AM, Christopher Hull chrishull42@gmail.com
wrote:

Hi all;

About 18 months ago I did a piece by piece install of Liberty (following
instructions on the Openstack page) and had been using it until somewhat
recently when it broke after a CentOS 7 update ran. I think Python/http is
what changed / broke (sessionless connections no longer supported?).
Basics like nova list still work, but Horizon won't run.

How can I "downgrade" my CentOS box?
Or alternatively, how painful is it to update to the most current
Openstack?

Also, I noticed that the CeotOS cloud SIG repo moved.

Please help. :-)

BTW, for installs I wrote a tool to help with setting up the many
Openstack config files. If interested, you can find it here.
http://chrishull.com/career/openstack/index.html
https://github.com/chrishull/github-openstack

Several months ago I tried to check this into the Openstack Operators GIT
repo, but had difficulty.

Anyway. Please advise on how I can fix my Liberty system if you can.

Thanks;
-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865 <(415)%20385-4865>
    chrishull42@gmail.com
    http://chrishull.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Tang Yaguang


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 10, 2017 by Christopher_Hull (2,220 points)   2 5 7
0 votes

On Tue, 10 Oct 2017 09:59:03 -0700, Christopher Hull wrote:
Browser hangs for a while then shows "Internal Server Error"

/var/log/httpd/error shows this...

[Tue Oct 10 09:32:53.860393 2017] [core:error] [pid 27391] [client
172.22.10.130:53239 ] End of script output
before headers: django.wsgi

Hope this looks familiar to someone.

It sounded familiar to me and I found these bugs [1][2] which might be
what you're seeing. The latest comments say "WSGIApplicationGroup
%{GLOBAL}" needs to be added to openstack-dashboard.conf and httpd
restarted.

Hope that helps.

-melanie

[1] https://bugs.launchpad.net/horizon/+bug/1573488
[2] https://bugs.launchpad.net/horizon/+bug/1708655


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 10, 2017 by melanie_witt (3,180 points)   2 5
0 votes

Hello Chris,

This may be related to a Horizon.Django issue we saw too on a RHEL7.3 to
RHEL7.4 update.

If the config is similar for CentOS7.4 packaging, if you check your Horizon
vhost under

/etc/httpd/conf.d/*-horizon_vhost.conf

on all your controllers serving Horizon there may be a WSGI directive that
needs editing;

WSGIApplicationGroup %{GLOBAL}

should go directly under the other WSGI directives in the vhost, and then
httpd restarted.

  • Paul

On 10 October 2017 at 17:59, Christopher Hull chrishull42@gmail.com wrote:

Hello Yaguang;

Thanks very much for the assist. Curiously, no errors in horizon. But
an error upstream in httpd...

Browser hangs for a while then shows "Internal Server Error"

/var/log/httpd/error shows this...

[Tue Oct 10 09:32:53.860393 2017] [core:error] [pid 27391] [client
172.22.10.130:53239] End of script output before headers: django.wsgi

Hope this looks familiar to someone.

Thanks all;

-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865 <(415)%20385-4865>
    chrishull42@gmail.com
    http://chrishull.com

On Tue, Oct 10, 2017 at 5:47 AM, Yaguang Tang heut2008@gmail.com wrote:

Can you paste out the horizon related error logs, may be caused by
horizon dependency issue.

On Tue, Oct 10, 2017 at 2:12 AM, Christopher Hull chrishull42@gmail.com
wrote:

Hi all;

About 18 months ago I did a piece by piece install of Liberty (following
instructions on the Openstack page) and had been using it until somewhat
recently when it broke after a CentOS 7 update ran. I think Python/http is
what changed / broke (sessionless connections no longer supported?).
Basics like nova list still work, but Horizon won't run.

How can I "downgrade" my CentOS box?
Or alternatively, how painful is it to update to the most current
Openstack?

Also, I noticed that the CeotOS cloud SIG repo moved.

Please help. :-)

BTW, for installs I wrote a tool to help with setting up the many
Openstack config files. If interested, you can find it here.
http://chrishull.com/career/openstack/index.html
https://github.com/chrishull/github-openstack

Several months ago I tried to check this into the Openstack Operators
GIT repo, but had difficulty.

Anyway. Please advise on how I can fix my Liberty system if you can.

Thanks;
-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865 <(415)%20385-4865>
    chrishull42@gmail.com
    http://chrishull.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Tang Yaguang


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--


Paul Browne
Research Computing Platforms
University Information Services
Roger Needham Building
JJ Thompson Avenue
University of Cambridge
Cambridge
United Kingdom
E-Mail: pfb29@cam.ac.uk
Tel: 0044-1223-746548


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 10, 2017 by Paul_Browne (400 points)  
0 votes

Paul, Melanie,
Thanks! That worked!

-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865
    chrishull42@gmail.com
    http://chrishull.com

On Tue, Oct 10, 2017 at 10:18 AM, Paul Browne pfb29@cam.ac.uk wrote:

Hello Chris,

This may be related to a Horizon.Django issue we saw too on a RHEL7.3 to
RHEL7.4 update.

If the config is similar for CentOS7.4 packaging, if you check your
Horizon vhost under

/etc/httpd/conf.d/*-horizon_vhost.conf

on all your controllers serving Horizon there may be a WSGI directive that
needs editing;

WSGIApplicationGroup %{GLOBAL}

should go directly under the other WSGI directives in the vhost, and then
httpd restarted.

  • Paul

On 10 October 2017 at 17:59, Christopher Hull chrishull42@gmail.com
wrote:

Hello Yaguang;

Thanks very much for the assist. Curiously, no errors in horizon. But
an error upstream in httpd...

Browser hangs for a while then shows "Internal Server Error"

/var/log/httpd/error shows this...

[Tue Oct 10 09:32:53.860393 2017] [core:error] [pid 27391] [client
172.22.10.130:53239] End of script output before headers: django.wsgi

Hope this looks familiar to someone.

Thanks all;

-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865 <(415)%20385-4865>
    chrishull42@gmail.com
    http://chrishull.com

On Tue, Oct 10, 2017 at 5:47 AM, Yaguang Tang heut2008@gmail.com wrote:

Can you paste out the horizon related error logs, may be caused by
horizon dependency issue.

On Tue, Oct 10, 2017 at 2:12 AM, Christopher Hull <chrishull42@gmail.com

wrote:

Hi all;

About 18 months ago I did a piece by piece install of Liberty
(following instructions on the Openstack page) and had been using it until
somewhat recently when it broke after a CentOS 7 update ran. I think
Python/http is what changed / broke (sessionless connections no longer
supported?).
Basics like nova list still work, but Horizon won't run.

How can I "downgrade" my CentOS box?
Or alternatively, how painful is it to update to the most current
Openstack?

Also, I noticed that the CeotOS cloud SIG repo moved.

Please help. :-)

BTW, for installs I wrote a tool to help with setting up the many
Openstack config files. If interested, you can find it here.
http://chrishull.com/career/openstack/index.html
https://github.com/chrishull/github-openstack

Several months ago I tried to check this into the Openstack Operators
GIT repo, but had difficulty.

Anyway. Please advise on how I can fix my Liberty system if you can.

Thanks;
-Chris

  • Christopher T. Hull
    My contract at NASA has ended and I am seeking new opportunities.
    For updated resume and other info, please click this link.
    http://faq.chrishull.com
    Sunnyvale CA. 94085
    (415) 385 4865 <(415)%20385-4865>
    chrishull42@gmail.com
    http://chrishull.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Tang Yaguang


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--


Paul Browne
Research Computing Platforms
University Information Services
Roger Needham Building
JJ Thompson Avenue
University of Cambridge
Cambridge
United Kingdom
E-Mail: pfb29@cam.ac.uk
Tel: 0044-1223-746548


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 10, 2017 by Christopher_Hull (2,220 points)   2 5 7
...