settingsLogin | Registersettings

[openstack-dev] [neutron][lbaas] Pool member status 'ACTIVE' even on health check failure

0 votes

Hi All,

I am trying to get LBaaS running on stable Juno. I can get all the LBaaS components correctly installed and working as expected. But I am facing some issues with the health-monitor. I am not quite sure if it?s working as expected.

I have 2 ubuntu servers as members of http-pool and I have stopped apache process on 1 of the servers. I have HTTP health-monitor configured on the pool which runs every 1 min and checks for 200 response code on HTTP GET. I was expecting it to FAIL after 3 retries and make the status ?INACTIVE? for the member where apache is not running. But for some reason, it?s always ACTIVE.

Can somebody help me with how is it suppose to work and if it?s a bug?

Also, currently I don?t see any health monitor stats with neutron. Is there any plan to get health monitor stats in future releases?

Thanks,
Varun
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openstack.org/pipermail/openstack-dev/attachments/20150119/a8d1bd7a/attachment.html

asked Jan 19, 2015 in openstack-dev by Varun_Lodaya (440 points)   2 3
retagged Jan 28, 2015 by admin

3 Responses

0 votes

Hi Varun,

Could you tell me which driver you are using? If you're running the
HaproxyOnHostPluginDriver then that should do a check every 6 seconds
for members being down. However, other drivers may not do this. It's
up to the driver.

As for providing health monitor stats, those currently are not being
provided. There haven't been any plans for that yet because everyone
has been focused on getting the v2 API out. Which is almost complete
and plan for that to be completed for Kilo-3. If you'd like to be able
to retrieve some health stats, please list them and let us know. We'll
hopefully be able to get them in after v2 has completed.

Thanks,
Brandon

On Mon, 2015-01-19 at 14:42 -0800, Varun Lodaya wrote:
Hi All,

I am trying to get LBaaS running on stable Juno. I can get all the
LBaaS components correctly installed and working as expected. But I am
facing some issues with the health-monitor. I am not quite sure if
it?s working as expected.

I have 2 ubuntu servers as members of http-pool and I have stopped
apache process on 1 of the servers. I have HTTP health-monitor
configured on the pool which runs every 1 min and checks for 200
response code on HTTP GET. I was expecting it to FAIL after 3 retries
and make the status ?INACTIVE? for the member where apache is not
running. But for some reason, it?s always ACTIVE.

Can somebody help me with how is it suppose to work and if it?s a bug?

Also, currently I don?t see any health monitor stats with neutron. Is
there any plan to get health monitor stats in future releases?

Thanks,
Varun


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 20, 2015 by Brandon_Logan (9,580 points)   1 2 5
0 votes

Hey Brandon,

Thanks for the response. My bad. Seems there is a small bug in horizon.
The moment you configure a health monitor, it shows up in the pool. I
thought it automatically got associated. But when I checked via CLI, it
was not. After associating it via CLI (not able to associate it via
horizon, the drop down for health-monitors doesn?t seem to work), it seems
to work fine :).

As per stats, ideally, it?s good to get counters like:
ICMP successful requests: x
ICMP response timeouts: y
ICMP response failures: z

HTTP successful responses: a
HTTP timeouts: b
.
.
.

Just an initial thought, this sort of verifies that monitors are working
as expected. Like in current situation, I had to manually login to the
server to see if the server is catering to any health-monitoring requests.

Even getting haproxy stats is not very straightforward, as you need to
open a unix socket in haproxy cfg and restart the haproxy instance which
might not be possible in production sometimes.

Thanks,
Varun

On 1/19/15, 8:21 PM, "Brandon Logan" <brandon.logan at RACKSPACE.COM> wrote:

Hi Varun,

Could you tell me which driver you are using? If you're running the
HaproxyOnHostPluginDriver then that should do a check every 6 seconds
for members being down. However, other drivers may not do this. It's
up to the driver.

As for providing health monitor stats, those currently are not being
provided. There haven't been any plans for that yet because everyone
has been focused on getting the v2 API out. Which is almost complete
and plan for that to be completed for Kilo-3. If you'd like to be able
to retrieve some health stats, please list them and let us know. We'll
hopefully be able to get them in after v2 has completed.

Thanks,
Brandon

On Mon, 2015-01-19 at 14:42 -0800, Varun Lodaya wrote:

Hi All,

I am trying to get LBaaS running on stable Juno. I can get all the
LBaaS components correctly installed and working as expected. But I am
facing some issues with the health-monitor. I am not quite sure if
it?s working as expected.

I have 2 ubuntu servers as members of http-pool and I have stopped
apache process on 1 of the servers. I have HTTP health-monitor
configured on the pool which runs every 1 min and checks for 200
response code on HTTP GET. I was expecting it to FAIL after 3 retries
and make the status ?INACTIVE? for the member where apache is not
running. But for some reason, it?s always ACTIVE.

Can somebody help me with how is it suppose to work and if it?s a bug?

Also, currently I don?t see any health monitor stats with neutron. Is
there any plan to get health monitor stats in future releases?

Thanks,
Varun


_
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 20, 2015 by Varun_Lodaya (440 points)   2 3
0 votes

Yeah before we get those stats in we'll need to finalize v2 because that
will affect how the API shows those types of stats to the user.

On Mon, 2015-01-19 at 22:33 -0800, Varun Lodaya wrote:
Hey Brandon,

Thanks for the response. My bad. Seems there is a small bug in horizon.
The moment you configure a health monitor, it shows up in the pool. I
thought it automatically got associated. But when I checked via CLI, it
was not. After associating it via CLI (not able to associate it via
horizon, the drop down for health-monitors doesn?t seem to work), it seems
to work fine :).

As per stats, ideally, it?s good to get counters like:
ICMP successful requests: x
ICMP response timeouts: y
ICMP response failures: z

HTTP successful responses: a
HTTP timeouts: b
.
.
.

Just an initial thought, this sort of verifies that monitors are working
as expected. Like in current situation, I had to manually login to the
server to see if the server is catering to any health-monitoring requests.

Even getting haproxy stats is not very straightforward, as you need to
open a unix socket in haproxy cfg and restart the haproxy instance which
might not be possible in production sometimes.

Thanks,
Varun

On 1/19/15, 8:21 PM, "Brandon Logan" <brandon.logan at RACKSPACE.COM> wrote:

Hi Varun,

Could you tell me which driver you are using? If you're running the
HaproxyOnHostPluginDriver then that should do a check every 6 seconds
for members being down. However, other drivers may not do this. It's
up to the driver.

As for providing health monitor stats, those currently are not being
provided. There haven't been any plans for that yet because everyone
has been focused on getting the v2 API out. Which is almost complete
and plan for that to be completed for Kilo-3. If you'd like to be able
to retrieve some health stats, please list them and let us know. We'll
hopefully be able to get them in after v2 has completed.

Thanks,
Brandon

On Mon, 2015-01-19 at 14:42 -0800, Varun Lodaya wrote:

Hi All,

I am trying to get LBaaS running on stable Juno. I can get all the
LBaaS components correctly installed and working as expected. But I am
facing some issues with the health-monitor. I am not quite sure if
it?s working as expected.

I have 2 ubuntu servers as members of http-pool and I have stopped
apache process on 1 of the servers. I have HTTP health-monitor
configured on the pool which runs every 1 min and checks for 200
response code on HTTP GET. I was expecting it to FAIL after 3 retries
and make the status ?INACTIVE? for the member where apache is not
running. But for some reason, it?s always ACTIVE.

Can somebody help me with how is it suppose to work and if it?s a bug?

Also, currently I don?t see any health monitor stats with neutron. Is
there any plan to get health monitor stats in future releases?

Thanks,
Varun


_
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jan 20, 2015 by Brandon_Logan (9,580 points)   1 2 5
...