settingsLogin | Registersettings

[openstack-dev] [stacktach] [oslo] stachtach -> kombu -> pika ??

0 votes

Hi all,

I saw the following on
https://etherpad.openstack.org/p/PHL-ops-rabbit-queue and was wondering
if there was more explanation of why?

The StackTach team is switching from Kombu to Pika (at the
recommendation of core rabbitmq devs). Hopefully oslo-messaging will do
the same.

I'm wondering why/what?

Pika seems to be less supported, has less support for things other than
rabbitmq, and seems less developed (it lacks python 3.3 support apparently).

What's the details on this idea listed there?

Any stachtack folks got any more details?

-Josh


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Mar 10, 2015 in openstack-dev by Joshua_Harlow (12,560 points)   1 4 4

6 Responses

0 votes

Hey J,

Our (old) notification consumer was using carrot, which is dead but worked. Lately though there have been conflicts with carrot and msgpack, so we had to change. Around the same time, we ran into a bug where we were writing to an unnamed exchange (completely valid, but too easy to do under kombu). During that debug process we ended up chatting with the rabbitmq core devs and they strongly recommended we use Pika instead of Kombu. We're going to follow that advice.

We're going to be adding support for consuming from and writing to Kafka as well and will likely use a kafka-specific library for that too.

You can ignore my hastily-written comment about oslo-messaging considering it. It's probably not important for your use-cases.

Sorry for any confusion this may have caused.

-S


From: Joshua Harlow harlowja@outlook.com
Sent: Tuesday, March 10, 2015 1:56 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [stacktach] [oslo] stachtach -> kombu -> pika ??

Hi all,

I saw the following on
https://etherpad.openstack.org/p/PHL-ops-rabbit-queue and was wondering
if there was more explanation of why?

The StackTach team is switching from Kombu to Pika (at the
recommendation of core rabbitmq devs). Hopefully oslo-messaging will do
the same.

I'm wondering why/what?

Pika seems to be less supported, has less support for things other than
rabbitmq, and seems less developed (it lacks python 3.3 support apparently).

What's the details on this idea listed there?

Any stachtack folks got any more details?

-Josh


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 10, 2015 by Sandy_Walsh (3,660 points)   2 3
0 votes

We're going to be adding support for consuming from and writing to Kafka as well and will likely use a kafka-specific library for that too.
is the plan to add this support to oslo.messaging? i believe there is interest from the oslo.messaging team in supporting Kafka and in addition to adding a Kafka publisher in ceilometer, there is suppose to be a bp to add support to oslo.messaging.
cheers,
gord __________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Mar 10, 2015 by gordon_chung (19,300 points)   2 3 8
0 votes

Maybe the plan for oslo.messaging should be to make it resolve some of
the operators issues first ;-)

https://etherpad.openstack.org/p/PHL-ops-large-deployments

I'd rather think we should like ummm, be thinking about fixing issues
instead of adding new things to oslo.messaging. IMHO let some other
project be the playground for these things (kombu, other...)...

-Josh

gordon chung wrote:

We're going to be adding support for consuming from and writing to
Kafka as well and will likely use a kafka-specific library for that too.

is the plan to add this support to oslo.messaging? i believe there is
interest from the oslo.messaging team in supporting Kafka and in
addition to adding a Kafka publisher in ceilometer, there is suppose to
be a bp to add support to oslo.messaging.

cheers,
/gord/


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 10, 2015 by Joshua_Harlow (12,560 points)   1 4 4
0 votes

No, we're adding this to Yagi first and perhaps Notabene later. We don't need rpc support, so it's too big a change for us to take on.


From: gordon chung gord@live.ca
Sent: Tuesday, March 10, 2015 3:58 PM
To: OpenStack Development Mailing List not for usage questions
Subject: Re: [openstack-dev] [stacktach] [oslo] stachtach -> kombu -> pika ??

We're going to be adding support for consuming from and writing to Kafka as well and will likely use a kafka-specific library for that too.

is the plan to add this support to oslo.messaging? i believe there is interest from the oslo.messaging team in supporting Kafka and in addition to adding a Kafka publisher in ceilometer, there is suppose to be a bp to add support to oslo.messaging.

cheers,
gord


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 10, 2015 by Sandy_Walsh (3,660 points)   2 3
0 votes

On 03/10/2015 12:56 PM, Joshua Harlow wrote:
Hi all,

I saw the following on
https://etherpad.openstack.org/p/PHL-ops-rabbit-queue and was wondering
if there was more explanation of why?

The StackTach team is switching from Kombu to Pika (at the
recommendation of core rabbitmq devs). Hopefully oslo-messaging will do
the same.

I'm wondering why/what?

Pika seems to be less supported, has less support for things other than
rabbitmq, and seems less developed (it lacks python 3.3 support
apparently).

So, something that came up was the supporting anything other MQ system
than Rabbit seemed irrelevant to the operator community. Rabbit is bad,
all others are demonstrably worse. Anything that makes the Rabbit path
better is good, anything that could regress the Rabbit path for the sake
of any of the other backends is bad (and also not useful to deployers).

What's the details on this idea listed there?

Any stachtack folks got any more details?

-Josh


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Sean Dague
http://dague.net


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 12, 2015 by Sean_Dague (66,200 points)   4 8 14
0 votes


From: Sean Dague sean@dague.net
Sent: Thursday, March 12, 2015 9:09 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [stacktach] [oslo] stachtach -> kombu -> pika ??

On 03/10/2015 12:56 PM, Joshua Harlow wrote:

I saw the following on
https://etherpad.openstack.org/p/PHL-ops-rabbit-queue and was wondering
if there was more explanation of why?

The StackTach team is switching from Kombu to Pika (at the
recommendation of core rabbitmq devs). Hopefully oslo-messaging will do
the same.

I'm wondering why/what?

Pika seems to be less supported, has less support for things other than
rabbitmq, and seems less developed (it lacks python 3.3 support
apparently).

So, something that came up was the supporting anything other MQ system
than Rabbit seemed irrelevant to the operator community. Rabbit is bad,
all others are demonstrably worse. Anything that makes the Rabbit path
better is good, anything that could regress the Rabbit path for the sake
of any of the other backends is bad (and also not useful to deployers).

True, and supporting Pika doesn't have to mean dumping Kombu. It could just
be a new driver. If Pika gives better stability or performance to the most
popular OpenStack MQ perhaps it's something to consider?


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 16, 2015 by Sandy_Walsh (3,660 points)   2 3
...