settingsLogin | Registersettings

[openstack-dev] [openstack][zuul] About devstack plugin orders and the log to contain the running local.conf

0 votes

Hi, everyone
I am trying to migrate tacker's functional CI job into new zuul v3 framework, but it seems:
1. the devstack plugin order is not the one I specified in the .zuull.yaml
https://review.openstack.org/#/c/516004/4/.zuul.yaml:I have:      devstack_plugins:
        heat: https://git.openstack.org/openstack/heat
        networking-sfc:  https://git.openstack.org/openstack/networking-sfc
        aodh: https://git.openstack.org/openstack/aodh
        ceilometer: https://git.openstack.org/openstack/ceilometer
        barbican: https://git.openstack.org/openstack/barbican
        mistral: https://git.openstack.org/openstack/mistral
        tacker: https://git.openstack.org/openstack/tacker
but the running order seems:http://logs.openstack.org/04/516004/4/check/tacker-functional-devstack/f365f21/job-output.txt.gz:
local plugins=,ceilometer,aodh,mistral,networking-sfc,heat,tacker,barbican
I need barbican to start before tacker.

  1. I cannot see the whole local.conf in the log dir
    the running local.conf is not shown in:http://logs.openstack.org/04/516004/4/check/tacker-functional-devstack/f365f21/controller/logs/

Do anyone help me to find how to fix the problem at https://review.openstack.org/#/c/516004?

thanks
yong sheng gongTacker team__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

asked Oct 30, 2017 in openstack-dev by gong_ys2004 (180 points)   1

3 Responses

0 votes

On Sat, Oct 28, 2017 at 8:25 PM, gong_ys2004 gong_ys2004@aliyun.com wrote:
2. I cannot see the whole local.conf in the log dir

the running local.conf is not shown in:
http://logs.openstack.org/04/516004/4/check/tacker-functional-devstack/f365f21/controller/logs/

Do anyone help me to find how to fix the problem at
https://review.openstack.org/#/c/516004?

I have a DevStack review[0] to grab local.conf and set the groundwork
for the other config files we used to get; ianw made some good
suggestions that I am not going to get to for a day or three (flying
to SYD tomorrow), feel free to take it over if you want to finish it
up before then.

dt

[0] https://review.openstack.org/#/c/515209/

--

Dean Troyer
dtroyer@gmail.com


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 29, 2017 by Dean_Troyer (13,100 points)   1 3 3
0 votes

On Sun, Oct 29, 2017 at 6:55 AM, gong_ys2004 gong_ys2004@aliyun.com wrote:
Hi, everyone

I am trying to migrate tacker's functional CI job into new zuul v3
framework, but it seems:

  1. the devstack plugin order is not the one I specified in the .zuull.yaml

https://review.openstack.org/#/c/516004/4/.zuul.yaml:
I have:
devstack_plugins:
heat: https://git.openstack.org/openstack/heat
networking-sfc: https://git.openstack.org/openstack/networking-sfc
aodh: https://git.openstack.org/openstack/aodh
ceilometer: https://git.openstack.org/openstack/ceilometer
barbican: https://git.openstack.org/openstack/barbican
mistral: https://git.openstack.org/openstack/mistral
tacker: https://git.openstack.org/openstack/tacker

but the running order seems:
http://logs.openstack.org/04/516004/4/check/tacker-functional-devstack/f365f21/job-output.txt.gz:

local plugins=,ceilometer,aodh,mistral,networking-sfc,heat,tacker,barbican

I need barbican to start before tacker.

To me it looks like the issue is because the way "devstack_plugins" is
being handled, it's being defined/handled(see [1]) as Dictionary.
Since dictionary order is not sequetial, the generated order would be
random.
Shouldn't devstack plugins be defined/handled as a list of items as
it's order matters?

[1] https://github.com/openstack-dev/devstack/blob/36ddea31a257b38aa22d3232f4c4389c851f5456/roles/write-devstack-local-conf/library/devstack_local_conf.py#L122-L124

  1. I cannot see the whole local.conf in the log dir

the running local.conf is not shown in:
http://logs.openstack.org/04/516004/4/check/tacker-functional-devstack/f365f21/controller/logs/

Do anyone help me to find how to fix the problem at
https://review.openstack.org/#/c/516004?

thanks

yong sheng gong
Tacker team


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 29, 2017 by ykarel_at_redhat.com (180 points)  
0 votes

"gong_ys2004" gong_ys2004@aliyun.com writes:

Hi, everyone
I am trying to migrate tacker's functional CI job into new zuul v3 framework, but it seems:
1. the devstack plugin order is not the one I specified in the .zuull.yaml
https://review.openstack.org/#/c/516004/4/.zuul.yaml:I have:      devstack_plugins:
        heat: https://git.openstack.org/openstack/heat
        networking-sfc:  https://git.openstack.org/openstack/networking-sfc
        aodh: https://git.openstack.org/openstack/aodh
        ceilometer: https://git.openstack.org/openstack/ceilometer
        barbican: https://git.openstack.org/openstack/barbican
        mistral: https://git.openstack.org/openstack/mistral
        tacker: https://git.openstack.org/openstack/tacker
but the running order seems:http://logs.openstack.org/04/516004/4/check/tacker-functional-devstack/f365f21/job-output.txt.gz:
local plugins=,ceilometer,aodh,mistral,networking-sfc,heat,tacker,barbican
I need barbican to start before tacker.

[I changed the subject to replace the 'openstack' tag with 'devstack',
which is what I assume was intended.]

As Yatin Karel later notes, this is handled as a regular python
dictionary which means we process the keys in an indeterminate order.

I can think of a few ways we can address this:

1) Add dependency information to devstack plugins so that devstack
itself is able to work out the correct order. This is perhaps the ideal
solution from a user experience perspective, but perhaps the most
difficult.

2) Add dependency information to the Ansible role so that it resolves
the order on its own. This is attractive because it solves a problem
that is unique to this Ansible role entirely within the role. However,
it means that new plugins would need to also update this role which is
in devstack itself, which partially defeats the purpose of plugins.

3) Add dependency information to devstack plugins, but rather than
having devstack resolve it, have the Ansible role which writes out the
local.conf read that information and resolve the order. This lets us
keep the actual information in plugins so we don't have to continually
update the role, but it lets us perform the processing in the role
(which is in Python) when writing the config file.

4) Alter Zuul's handling of this to an ordered dictionary. Then when
you specify a series of plugins, they would be processed in that order.
However, I'm not sure this works very well with Zuul job inheritance.
Imagine that a parent job enabled the barbican plugin, and a child job
enabled ceilometer, needed ceilometer to start before barbican. There
would be no way to express that.

5) Change the definition of the dictionary to encode ordering
information. Currently the dictionary schema is simply the name of the
plugin as the key, and either the contents of the "enable_plugin" line,
or "null" if the plugin should be disabled. We could alter it to be:

devstack_plugins:
barbican:
enabled: true
url: https://git.openstack.org/openstack/barbican
branch: testing
tacker:
enabled: true
url: https://git.openstack.org/openstack/tacker
requires:
barbican: true

This option is very flexible, but makes using the jobs somewhat more
difficult because of the complexity of the data structure.

After considering all of those, I think I favor option 3, because we
should be able to implement it without too much difficulty, it will
improve things by providing a known and documented location for plugins
to specify dependencies, and once it is in place, we can still implement
option 1 later if we want, using the same declaration.

-Jim


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Oct 30, 2017 by James_E._Blair (5,080 points)   1 5 7
...