settingsLogin | Registersettings

[openstack-dev] Status of Zuul v3

0 votes

Greetings!

Welcome to the first-ever Zuul v3 status update. :)

This periodic update is primarily intended as a way to keep
contributors to the OpenStack community apprised of Zuul v3 project
status, including future changes and milestones on our way to use in
production. Additionally, the numerous existing and future users of
Zuul outside of the OpenStack community may find this update useful as
a way to track Zuul v3 development status. This status update email is
anticipated to be sent on an approximately bi-weekly basis, though
more frequent updates may occur as we approach in-production dates.

== Wait, what’s going on? ==

Over the coming months, OpenStack’s infrastructure team will be
migrating Zuul, OpenStack’s project gating and automation system, from
Zuul v2.5 to Zuul v3. There are significant changes associated with
this migration, particularly with regards to how jobs are written, as
one of those changes is the move from JJB to Ansible. Detailed
information about Zuul v3, including specs, motivation, and community
processes, is listed below the “project status and updates” portion of
this mail.

Significant dates, milestones, and changes directly affecting
contributors in the OpenStack community will be announced on the
appropriate mailing lists for OpenStack (primarily openstack-dev and
openstack-infra), as well as in future versions of this project status
update.

And for those wondering:
* Zuul v3 is NOT YET USED IN PRODUCTION. By anyone. OpenStack’s
infrastructure team has a test deployment up and running for use in
during development (http://zuulv3-dev.openstack.org). OpenStack’s
production version of Zuul currently uses Zuul 2.5.
* No, you will not have to rewrite all your jobs. We will be
automagically converting existing jobs into Ansible jobs, and
collaborating directly with project teams in OpenStack in an orderly
fashion as we approach the move to in-production status. More on this
below.
* Yes, this email will be less verbose in the future. :)

== Zuul v3 project status and updates ==

** It’s alive! **

Last week on Tuesday at the inaugural OpenStack PTG (Feb. 20-24,
2017), for the very first time, Zuul v3 was brought online in a test
environment, and a simple hello world job was successfully executed on
a single node. That job was immediately followed by a successful
“hello worlds” job on multiple nodes. That’s right: Zuul v3 went from
single-node to multi-node functionality in a matter of minutes! There
was much rejoicing! A super exciting time for the folks who have been
working towards this moment. (By the way: if you, fine reader, know
anyone who has been hard at work on Zuul v3, send them your
congratulations. It’s a Big Deal!)

Even though the Infra team’s official days at the PTG were Monday and
Tuesday, folks continued hacking on Zuul throughout the rest of the
week, getting general tox-based jobs running, and enabling the ability
to collect and publish logs.

Monty Taylor, known to many of us as mordred, detailed these
accomplishments, and outlined the bigger picture of the future of
Zuul, in a blog post this week. For the extra-curious, since folks
have asked, "What does a job look when we are using Ansible?" -- he
has also graciously provided some example code snippets. Read more
here: http://inaugust.com/posts/whats-coming-zuulv3.html

Also,

** FULL DISCLAIMER **

This may be obvious to some readers, given that this is a progress and
status update email, but for all the aspirational, daring folks out
there:

At this point, Zuul v3 is NOT yet secure or stable. You should not,
not, please, DO NOT ATTEMPT TO RUN IT IN PRODUCTION YOURSELF yet. It
is still under heavy development, and v3 is not currently used in
production for OpenStack. Zuul v3 content (jobs, etc.) should not
undergo heavy creation as syntax is still under development and
expected to change between now and release date.

We know you’re excited. We are too. Just be patient. The time will come. :)

Speaking of when that time will come, many folks have asked: When is
that time actually coming?

Commitment is hard, you see, especially when making it more awesome
for people to commit code :) The infra team expects this migration to
happen in the next few months. In the meantime: please know that the
infrastructure team intends to communicate milestones, information,
and significant dates of change to the openstack-dev mailing list, as
they would any other significant project infrastructure-related
changes. As we move closer to rolling out Zuul v3, significant dates
to be aware of, as well as reminders, will be announced broadly, and
we will be working with individual project teams to assist them in
migration, including the scripted conversion of existing jobs from JJB
to Ansible. The vast quantity of documentation in OpenStack that
directly or indirectly relates to Zuul, including creating guides for
writing future jobs, will also be updated.

Upcoming tasks and focus:
* Config syntax, including reporting configuration failures, and
taking care of any outstanding tasks that will affect the config
language.
* Some additional near-term tasks are listed in the Zuul PTG etherpad:
https://etherpad.openstack.org/p/pike-ptg-zuul
* Full task list and plan is in the Zuul v3 storyboard:
https://storyboard.openstack.org/#!/board/41

IRC Meeting minutes & logs (2017-02-27):
* Minutes: http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-02-27-22.03.html
* Full log: http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-02-27-22.03.log.html

== About Zuul and Zuul v3 ==
Zuul is a pipeline-oriented project gating system, driving the project
automation necessary to enable a continuous integration environment
for the OpenStack community. It directs the OpenStack community’s
testing, running tens of thousands of jobs each day, responding to
events from the code review system and stacking potential changes to
be tested together. Testing and continuous integration for drivers is
also enabled by OpenStack’s deployment of Zuul for ~50 third-party
projects.

Zuul v3 is the upcoming, not-yet-in-production version of Zuul
currently under development, bringing (amongst many others) the
following improvements and features:
* simplifying the ability to run jobs in multi-node environments
* improved management of large numbers of jobs and job variations
* support for in-tree job configuration
* ability to define jobs using Ansible (http://github.com/ansible/ansible)

Even though prior versions of Zuul are already in use by numerous
projects and companies not related to OpenStack efforts, a primary
goal of Zuul v3 is to make Zuul a universally useful CI / CD engine
for any size use case. The design of Zuul v3 improves the modularity
of the system, and enables new triggers (such as GitHub) and reporters
(where results are sent) to be more easily integrated with Zuul;
additionally, the ability to execute jobs on non-OpenStack clouds,
such as, well, the other ones :D, as well as non-cloud environments,
will be possible. (These "beyond OpenStack" changes are expected in
Zuul 3.1, which will follow the moment after Zuul v3 is live and in
production for the OpenStack community. Finally, the move from Jenkins
to Ansible, in addition to improving scalability for large
implementations, allows users to write tests using Ansible,
effectively allowing one to “test in the way that you deploy,” and
enables simpler writing of test jobs.

OpenStack’s current production use of Zuul supports approximately 2500
developers, more than 1500 git repositories, at a rate of more than
2KJPH (kilo-jobs / hour).

For more background and information on the history, and future, of
Zuul, please see:
* Zuul v3 spec:
https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
* Jim Blair’s Zuul v3 talk at OpenStack Summit in Barcelona:
https://www.youtube.com/watch?v=R4EmE1QEvNU
* “There is no Jenkins, only Zuul” (blog post, Jim Blair, 2016/06/16):
http://amo-probos.org/post/18
* "Zuul v3, What’s Coming" (blog post, Monty Taylor, 2017/02/28):
http://inaugust.com/posts/whats-coming-zuulv3.html

== Zuul Community Information ==

Zuul v3 Storyboard, which tracks tasks and status related to operating
Zuul v3 in production for OpenStack:
https://storyboard.openstack.org/#!/board/41

Mailing list: Discussions related to Zuul development, and current /
future operation in production in the OpenStack community, primarily
occur on openstack-infra
(http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra).
(Note that this mailing list encompasses the numerous infrastructure
projects supported by OpenStack’s infrastructure team, and is not
exclusively Zuul-focused.)

IRC Discussion:
Join #openstack-infra on Freenode to discuss Zuul's use in OpenStack,
or #zuul to get into the weeds on general Zuul topics and v3
development. For those unacquainted with IRC, see this friendly guide
to using IRC: https://wiki.openstack.org/wiki/IRC

Zuul v3 Weekly IRC meeting:
* Mondays at 2200UTC, in #openstack-meeting-alt on Freenode
* Historical meeting logs: http://eavesdrop.openstack.org/meetings/zuul/
* Further information, including upcoming meeting agenda:
https://wiki.openstack.org/wiki/Meetings/Zuul

Cheers,

-Robyn (on behalf of the openstack-infra team)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Mar 1, 2017 in openstack-dev by Robyn_Bergeron (160 points)  

1 Response

0 votes

Greetings!

This periodic update is primarily intended as a way to keep
contributors to the OpenStack community apprised of Zuul v3 project
status, including future changes and milestones on our way to use in
production. Additionally, the numerous existing and future users of
Zuul outside of the OpenStack community may find this update useful as
a way to track Zuul v3 development status.

If "changes are coming in the land of Zuul" is new news to you, please
read the section "About Zuul and Zuul v3" towards the end of this
email.

== Zuul v3 project status and updates ==

The Big Big news: Updates to Nodepool to support Zuul v3 are done!
This has been a large effort (approximately the size of one Stay-Puft
marshmallow man), and the team is super excited to now have this in
place. Give shrews a high-five if you see him! We still have bugs to
shake out, documentation to update, and a backwards-incompatible
configuration syntax change to make
(http://lists.openstack.org/pipermail/openstack-infra/2017-January/005018.html),
but we can consider it feature complete.

Continuing to work on and discuss sample jobs: Putting together an
ideal "sample job" and documenting what best practices look like is
essential in enabling future users of Zuul to quickly and easily
create jobs of their own. Paul Belanger has been making progress on
this in the form of a generic tox job (see
https://review.openstack.org/438281), and we're now at the point of
discussing why / how this is a foundational example.

Additionally, as more of the major subsystems of Zuul complete their
refactoring, the ability for one to contribute to Zuul v3 continues to
get a bit easier. Even though at the moment we still recommend having
deep familiarity with one or more of the subsystems, we do now have
some "low-hanging fruit" tasks listed in storyboard. Use this magical
link to find your way:
https://storyboard.openstack.org/#!/story/list?status=active&tags=low-hanging-fruit&project_id=679

Some new specs for enhancements and/or features have been put forth as well:
* An interface for Zuul Job Reporting https://review.openstack.org/444088
* Zuulv3 Executor Security Enhancement https://review.openstack.org/444495
* Update job trees to graphs https://review.openstack.org/443985 (yes,
technically this is an update to a spec, not a new one.)

Upcoming tasks and focus:
* Re-enabling disabled tests: We're continuing to make our way through
the list of remaining tests that need enabling. See the list, which
includes an annotation as to complexity for each test, here:
https://etherpad.openstack.org/p/zuulv3skips
* Full task list and plan is in the Zuul v3 storyboard:
https://storyboard.openstack.org/#!/board/41

Recent changes:
* Zuul v3: https://review.openstack.org/#/q/status:closed+project:openstack-infra/zuul+branch:feature/zuulv3,25
(Early adoptors should be aware we renamed zuul-launcher to
zuul-executor, this is a breaking change:
https://review.openstack.org/#/c/445594)
* Nodepool: https://review.openstack.org/#/q/status:closed+project:openstack-infra/nodepool+branch:feature/zuulv3,25

Previous IRC Meeting minutes & logs:
* 2017-03-06 Minutes:
http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-06-22.03.html
* 2017-03-06 Full log:
http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-06-22.03.log.html
* 2017-03-13 Minutes:
http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-13-22.02.html
* 2017-03-13 Full log:
http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-13-22.02.log.html

== About Zuul and Zuul v3 ==
Zuul is a pipeline-oriented project gating system, driving the project
automation necessary to enable a continuous integration environment
for the OpenStack community. It directs the OpenStack community’s
testing, running tens of thousands of jobs each day, responding to
events from the code review system and stacking potential changes to
be tested together. Testing and continuous integration for drivers is
also enabled by OpenStack’s deployment of Zuul for ~50 third-party
projects.

Zuul v3 is the upcoming, not-yet-in-production version of Zuul
currently under development, bringing (amongst many others) the
following improvements and features:
* simplifying the ability to run jobs in multi-node environments
* improved management of large numbers of jobs and job variations
* support for in-tree job configuration
* ability to define jobs using Ansible (http://github.com/ansible/ansible)

Even though prior versions of Zuul are already in use by numerous
projects and companies not related to OpenStack efforts, a primary
goal of Zuul v3 is to make Zuul a universally useful CI / CD engine
for any size use case. The design of Zuul v3 improves the modularity
of the system, and enables new triggers (such as GitHub) and reporters
(where results are sent) to be more easily integrated with Zuul;
additionally, the ability to execute jobs on non-OpenStack clouds,
such as, well, the other ones :D, as well as non-cloud environments,
will be possible. (These "beyond OpenStack" changes are expected in
Zuul 3.1, which will follow the moment after Zuul v3 is live and in
production for the OpenStack community. Finally, the move from Jenkins
to Ansible, in addition to improving scalability for large
implementations, allows users to write tests using Ansible,
effectively allowing one to “test in the way that you deploy,” and
enables simpler writing of test jobs.

OpenStack’s current production use of Zuul supports approximately 2500
developers, more than 1500 git repositories, at a rate of more than
2KJPH (kilo-jobs / hour).

For more background and information on the history, and future, of
Zuul, please see:
* Zuul v3 spec:
https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html
* Jim Blair’s Zuul v3 talk at OpenStack Summit in Barcelona:
https://www.youtube.com/watch?v=R4EmE1QEvNU
* “There is no Jenkins, only Zuul” (blog post, Jim Blair, 2016/06/16):
http://amo-probos.org/post/18
* "Zuul v3, What’s Coming" (blog post, Monty Taylor, 2017/02/28):
http://inaugust.com/posts/whats-coming-zuulv3.html

Significant dates, milestones, and changes directly affecting
contributors in the OpenStack community will be announced on the
appropriate mailing lists for OpenStack (primarily openstack-dev and
openstack-infra), as well as in future versions of this project status
update.

NOTE: At this point, Zuul v3 is NOT yet secure or stable. You should
not, not, please, DO NOT ATTEMPT TO RUN IT IN PRODUCTION YOURSELF yet.
It is still under heavy development, and v3 is not currently used in
production for OpenStack. Zuul v3 content (jobs, etc.) should not
undergo heavy creation as syntax is still under development and
expected to change between now and release date.

== Zuul Community Information ==

Zuul v3 Storyboard, which tracks tasks and status related to operating
Zuul v3 in production for OpenStack:
https://storyboard.openstack.org/#!/board/41

Mailing list: Discussions related to Zuul development, and current /
future operation in production in the OpenStack community, primarily
occur on openstack-infra
(http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra).
(Note that this mailing list encompasses the numerous infrastructure
projects supported by OpenStack’s infrastructure team, and is not
exclusively Zuul-focused.)

IRC Discussion:
Join #openstack-infra on Freenode to discuss Zuul's use in OpenStack
or #zuul to get into the weeds on general Zuul topics and v3
development. For those unacquainted with IRC, see this friendly guide
to using IRC: https://wiki.openstack.org/wiki/IRC

Zuul v3 Weekly IRC meeting:
* Mondays at 2200UTC, in #openstack-meeting-alt on Freenode
* Historical meeting logs: http://eavesdrop.openstack.org/meetings/zuul/
* Further information, including upcoming meeting agenda:
https://wiki.openstack.org/wiki/Meetings/Zuul

Cheers,

-Robyn (on behalf of the openstack-infra team)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 16, 2017 by Robyn_Bergeron (160 points)  
...