settingsLogin | Registersettings

[openstack-dev] [all] 3rd Party CI vs. Gerrit

0 votes

corvus at inaugust.com (James E. Blair) writes:

Sean Dague writes:

This has all gone far enough that someone actually wrote a Grease Monkey
script to purge all the 3rd Party CI content out of Jenkins UI. People
are writing mail filters to dump all the notifications. Dan Berange
filters all them out of his gerrit query tools.

I should also mention that there is a pending change to do something
similar via site-local Javascript in our Gerrit:

https://review.openstack.org/#/c/95743/

I don't think it's an ideal long-term solution, but if it works, we may
have some immediate relief without all having to install greasemonkey
scripts.

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

The table only includes CI systems that leave their results in the same
format that we use for Jenkins; we will update the recommendations for
third party CI systems to encourage the use of that format.

This is all still fairly brittle, based mostly on javascript-powered
screen scraping. However, I'm hoping we can get something like it in a
Gerrit plugin for a more long-term solution to the problem.

Thanks again to Radoslav Gerganov for writing the original change.

-Jim

asked Aug 13, 2014 in openstack-dev by James_E._Blair (5,080 points)   1 6 8
retagged Feb 25, 2015 by admin

12 Responses

0 votes

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

Friggin. Awesome.

Thanks again to Radoslav Gerganov for writing the original change.

Thanks to all involved, as this is a major improvement for everyone!

One thing that we discussed at the nova meetup was having a space for
each CI we expect to vote. I haven't looked at the implementation
here, but I assume it just parses the comments to generate the table.
How hard would it be to make the table show all the CI systems we expect
so that it's very obvious that one has gone MIA (as they often do)
before we merge a patch? I think we struggle right now with merging
things that a CI system would have NAKed, but only because we didn't
notice that it hadn't voted.

--Dan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 538 bytes
Desc: OpenPGP digital signature
URL:

responded Aug 13, 2014 by Dan_Smith (9,860 points)   1 2 4
0 votes

Dan Smith writes:

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

Friggin. Awesome.

Thanks again to Radoslav Gerganov for writing the original change.

Thanks to all involved, as this is a major improvement for everyone!

One thing that we discussed at the nova meetup was having a space for
each CI we expect to vote. I haven't looked at the implementation
here, but I assume it just parses the comments to generate the table.
How hard would it be to make the table show all the CI systems we expect
so that it's very obvious that one has gone MIA (as they often do)
before we merge a patch? I think we struggle right now with merging
things that a CI system would have NAKed, but only because we didn't
notice that it hadn't voted.

I think my preferred method of doing this would be to drive all
third-party CI systems from OpenStack's Zuul. We are not far from being
able to do that technically, though it's not clear to me that there is
the will for third party CI systems to do that. However, if we really
are expecting them to report back and are willing to hold up merging a
change because of it, then we really should consider that.

As we look into using a Gerrit plugin for test results, we could see
about adding that as a feature.

Something that we could technically do immediately would be to add
third-party CI systems as reviewers to changes when they are uploaded.
Then they would show up in the list of reviewers at the top of the
change. This would require maintaining a list of which systems were
expected to vote on which repositories. Rather than keeping that in
git, we could use group membership for it (and implement something we
have talked about off-and-on, which is using Gerrit group membership to
grant voting privileges to each individual repository). That would also
allow projects to self-manage both who is permitted to vote on their
repos and who is expected to vote.

-Jim

responded Aug 13, 2014 by James_E._Blair (5,080 points)   1 6 8
0 votes

On Wed, Aug 13, 2014 at 3:05 PM, James E. Blair wrote:

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

Very cool!! this is really nice UI, super useful....

one litle suggestions for the folk that knows how to do that if that's
possible to do is to sort between the voting and the non-voting so we can
easily spot which one are worthwhile to look at or not.

Chmouel
-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 13, 2014 by chmouel_at_enovance. (2,660 points)   2 2
0 votes

On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote:
corvus at inaugust.com (James E. Blair) writes:

Sean Dague writes:

This has all gone far enough that someone actually wrote a Grease Monkey
script to purge all the 3rd Party CI content out of Jenkins UI. People
are writing mail filters to dump all the notifications. Dan Berange
filters all them out of his gerrit query tools.

I should also mention that there is a pending change to do something
similar via site-local Javascript in our Gerrit:

https://review.openstack.org/#/c/95743/

I don't think it's an ideal long-term solution, but if it works, we may
have some immediate relief without all having to install greasemonkey
scripts.

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

Beautiful! Thank you so much to everyone involved.

Mark.

responded Aug 13, 2014 by Mark_McLoughlin (5,180 points)   1 4 6
0 votes

Chmouel Boudjnah writes:

On Wed, Aug 13, 2014 at 3:05 PM, James E. Blair wrote:

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

Very cool!! this is really nice UI, super useful....

one litle suggestions for the folk that knows how to do that if that's
possible to do is to sort between the voting and the non-voting so we can
easily spot which one are worthwhile to look at or not.

If it is not worth looking at a job that is run by the OpenStack CI
system, please propose a patch to openstack-infra/config to delete it
from the Zuul config. We only want to run what's useful, and we have
other methods (the silent and experimental queues) to develop new jobs
without creating noise.

If there is a third-party CI system reporting non-voting jobs, um, I
don't know what that means. If it bothers you, you might ask the
third-party CI system to disable them and if they don't, then ask us to
disable the third-party CI system.

-Jim

responded Aug 13, 2014 by James_E._Blair (5,080 points)   1 6 8
0 votes

On Wed, Aug 13, 2014 at 6:27 PM, James E. Blair wrote:

If it is not worth looking at a job that is run by the OpenStack CI
system, please propose a patch to openstack-infra/config to delete it
from the Zuul config. We only want to run what's useful, and we have
other methods (the silent and experimental queues) to develop new jobs
without creating noise.

If there is a third-party CI system reporting non-voting jobs, um, I
don't know what that means. If it bothers you, you might ask the
third-party CI system to disable them and if they don't, then ask us to
disable the third-party CI system.

I didn't meant that they were not worthwhile to look at I was just thinking
it could be useful to sort them so we can easily identify from a UI
perspective which one was voting or not.

Chmouel
-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 13, 2014 by chmouel_at_enovance. (2,660 points)   2 2
0 votes

On 08/13/2014 06:23 PM, Mark McLoughlin wrote:
On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote:

corvus at inaugust.com (James E. Blair) writes:

Sean Dague writes:

This has all gone far enough that someone actually wrote a Grease Monkey
script to purge all the 3rd Party CI content out of Jenkins UI. People
are writing mail filters to dump all the notifications. Dan Berange
filters all them out of his gerrit query tools.

I should also mention that there is a pending change to do something
similar via site-local Javascript in our Gerrit:

https://review.openstack.org/#/c/95743/

I don't think it's an ideal long-term solution, but if it works, we may
have some immediate relief without all having to install greasemonkey
scripts.

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

Beautiful! Thank you so much to everyone involved.

+1! Love this.

--
Russell Bryant

responded Aug 13, 2014 by Russell_Bryant (19,240 points)   2 3 8
0 votes

On 08/13/2014 06:35 PM, Russell Bryant wrote:
On 08/13/2014 06:23 PM, Mark McLoughlin wrote:

On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote:

corvus at inaugust.com (James E. Blair) writes:

Sean Dague writes:

This has all gone far enough that someone actually wrote a Grease Monkey
script to purge all the 3rd Party CI content out of Jenkins UI. People
are writing mail filters to dump all the notifications. Dan Berange
filters all them out of his gerrit query tools.

I should also mention that there is a pending change to do something
similar via site-local Javascript in our Gerrit:

https://review.openstack.org/#/c/95743/

I don't think it's an ideal long-term solution, but if it works, we may
have some immediate relief without all having to install greasemonkey
scripts.

You may have noticed that this has merged, along with a further change
that shows the latest results in a table format. (You may need to
force-reload in your browser to see the change.)

Beautiful! Thank you so much to everyone involved.

+1! Love this.

Indeed. Amazeballs.

-jay

responded Aug 13, 2014 by Jay_Pipes (59,760 points)   3 11 14
0 votes

Chmouel Boudjnah writes:

On Wed, Aug 13, 2014 at 6:27 PM, James E. Blair wrote:

If it is not worth looking at a job that is run by the OpenStack CI
system, please propose a patch to openstack-infra/config to delete it
from the Zuul config. We only want to run what's useful, and we have
other methods (the silent and experimental queues) to develop new jobs
without creating noise.

If there is a third-party CI system reporting non-voting jobs, um, I
don't know what that means. If it bothers you, you might ask the
third-party CI system to disable them and if they don't, then ask us to
disable the third-party CI system.

I didn't meant that they were not worthwhile to look at I was just thinking
it could be useful to sort them so we can easily identify from a UI
perspective which one was voting or not.

I think part of why I responded with that is because this is not the
first time I have heard someone say they don't want to see the
non-voting jobs. If that's a real desire, we should get to the bottom
of it. We don't actually want the CI system to be annoying. :)

From my perspective, anything red should represent something that
someone needs to process and either correct or make an informed decision
to ignore. That is, a failing non-voting job should either cause the
submitter to fix a problem with their code (same as a failing voting
job), or investigate the result and determine that in this case, the
non-voting job can be safely ignored (ie, is an expected result of the
change).

If we have non-voting jobs that don't match that criteria, we should
remove them. Or make them voting.

-Jim

responded Aug 13, 2014 by James_E._Blair (5,080 points)   1 6 8
0 votes

On Wed, 13 Aug 2014 18:52:05 -0400
Jay Pipes wrote:

On 08/13/2014 06:35 PM, Russell Bryant wrote:

On 08/13/2014 06:23 PM, Mark McLoughlin wrote:

On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote:

corvus at inaugust.com (James E. Blair) writes:

Sean Dague writes:

This has all gone far enough that someone actually wrote a
Grease Monkey script to purge all the 3rd Party CI content out
of Jenkins UI. People are writing mail filters to dump all the
notifications. Dan Berange filters all them out of his gerrit
query tools.

I should also mention that there is a pending change to do
something similar via site-local Javascript in our Gerrit:

https://review.openstack.org/#/c/95743/

I don't think it's an ideal long-term solution, but if it works,
we may have some immediate relief without all having to install
greasemonkey scripts.

You may have noticed that this has merged, along with a further
change that shows the latest results in a table format. (You may
need to force-reload in your browser to see the change.)

Beautiful! Thank you so much to everyone involved.

+1! Love this.

Indeed. Amazeballs.

Agreed! This is a really nice improvement

Chris

responded Aug 14, 2014 by Christopher_Yeoh (8,400 points)   1 3 4
...