settingsLogin | Registersettings

[openstack-dev] [Cinder][third-party] How to make progress on cinder driver CI post J3

0 votes

Many of us are still working on setting up and testing 3rd party ci for cinder drivers.

None of them currently have gerrit +1/-1 voting, but I heard there's a plan to "disable" those that are "not working" post-J3 (Please correct me if I misunderstood).

However, Post-J3 is a great time to work on 3rd party ci for a few reasons:

  1. Increases the overall QA effort when it's most needed.

  2. Many contributors have more time available since feature work is complete.

Since erroneous failing tests from non-voting ci accounts can still distract reviewers, I'd like to propose a compromise:
Marking 3rd party ci jobs still undergoing testing and stability as (non-voting).
e.g.
dsvm-tempest-my-driverhttp://15.126.198.151/67/106567/30/check/dsvm-tempest-hp-lefthand/a212407

SUCCESS in 39m 27s (non-voting)

dsvm-tempest-my-driver

FAILURE in 39m 54s (non-voting)

This way, progress can still be made by cinder vendors working on setting up 3rd party ci under 'real' load post J3 while minimizing distractions for cinder reviewers and other stakeholders.
I think this is consistent with how the OpenStack "Jenkins" CI marks potentially unstable jobs.

Please share your thoughts.

Thanks,
Ramy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openstack.org/pipermail/openstack-dev/attachments/20140822/11559851/attachment.html

asked Aug 22, 2014 in openstack-dev by Asselin,_Ramy (4,700 points)   1 2 5
retagged Jan 28, 2015 by admin

2 Responses

0 votes

On Fri, Aug 22, 2014 at 11:19 AM, Asselin, Ramy <ramy.asselin at hp.com> wrote:

Many of us are still working on setting up and testing 3rd party ci for
cinder drivers.

None of them currently have gerrit +1/-1 voting, but I heard there?s a
plan to ?disable? those that are ?not working? post-J3 (Please correct me
if I misunderstood).

However, Post-J3 is a great time to work on 3rd party ci for a few
reasons:

  1. Increases the overall QA effort when it?s most needed.

  2. Many contributors have more time available since feature work is
    complete.

Since erroneous failing tests from non-voting ci accounts can still
distract reviewers, I?d like to propose a compromise:

Marking 3rd party ci jobs still undergoing testing and stability as
(non-voting).

e.g.

dsvm-tempest-my-driver

SUCCESS in 39m 27s (non-voting)

dsvm-tempest-my-driver

FAILURE in 39m 54s (non-voting)

This way, progress can still be made by cinder vendors working on setting
up 3rd party ci under ?real? load post J3 while minimizing distractions
for cinder reviewers and other stakeholders.

I think this is consistent with how the OpenStack ?Jenkins? CI marks
potentially unstable jobs.

Please share your thoughts.

Thanks,

Ramy


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

?Just FYI, I haven't necessarily heard that drivers etc are going to be
removed. I would say however that the job should be removed until it's
stable. There's no reason you can't do that (just don't submit your
results) and still work on fixing things up. My system for example has
been running on every OpenStack commit today but I'm dumping results to a
test location while I make sure that things are stable and probably won't
turn it on until late next week when I'm sure it will perform. Submitting
a system that fails to start 80% of the time doesn't help any of us out IMO.

I realize there's a LOT of really hard work going on and progress being
made, not discounting that at all. Just pointing out that populating every
review with "ci-system-xyz failed" doesn't help us out much either. If
it's backend problems they need fixed, if it's infra problems they need
fixed. That's the whole point of this exercise to begin with IIRC.

Thanks
John?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 22, 2014 by john.griffith_at_sol (4,960 points)   1 2 3
0 votes

Fungi (on irc) suggested the following option to run the tests without reporting them to gerrit.
I?m still testing it out, but for the benefit of others using the ?zuul? ci approach, try the ?silent? pipeline [1]

Thanks,
Ramy

[1]https://github.com/openstack-infra/zuul/blob/master/doc/source/zuul.rst:
This will trigger jobs whenever a change is merged to a named branch (e.g., master). No output will be reported to Gerrit. This is useful for side effects such as creating per-commit tarballs.
- name: silent
manager: IndependentPipelineManager
trigger:
- event: patchset-created

From: John Griffith [mailto:john.griffith at solidfire.com]
Sent: Friday, August 22, 2014 2:25 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Cinder][third-party] How to make progress on cinder driver CI post J3

On Fri, Aug 22, 2014 at 11:19 AM, Asselin, Ramy <ramy.asselin at hp.com<mailto:ramy.asselin at hp.com>> wrote:
Many of us are still working on setting up and testing 3rd party ci for cinder drivers.

None of them currently have gerrit +1/-1 voting, but I heard there?s a plan to ?disable? those that are ?not working? post-J3 (Please correct me if I misunderstood).

However, Post-J3 is a great time to work on 3rd party ci for a few reasons:

  1. Increases the overall QA effort when it?s most needed.

  2. Many contributors have more time available since feature work is complete.

Since erroneous failing tests from non-voting ci accounts can still distract reviewers, I?d like to propose a compromise:
Marking 3rd party ci jobs still undergoing testing and stability as (non-voting).
e.g.
dsvm-tempest-my-driver

SUCCESS in 39m 27s (non-voting)

dsvm-tempest-my-driver

FAILURE in 39m 54s (non-voting)

This way, progress can still be made by cinder vendors working on setting up 3rd party ci under ?real? load post J3 while minimizing distractions for cinder reviewers and other stakeholders.
I think this is consistent with how the OpenStack ?Jenkins? CI marks potentially unstable jobs.

Please share your thoughts.

Thanks,
Ramy


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
?Just FYI, I haven't necessarily heard that drivers etc are going to be removed. I would say however that the job should be removed until it's stable. There's no reason you can't do that (just don't submit your results) and still work on fixing things up. My system for example has been running on every OpenStack commit today but I'm dumping results to a test location while I make sure that things are stable and probably won't turn it on until late next week when I'm sure it will perform. Submitting a system that fails to start 80% of the time doesn't help any of us out IMO.

I realize there's a LOT of really hard work going on and progress being made, not discounting that at all. Just pointing out that populating every review with "ci-system-xyz failed" doesn't help us out much either. If it's backend problems they need fixed, if it's infra problems they need fixed. That's the whole point of this exercise to begin with IIRC.
Thanks
John?

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 23, 2014 by Asselin,_Ramy (4,700 points)   1 2 5
...