settingsLogin | Registersettings

[openstack-dev] [neutron] [third-party] What tests are required to be run

0 votes

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

asked Aug 14, 2014 in openstack-dev by Kyle_Mestery (16,960 points)   3 3 7
retagged Mar 19, 2015 by admin

59 Responses

0 votes

Hi-

I'm hitting bugs for (basic/advanced)serverops testing.

https://bugs.launchpad.net/nova/+bug/1232303

Kindly help me with a fix to this.

Thanking you.

--
Trinath Somanchi - B39208
trinath.somanchi at freescale.com | extn: 4048

-----Original Message-----
From: Kyle Mestery [mailto:mestery at mestery.com]
Sent: Thursday, August 14, 2014 9:00 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [neutron] [third-party] What tests are required to be run

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running more than just Tempest API tests, which I still see most neutron third-party CI setups doing. I'd like to ask everyone who operates a third-party CI account for Neutron to please look at the link below and make sure you are running appropriate tests. If you have questions, the weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 14, 2014 by trinath.somanchi_at_ (3,420 points)   3 7
0 votes

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 15, 2014 by Edgar_Magana (13,760 points)   2 5 6
0 votes

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches
to be merged, but I was wondering if it is a requirement that CIs vote
ALL the Neutron patches. Some may have missed your call just because of
their filters (see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting

On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana <edgar.magana at workday.com>
wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 15, 2014 by Ivar_Lazzaro (1,880 points)   1 3
0 votes

Correction

The right links are:

https://review.openstack.org/#/c/114629/

Not this one:
https://review.openstack.org/#/c/40296/

Edgar

On 8/15/14, 3:35 PM, "Edgar Magana" <edgar.magana at workday.com> wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugi
n
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron
tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty
job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 15, 2014 by Edgar_Magana (13,760 points)   2 5 6
0 votes

Ivar,

Very valid point. This is why I need clarification from those CI owner.
I will run a new test with a basic change in the Neutron DB code. That should be covered by almost all CI systems.

Edgar

From: Ivar Lazzaro >
Reply-To: "OpenStack Development Mailing List (not for usage questions)" >
Date: Friday, August 15, 2014 at 3:47 PM
To: "OpenStack Development Mailing List (not for usage questions)" >
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches to be merged, but I was wondering if it is a requirement that CIs vote ALL the Neutron patches. Some may have missed your call just because of their filters (see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting

On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana <edgar.magana at workday.com<mailto:edgar.magana at workday.com>> wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty job!

On 8/14/14, 8:30 AM, "Kyle Mestery" > wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 15, 2014 by Edgar_Magana (13,760 points)   2 5 6
0 votes

VMware minesweeper has filters which have been designed to cover the
largest possible subset of submissions without add unnecessary load to our
scarce resources for CI validation. This is probably why the analysis
reveals not all patches are covered.

Therefore our filters exclude neutral changes such as those in README.rst
(they're unlikely to crash neutron are they?), or changes in wsgi.py. The
latter is because the wsgi framework is shared with all plugins, and when
it breaks the NSX plugin it's very likely upstream tests will break too.

I think the areas which are important to cover are:
- neutron/agents/.* (or at least the agents you use)
- neutron/openstack/common/.*
- neutron/notifiers/.* (if your drivers report vif plugging events to nova)
- neutron/db/.*
- neutron/services/${thepluginsyouruninyourci_eveniftheyrenotyours}
- your plugin patch

Salvatore

PS: Openstack proposal bot submission's checks have been suspended for
vmware minesweeper about 2 days ago. They will resume soon.

On 16 August 2014 01:02, Edgar Magana <edgar.magana at workday.com> wrote:

Ivar,

Very valid point. This is why I need clarification from those CI owner.
I will run a new test with a basic change in the Neutron DB code. That
should be covered by almost all CI systems.

Edgar

From: Ivar Lazzaro
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
openstack-dev at lists.openstack.org>
Date: Friday, August 15, 2014 at 3:47 PM
To: "OpenStack Development Mailing List (not for usage questions)" <
openstack-dev at lists.openstack.org>
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain
patches to be merged, but I was wondering if it is a requirement that CIs
vote ALL the Neutron patches. Some may have missed your call just because
of their filters (see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting

On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana <edgar.magana at workday.com>
wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:

https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron
tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty
job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 15, 2014 by Salvatore_Orlando (12,280 points)   2 5 8
0 votes

That's great!

Also, I would suggest we decide on a 'standard' way for CI owners to
comunicate their CIs status (eg. MyCompany CI is not voting waiting for bug

9001 to be fixed).

Using the ML for that may be confusing... What about an etherpad (linked
from [0]) or a dedicated section of [0] itself?

Ivar.

[0]
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin_and_Drivers

On Sat, Aug 16, 2014 at 1:02 AM, Edgar Magana <edgar.magana at workday.com>
wrote:

Ivar,

Very valid point. This is why I need clarification from those CI owner.
I will run a new test with a basic change in the Neutron DB code. That
should be covered by almost all CI systems.

Edgar

From: Ivar Lazzaro
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
openstack-dev at lists.openstack.org>
Date: Friday, August 15, 2014 at 3:47 PM
To: "OpenStack Development Mailing List (not for usage questions)" <
openstack-dev at lists.openstack.org>
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are
required to be run

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain
patches to be merged, but I was wondering if it is a requirement that CIs
vote ALL the Neutron patches. Some may have missed your call just because
of their filters (see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting

On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana <edgar.magana at workday.com>
wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:

https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron
tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty
job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 15, 2014 by Ivar_Lazzaro (1,880 points)   1 3
0 votes

You didn't wait long enough for the Big Switch CI to reach your negative
test. :-)
Currently the CI system is backed by about 100 patches to push responses
out ~22 hours.

This does bring up an old discussion that was never resolved.
What is the minimum expected response time for CI systems?

On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana <edgar.magana at workday.com>
wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Kevin Benton
-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 15, 2014 by Kevin_Benton (24,800 points)   3 5 7
0 votes

Hi Edgar,

I am running the Nuage CI.
The Nuage CI has run and posted successfully the result for the first
review.

Because of infra issue(internet can be down, etc) we are running manually
the -1
-1 take some time away for developper and before doing a -1, I want to be
sure that it's a valid issue. (we had a talk about it during the
third-party meeting on mondays)

I have posted the failure report and you can see that it was run, uploaded
the result to our server but we didn't vote (reason mention above)
http://208.113.169.228/nuage-ci/29_114629_1/ (seetimestamp of the files)

Let me know if you have any questions.

Franck (lyxus)

Franck

On Fri, Aug 15, 2014 at 3:35 PM, Edgar Magana <edgar.magana at workday.com>
wrote:

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:

responded Aug 16, 2014 by Franck_Yelles (300 points)   1
0 votes

Edgar:

For the Notes for the Cisco APIC, can you change the comment "results are fake" to something like "results are only valid for APIC-related commits"? I think this more accurately represents our current results (for reasons we chatted about on another thread).

Thanks,
Dane

-----Original Message-----
From: Edgar Magana [mailto:edgar.magana at workday.com]
Sent: Friday, August 15, 2014 6:36 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run
Importance: High

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
andDrivers

We will discuss the actions to take on the next Neutron IRC meeting. So please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/

I would expect all plugins and drivers passing on the first one and failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor here? but someone has to do the dirty job!

On 8/14/14, 8:30 AM, "Kyle Mestery" wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below and
make sure you are running appropriate tests. If you have questions, the
weekly third-party meeting [2] is a great place to ask questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 16, 2014 by Dane_Leblanc_(leblan (920 points)   2
...