settingsLogin | Registersettings

[openstack-dev] [infra][tripleo] status of tripleo-test-cloud-rh1

0 votes

Greetings,

5 months ago fungi posted:

[tripleo] becoming third party CI (was: enabling third party CI)[1]

About having the discussion whether the existing TripleO CI should itself follow
our third-party integration model instead of the current implementation relying
on our main community Zuul/Nodepool/Jenkins servers.

The result of the thread had some pros and cons, which I encourge people to
re-read.

At the Austin summit we continued the topic of moving tripleo-ci into 3rd party
CI. Again, consensus could not be reached however we made some progress. I
would take on the responsibility to help bring tripleo-test-cloud-rh1 more
inline with openstack-infra tooling.

That includes, but is not limited to:

And various other reviews adding AFS mirrors for centos / epel. Updates to
tripleo-ci using our openstack-infra AFS mirrors along with providing general
support for both tripleo-test-cloud-rh1 and tripleo-test-cloud-rh2.

In a short amount of time, we've made great progress with
tripleo-test-cloud-rh1, helping bring it more inline with openstack-infra
tooling. While we are not finished, there is still some private infrastrucuture
that tripleo-ci is depending on. I am confident in the next 3 months we should
have that all replaced and using openstack community infrastruture.

However on Friday[2], we started talking about tripleo-test-cloud-rh1 again in

openstack-infra and found ourselves revisiting the original email. It is all

driven from the current effort from tripleo to start using move community clouds
for running tripleo-ci jobs. Today, 3 different type of tripleo-ci jobs are now
run across all our clouds, for example there is a centos-7-2-node jobs. However,
tripleo-test-cloud-rh1 is only today setup to accept only tripleo-ci jobs. This
job does not run on tripleo-test-cloud-rh1.

jeblair posted the following statement:

It feels like the tripleo cloud has been grandfathered in its current state
for a while. I'd just like to make sure we're being fair to everyone. So if
tripleo wants to run tripleo jobs, then i think we should move it to 3rd party
ci. I think that's a fine choice and we can continue to work together
(please!) but with better division of reponsibilities. Or, if we want to
revise the idea of a multi-provider hardware platform that's available for all
openstack projects, i'm game for that. It would be great, but more work.

Should we continue the push to move tripleo-test-cloud-rh1 to 3rd party CI
(removing from nodepool.o.o) or do we start enabling more jobs on
tripleo-test-cloud-rh1 bringing the cloud even more into openstack-infra?

My personal thoughts, as somebody who's been working on it for the last 4
months, I still feel tripleo-test-cloud-rh1 should move to 3rd party CI.
However, with the work done in the last 4 months, I believe
tripleo-test-cloud-rh1 could start running additional jobs based on the work
above.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html
[2] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-08-05.log.html#t2016-08-05T23:07:35


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Aug 7, 2016 in openstack-dev by pabelanger_at_redhat (6,560 points)   1 1 2
retagged Jan 26, 2017 by admin

6 Responses

0 votes

On 08/06/2016 07:17 PM, Paul Belanger wrote:
Greetings,

5 months ago fungi posted:

[tripleo] becoming third party CI (was: enabling third party CI)[1]

About having the discussion whether the existing TripleO CI should itself follow
our third-party integration model instead of the current implementation relying
on our main community Zuul/Nodepool/Jenkins servers.

The result of the thread had some pros and cons, which I encourge people to
re-read.

At the Austin summit we continued the topic of moving tripleo-ci into 3rd party
CI. Again, consensus could not be reached however we made some progress. I
would take on the responsibility to help bring tripleo-test-cloud-rh1 more
inline with openstack-infra tooling.

That includes, but is not limited to:

And various other reviews adding AFS mirrors for centos / epel. Updates to
tripleo-ci using our openstack-infra AFS mirrors along with providing general
support for both tripleo-test-cloud-rh1 and tripleo-test-cloud-rh2.

In a short amount of time, we've made great progress with
tripleo-test-cloud-rh1, helping bring it more inline with openstack-infra
tooling. While we are not finished, there is still some private infrastrucuture
that tripleo-ci is depending on. I am confident in the next 3 months we should
have that all replaced and using openstack community infrastruture.

However on Friday[2], we started talking about tripleo-test-cloud-rh1 again in

openstack-infra and found ourselves revisiting the original email. It is all

driven from the current effort from tripleo to start using move community clouds
for running tripleo-ci jobs. Today, 3 different type of tripleo-ci jobs are now
run across all our clouds, for example there is a centos-7-2-node jobs. However,
tripleo-test-cloud-rh1 is only today setup to accept only tripleo-ci jobs. This
job does not run on tripleo-test-cloud-rh1.

jeblair posted the following statement:

It feels like the tripleo cloud has been grandfathered in its current state
for a while. I'd just like to make sure we're being fair to everyone. So if
tripleo wants to run tripleo jobs, then i think we should move it to 3rd party
ci. I think that's a fine choice and we can continue to work together
(please!) but with better division of reponsibilities. Or, if we want to
revise the idea of a multi-provider hardware platform that's available for all
openstack projects, i'm game for that. It would be great, but more work.

Should we continue the push to move tripleo-test-cloud-rh1 to 3rd party CI
(removing from nodepool.o.o) or do we start enabling more jobs on
tripleo-test-cloud-rh1 bringing the cloud even more into openstack-infra?

My personal thoughts, as somebody who's been working on it for the last 4
months, I still feel tripleo-test-cloud-rh1 should move to 3rd party CI.
However, with the work done in the last 4 months, I believe
tripleo-test-cloud-rh1 could start running additional jobs based on the work
above.

I was always in favor of going third-party, so +1 to that option. If we
still insist on staying integrated then I think it's totally fair to say
we need to start running other jobs too though.

Long term we'd still like to get away from needing to run on a custom
cloud, but until OVB support is available in more of the regular infra
clouds I think we're stuck with our custom one.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html
[2] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-08-05.log.html#t2016-08-05T23:07:35


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2016 by Ben_Nemec (19,660 points)   2 3 3
0 votes

On Sat, Aug 6, 2016 at 8:17 PM, Paul Belanger pabelanger@redhat.com wrote:
Greetings,

5 months ago fungi posted:

[tripleo] becoming third party CI (was: enabling third party CI)[1]

About having the discussion whether the existing TripleO CI should itself follow
our third-party integration model instead of the current implementation relying
on our main community Zuul/Nodepool/Jenkins servers.

The result of the thread had some pros and cons, which I encourge people to
re-read.

At the Austin summit we continued the topic of moving tripleo-ci into 3rd party
CI. Again, consensus could not be reached however we made some progress. I
would take on the responsibility to help bring tripleo-test-cloud-rh1 more
inline with openstack-infra tooling.

That includes, but is not limited to:

And various other reviews adding AFS mirrors for centos / epel. Updates to
tripleo-ci using our openstack-infra AFS mirrors along with providing general
support for both tripleo-test-cloud-rh1 and tripleo-test-cloud-rh2.

In a short amount of time, we've made great progress with
tripleo-test-cloud-rh1, helping bring it more inline with openstack-infra
tooling. While we are not finished, there is still some private infrastrucuture
that tripleo-ci is depending on. I am confident in the next 3 months we should
have that all replaced and using openstack community infrastruture.

However on Friday[2], we started talking about tripleo-test-cloud-rh1 again in

openstack-infra and found ourselves revisiting the original email. It is all

driven from the current effort from tripleo to start using move community clouds
for running tripleo-ci jobs. Today, 3 different type of tripleo-ci jobs are now
run across all our clouds, for example there is a centos-7-2-node jobs. However,
tripleo-test-cloud-rh1 is only today setup to accept only tripleo-ci jobs. This
job does not run on tripleo-test-cloud-rh1.

jeblair posted the following statement:

It feels like the tripleo cloud has been grandfathered in its current state
for a while. I'd just like to make sure we're being fair to everyone. So if
tripleo wants to run tripleo jobs, then i think we should move it to 3rd party
ci. I think that's a fine choice and we can continue to work together
(please!) but with better division of reponsibilities. Or, if we want to
revise the idea of a multi-provider hardware platform that's available for all
openstack projects, i'm game for that. It would be great, but more work.

Should we continue the push to move tripleo-test-cloud-rh1 to 3rd party CI
(removing from nodepool.o.o) or do we start enabling more jobs on
tripleo-test-cloud-rh1 bringing the cloud even more into openstack-infra?

My personal thoughts, as somebody who's been working on it for the last 4
months, I still feel tripleo-test-cloud-rh1 should move to 3rd party CI.
However, with the work done in the last 4 months, I believe
tripleo-test-cloud-rh1 could start running additional jobs based on the work
above.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html
[2] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-08-05.log.html#t2016-08-05T23:07:35

I'd like to provide some additional context about where we'd like to
go with the CI jobs running on all cloud providers. We've added
centos7 2-node multinode jobs, and we have some single node jobs as
well. What I'd like clarification around is there is
hesitation/concern around TripleO building out additional mutlinode
jobs that run on any cloud.

I'd like to continue down this path of adding more CI jobs that can
run on any nodepool managed cloud provider. To do so, we'd need to
have 3+ node jobs. A 3 node job would let us test with an undercloud
and a 2 node overcloud, while a 4 node job would let us test with an
undercloud and a 3 node overcloud so we could test HA. 4 node HA jobs
seem to work fine in my testing that I've done on
tripleo-test-cloud-rh2, using the same network setup and infra images
that we do for the 2 node job.

tripleo-ci can't move entirely to using these types of multinode jobs
though, because they do not exercise the nova/ironic deployment during
the tests. So, there will always be some subset of tripleo-ci jobs
that still need tripleo-test-cloud-rh2{1,2} as those clouds are
configured with the necessary OpenStack features enabled to allow for
pxe booting of instances.

I'm certainly open and supportive of the idea of running all job types
on the tripleo clouds. The work Paul has done should make this
possible. rh2 is on "loan" to the TripleO project team while rh1 moved
datacenters. But, I'm told it can become permanent if there is a need.
Keep in mind this is not a single cloud with 2 regions. rh1 and rh2
are 2 separate single region clouds, however they offer the equivalent
functionality to run all tripleo-ci and infra job types (barring just
a few outstanding bugs/configuration issues). So, there is failover
for tripleo-ci jobs (if we re-enable rh2).

At the same time, if there are concerns about capacity from Infra's
perspective as we'd like to build out to at least 4-node multinode
jobs, then I think it makes good sense to open up rh1/rh2 to all job
types, so that they help with the capacity.
In that case, it would not make sense to move tripleo-ci to be a 3rd
party CI setup.

If there is plenty of capacity on the existing cloud providers and no
concerns about building out additional multinode jobs, then there
might not be a huge benefit in enabling all job types on rh1/rh2. If
that's the case then we might as well just move them to 3rd party CI.

I suppose it's also possible that we might be pushing too strongly
down the multinode path? Is the general concensus in infra that they'd
like to help enable project teams to eventually add 3 and 4 (and maybe
more) node multinode jobs? If not, then I agree it probably does make
sense to move most of tripleo-ci to be 3rd party, and we'd just keep
our existing check jobs as-is instead of proposing new ones.

My point is: if there's concensus around TripleO building out to 3+
node multinode jobs, then I think tripleo-ci should stay where it is
so that we can also work towards contributing to available capacity,
etc.

If there is not, and tripleo-ci just keeps adding more jobs that will
only run on our own clouds, then I agree with Paul and Ben that
tripleo-ci feels more naturally to be 3rd party CI.

--
-- James Slagle
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2016 by James_Slagle (7,000 points)   1 3 3
0 votes

On Mon, Aug 8, 2016 at 11:47 AM, James Slagle james.slagle@gmail.com wrote:
On Sat, Aug 6, 2016 at 8:17 PM, Paul Belanger pabelanger@redhat.com wrote:

Greetings,

5 months ago fungi posted:

[tripleo] becoming third party CI (was: enabling third party CI)[1]

About having the discussion whether the existing TripleO CI should itself follow
our third-party integration model instead of the current implementation relying
on our main community Zuul/Nodepool/Jenkins servers.

The result of the thread had some pros and cons, which I encourge people to
re-read.

At the Austin summit we continued the topic of moving tripleo-ci into 3rd party
CI. Again, consensus could not be reached however we made some progress. I
would take on the responsibility to help bring tripleo-test-cloud-rh1 more
inline with openstack-infra tooling.

That includes, but is not limited to:

And various other reviews adding AFS mirrors for centos / epel. Updates to
tripleo-ci using our openstack-infra AFS mirrors along with providing general
support for both tripleo-test-cloud-rh1 and tripleo-test-cloud-rh2.

In a short amount of time, we've made great progress with
tripleo-test-cloud-rh1, helping bring it more inline with openstack-infra
tooling. While we are not finished, there is still some private infrastrucuture
that tripleo-ci is depending on. I am confident in the next 3 months we should
have that all replaced and using openstack community infrastruture.

However on Friday[2], we started talking about tripleo-test-cloud-rh1 again in

openstack-infra and found ourselves revisiting the original email. It is all

driven from the current effort from tripleo to start using move community clouds
for running tripleo-ci jobs. Today, 3 different type of tripleo-ci jobs are now
run across all our clouds, for example there is a centos-7-2-node jobs. However,
tripleo-test-cloud-rh1 is only today setup to accept only tripleo-ci jobs. This
job does not run on tripleo-test-cloud-rh1.

jeblair posted the following statement:

It feels like the tripleo cloud has been grandfathered in its current state
for a while. I'd just like to make sure we're being fair to everyone. So if
tripleo wants to run tripleo jobs, then i think we should move it to 3rd party
ci. I think that's a fine choice and we can continue to work together
(please!) but with better division of reponsibilities. Or, if we want to
revise the idea of a multi-provider hardware platform that's available for all
openstack projects, i'm game for that. It would be great, but more work.

Should we continue the push to move tripleo-test-cloud-rh1 to 3rd party CI
(removing from nodepool.o.o) or do we start enabling more jobs on
tripleo-test-cloud-rh1 bringing the cloud even more into openstack-infra?

My personal thoughts, as somebody who's been working on it for the last 4
months, I still feel tripleo-test-cloud-rh1 should move to 3rd party CI.
However, with the work done in the last 4 months, I believe
tripleo-test-cloud-rh1 could start running additional jobs based on the work
above.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html
[2] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-08-05.log.html#t2016-08-05T23:07:35

I'd like to provide some additional context about where we'd like to
go with the CI jobs running on all cloud providers. We've added
centos7 2-node multinode jobs, and we have some single node jobs as
well. What I'd like clarification around is there is
hesitation/concern around TripleO building out additional mutlinode
jobs that run on any cloud.

I'd like to continue down this path of adding more CI jobs that can
run on any nodepool managed cloud provider. To do so, we'd need to
have 3+ node jobs. A 3 node job would let us test with an undercloud
and a 2 node overcloud, while a 4 node job would let us test with an
undercloud and a 3 node overcloud so we could test HA. 4 node HA jobs
seem to work fine in my testing that I've done on
tripleo-test-cloud-rh2, using the same network setup and infra images
that we do for the 2 node job.

tripleo-ci can't move entirely to using these types of multinode jobs
though, because they do not exercise the nova/ironic deployment during
the tests. So, there will always be some subset of tripleo-ci jobs
that still need tripleo-test-cloud-rh2{1,2} as those clouds are
configured with the necessary OpenStack features enabled to allow for
pxe booting of instances.

I'm certainly open and supportive of the idea of running all job types
on the tripleo clouds. The work Paul has done should make this
possible. rh2 is on "loan" to the TripleO project team while rh1 moved
datacenters. But, I'm told it can become permanent if there is a need.
Keep in mind this is not a single cloud with 2 regions. rh1 and rh2
are 2 separate single region clouds, however they offer the equivalent
functionality to run all tripleo-ci and infra job types (barring just
a few outstanding bugs/configuration issues). So, there is failover
for tripleo-ci jobs (if we re-enable rh2).

At the same time, if there are concerns about capacity from Infra's
perspective as we'd like to build out to at least 4-node multinode
jobs, then I think it makes good sense to open up rh1/rh2 to all job
types, so that they help with the capacity.
In that case, it would not make sense to move tripleo-ci to be a 3rd
party CI setup.

If there is plenty of capacity on the existing cloud providers and no
concerns about building out additional multinode jobs, then there
might not be a huge benefit in enabling all job types on rh1/rh2. If
that's the case then we might as well just move them to 3rd party CI.

I suppose it's also possible that we might be pushing too strongly
down the multinode path? Is the general concensus in infra that they'd
like to help enable project teams to eventually add 3 and 4 (and maybe
more) node multinode jobs? If not, then I agree it probably does make
sense to move most of tripleo-ci to be 3rd party, and we'd just keep
our existing check jobs as-is instead of proposing new ones.

My point is: if there's concensus around TripleO building out to 3+
node multinode jobs, then I think tripleo-ci should stay where it is
so that we can also work towards contributing to available capacity,
etc.

I think there are some values to test 3+ node multinode jobs:

  • Testing High-Availability: we currently have pacemaker scenarios,
    that require at least 3 controllers and 1 compute to test something
    realistic. Though we have nothing in our tests that disable a
    controller and run tests again. So unless we have it, I think we could
    run our tests on a single controller node.

  • Testing Nova live migration (even if it's tested by devstack) in
    TripleO, to validate configuration that makes it possible.

  • Testing 3+ means at least 2 nodes for overcloud, which is the first
    realistic way to test a multi-node overcloud, and validate TripleO
    Orchestration, for both initial deployment and upgrades. We are an
    installer, we rely on orchestration and configuration management, so I
    think we really need this minimal architecture.

Regarding the contribution to OpenStack Infra to give some capacity,
it would be certainly fair from us to do it, big +1.

If there is not, and tripleo-ci just keeps adding more jobs that will
only run on our own clouds, then I agree with Paul and Ben that
tripleo-ci feels more naturally to be 3rd party CI.

--
-- James Slagle
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Emilien Macchi


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2016 by emilien_at_redhat.co (36,940 points)   2 6 9
0 votes

On 2016-08-08 11:47:56 -0400 (-0400), James Slagle wrote:
[...]
I suppose it's also possible that we might be pushing too strongly
down the multinode path? Is the general concensus in infra that they'd
like to help enable project teams to eventually add 3 and 4 (and maybe
more) node multinode jobs?
[...]

We've not outright rejected the idea, but do want to make sure that
there's been suitable due diligence done explaining how the things
you'll be able to test with >2 job nodes effectively can't be done
with <=2. Also we want to be sure that projects who are interested
in multi-node jobs start with just 2 job nodes and get some initial
tests performing well and returning stable results before trying to
push past 2.
--
Jeremy Stanley


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2016 by Jeremy_Stanley (56,700 points)   3 5 7
0 votes

On Mon, Aug 8, 2016 at 1:06 PM, Jeremy Stanley fungi@yuggoth.org wrote:
On 2016-08-08 11:47:56 -0400 (-0400), James Slagle wrote:
[...]

I suppose it's also possible that we might be pushing too strongly
down the multinode path? Is the general concensus in infra that they'd
like to help enable project teams to eventually add 3 and 4 (and maybe
more) node multinode jobs?
[...]

We've not outright rejected the idea, but do want to make sure that
there's been suitable due diligence done explaining how the things
you'll be able to test with >2 job nodes effectively can't be done
with <=2.

Our current 2 node job uses the first node as the undercloud which
deploys an AIO Overcloud on the 2nd node. TripleO traditionally has
also been able to deploy standalone Compute, Cinder, Swift, and Ceph
nodes. Additionally in this cycle, a lot of work has gone into making
it fully customizable what services are deployed on which roles. You
can deploy nodes that are just API services, or just a DB server, or
rabbitmq, etc. In order to test the composability feature we need to
deploy to more than one node.

Also, we'd need at least 3 Overcloud nodes to successfully test that
we can deploy a Pacemaker managed cluster successfully.

Also we want to be sure that projects who are interested
in multi-node jobs start with just 2 job nodes and get some initial
tests performing well and returning stable results before trying to
push past 2.

I think that the 2 node job that we've added has been stable. We've
worked a few issues out that we've hit depending on which cloud
provider we land on, but generally speaking it has been very stable.

We make use of the ovsvxlanbridge function from devstack-gate to
configure the private networking among the nodes. I think this was a
good first step since that has been a proven way in the devstack
multinode jobs. I'd like to move to using TripleO's os-net-config in
the future though, since that is the tool used in TripleO. The end
result of the network configuration would be the same (using ovs vxlan
bridges), we'd just use a different tool to get there.

--
-- James Slagle
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Aug 8, 2016 by James_Slagle (7,000 points)   1 3 3
0 votes

On Mon, Aug 8, 2016 at 1:47 PM, James Slagle james.slagle@gmail.com wrote:
On Mon, Aug 8, 2016 at 1:06 PM, Jeremy Stanley fungi@yuggoth.org wrote:

On 2016-08-08 11:47:56 -0400 (-0400), James Slagle wrote:
[...]

I suppose it's also possible that we might be pushing too strongly
down the multinode path? Is the general concensus in infra that they'd
like to help enable project teams to eventually add 3 and 4 (and maybe
more) node multinode jobs?
[...]

We've not outright rejected the idea, but do want to make sure that
there's been suitable due diligence done explaining how the things
you'll be able to test with >2 job nodes effectively can't be done
with <=2.

Our current 2 node job uses the first node as the undercloud which
deploys an AIO Overcloud on the 2nd node. TripleO traditionally has
also been able to deploy standalone Compute, Cinder, Swift, and Ceph
nodes. Additionally in this cycle, a lot of work has gone into making
it fully customizable what services are deployed on which roles. You
can deploy nodes that are just API services, or just a DB server, or
rabbitmq, etc. In order to test the composability feature we need to
deploy to more than one node.

Also, we'd need at least 3 Overcloud nodes to successfully test that
we can deploy a Pacemaker managed cluster successfully.

Also we want to be sure that projects who are interested
in multi-node jobs start with just 2 job nodes and get some initial
tests performing well and returning stable results before trying to
push past 2.

I think that the 2 node job that we've added has been stable. We've
worked a few issues out that we've hit depending on which cloud
provider we land on, but generally speaking it has been very stable.

We make use of the ovsvxlanbridge function from devstack-gate to
configure the private networking among the nodes. I think this was a
good first step since that has been a proven way in the devstack
multinode jobs. I'd like to move to using TripleO's os-net-config in
the future though, since that is the tool used in TripleO. The end
result of the network configuration would be the same (using ovs vxlan
bridges), we'd just use a different tool to get there.

--
-- James Slagle
--

Reviving this thread to continue the discussion. I'd like to keep the
discussion going with hopes that we can set the stage to finalize a
plan for what we want to tackle in Ocata for tripleo-ci at the
Summit[1].

State of rh1 and rh2
==============
Both rh1 and rh2 are OVB (OpenStack Virtual Baremetal) enabled
clouds[2]. OVB allows us to treat OpenStack instances as baremetal
instances for traditional tripleo-ci testing (PXE booting, etc).
Currently only rh1 is enabled in nodepool. We could re-enable rh2 if
we wanted (the previous ntp issue is resolved now).

As Paul indicated, he's done signficant work to bring these 2 clouds
in alignment with standard Infra tooling. If we wanted to move forward
with opening up these clouds to run other jobs besides tripleo-ci, we
could do that.

Multinode jobs
==========
We've continued to add additional CI jobs using the multinode support
in nodepool and tripleo-ci, running on all the enabled clouds (except
rh1) in nodepool. We are still only at using 2 nodes. I'd like to add
additional jobs and increase this to 3 nodes initially (probably
deploying ceph on the additional node), and then 4 nodes for doing an
HA deployment.

Becoming 3rd party CI
=================
tripleo-ci becoming 3rd party CI continues to come up in discussion. I
agree that the OVB based tripleo-ci jobs align better with the 3rd
party CI model since they do require a specially configured OpenStack
cloud. However, the previous points about opening up rh1/rh2 for non
tripleo jobs and scaling out multinode jobs muddies the water for me a
bit when this topic comes up.

Given we'd like to scale out and add more multinode jobs, I'd like to
counter that by offering some capacity back to nodepool by opening up
the rh1/rh2 clouds to all job types.

However, if tripleo-ci becomes 3rd party CI, we need some
infrastructure to run that CI on and resources to set it up and
maintain the CI tooling. At that point, the TripleO team would be
trying to maintain a 3rd party CI system, and keep 2 public clouds
running for normal infra jobs. That may be possible to do, but it is
additional commitment.

Just to be clear, I'm not trying to say that if tripleo-ci becomes 3rd
party CI, we will "just take our cloud and go home" :-). We want to be
better aligned and integrated with infra tooling and jobs. Maintaining
a 3rd party CI system and 2 public clouds integrated with Infra's CI
system is additional work though, and like a lot of project teams, we
have to prioritize and make trade offs.

Further, even if tripleo-ci becomes 3rd party CI for OVB jobs, and
there are capacity concerns about us scaling out our multinode jobs
onto the other enabled clouds in nodepool, we may still prioritize the
work to maintain these 2 clouds for Infra's general use.

We want to work more closely with Infra overall. But if there is
little perceived benefit in that there are no capacity concerns and no
concerns about us going to 3+ node multinode jobs, then I think we'd
probably just disable our clouds in nodepool, make tripleo-ci OVB jobs
3rd party, and press on that way.

Better alignment with infra CI tools
=========================
When the topic of 3rd party CI comes up, it is often accompanied with
the fact that tripleo-ci is not aligned with other infra tools
(devstack-gate, zuul-cloner, others?). We do plan to continue to
address these things and strive for better alignment. I'm not sure of
all the historical context around what the "original" plans were for
tripleo-ci, and it doesn't really matter all that much anyway.

If the repo needs some modernization to be in better alignment with
tooling or to take advantage of new features in nodepool/zuul (I know
Paul has some ideas around pipelining), then I think we will work on
that regardless.

Some of my goals for tripleo-ci are to continue to try and make it
easier to consume externally, and to use the TripleO production
tooling where possible.

However, we may not be able to align perfectly with how other jobs are
run given the nature of the project (we don't use source installs, pip
installs, devstack, etc). Using the same production tooling in
tripleo-ci that we expect TripleO users to also use when they deploy
is a goal of tripleo-ci.

As an example, we will likely not continue to use devstack-gate to
setup multinode networking, and instead use the tool TripleO uses for
production: os-net-config. Does that have any impact on the decision
of tripleo-ci becoming 3rd party CI or not? I'm not honestly sure what
the expectations are around things like that (e.g., must use something
from devstack-gate).

Personally, I would like to see more testing in the check and gate
queues with production deployment tools across the board (fuel, kolla,
tripleo, etc), because it makes all of OpenStack better when issues
are found earlier rather than later. I think the progress we've made
so far with the TripleO multinode jobs have proven that this is
possible.

[1] We have TripleO sessions proposed to talk about the state of CI:
https://etherpad.openstack.org/p/ocata-tripleo
[2] https://github.com/cybertron/openstack-virtual-baremetal

--
-- James Slagle
--


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 22, 2016 by James_Slagle (7,000 points)   1 3 3
...