settingsLogin | Registersettings

[openstack-dev] [oslo] Can we stop global requirements update?

0 votes

Hoy,

So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr" and
some new release of oslo.* depends on pbr!=2.1.0.

Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
that got in banished by requirements Gods. It does not prevent it to be
used e.g. to install the software or get version information. But it
does break anything that is not in OpenStack because well, pip installs
the latest pbr (2.1.0) and then oslo.* is unhappy about it.

So I understand the culprit is probably pip installation scheme, and we
can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
avoid the entire issue.

But for the future, could we stop updating the requirements in oslo libs
for no good reason? just because some random OpenStack project hit a bug
somewhere?

For example, I've removed requirements update on tooz¹ for more than a
year now, which did not break anything in the meantime, proving that
this process is giving more problem than solutions. Oslo libs doing that
automatic update introduce more pain for all consumers than anything (at
least not in OpenStack).

So if we care about Oslo users outside OpenStack, I beg us to stop this
crazyness. If we don't, we'll just spend time getting rid of Oslo over
the long term…

My 2c,

Cheers,

¹ Unless some API changed in a dep and we needed to raise the dep,
obviously.

--
Julien Danjou

Free Software hacker

https://julien.danjou.info


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

asked May 19, 2017 in openstack-dev by Julien_Danjou (20,500 points)   2 4 6

30 Responses

0 votes

On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:
Hoy,

So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr" and
some new release of oslo.* depends on pbr!=2.1.0.

Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
that got in banished by requirements Gods. It does not prevent it to be
used e.g. to install the software or get version information. But it
does break anything that is not in OpenStack because well, pip installs
the latest pbr (2.1.0) and then oslo.* is unhappy about it.

It actually breaks everything, including OpenStack. Shade and others are
affected by this as well. The specific problem here is that PBR is a
setuprequires which means it gets installed by easyinstall before
anything else. This means that the requirements restrictions are not
applied to it (neither are the constraints). So you get latest PBR from
easyinstall then later when something checks the requirements
(pkg
resources console script entrypoints?) they break because latest
PBR isn't allowed.

We need to stop pinning PBR and more generally stop pinning any
setup_requires (there are a few more now since setuptools itself is
starting to use that to list its deps rather than bundling them).

So I understand the culprit is probably pip installation scheme, and we
can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
avoid the entire issue.

Yes, a new release of PBR undoing the "pin" is the current sane step
forward for fixing this particular issue. Monty also suggested that we
gate global requirements changes on requiring changes not pin any
setup_requires.

But for the future, could we stop updating the requirements in oslo libs
for no good reason? just because some random OpenStack project hit a bug
somewhere?

For example, I've removed requirements update on tooz¹ for more than a
year now, which did not break anything in the meantime, proving that
this process is giving more problem than solutions. Oslo libs doing that
automatic update introduce more pain for all consumers than anything (at
least not in OpenStack).

You are likely largely shielded by the constraints list here which is
derivative of the global requirements list. Basically by using
constraints you get distilled global requirements and even without being
part of the requirements updates you'd be shielded from breakages when
installed via something like devstack or other deployment method using
constraints.

So if we care about Oslo users outside OpenStack, I beg us to stop this
crazyness. If we don't, we'll just spend time getting rid of Oslo over
the long term…

I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

My 2c,

Cheers,

¹ Unless some API changed in a dep and we needed to raise the dep,
obviously.

--
Julien Danjou

Free Software hacker

https://julien.danjou.info

I don't have all the answers, but am fairly certain the situation we
have today is better than the one from several years ago. It is just not
perfect. I think we are better served by refining the current setup or
replacing it with something better but not by reverting.

Clark


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 19, 2017 by Clark_Boylan (8,800 points)   1 2 4
0 votes

On Wed, Apr 19 2017, Clark Boylan wrote:

I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

Most of the problem that requirements is trying to solve is related to
upper-constraints, blocking new releases. And this upper constraints are
used in most jobs, preventing most failure that are seen in gates. It
would have "covered" the pbr issue.

What I want to stop here, is the automatic push of blacklisting/capping
of stuff to everything in OpenStack as soon as one project have a
problem with something.

I don't have all the answers, but am fairly certain the situation we
have today is better than the one from several years ago. It is just not
perfect. I think we are better served by refining the current setup or
replacing it with something better but not by reverting.

Agreed, I'm not suggesting to revert everything. Just the automatic push
of random requirements limits and binding to Oslo. And other projects if
you like, we don't do it for a good year anymore in Telemetry, and again
here we saw 0 breakage due to that change. Just more easyness to
install stuff.

--
Julien Danjou
-- Free Software hacker
-- https://julien.danjou.info


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Apr 19, 2017 by Julien_Danjou (20,500 points)   2 4 6
0 votes

Excerpts from Clark Boylan's message of 2017-04-19 08:10:43 -0700:

On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:

Hoy,

So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr" and
some new release of oslo.* depends on pbr!=2.1.0.

Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
that got in banished by requirements Gods. It does not prevent it to be
used e.g. to install the software or get version information. But it
does break anything that is not in OpenStack because well, pip installs
the latest pbr (2.1.0) and then oslo.* is unhappy about it.

It actually breaks everything, including OpenStack. Shade and others are
affected by this as well. The specific problem here is that PBR is a
setuprequires which means it gets installed by easyinstall before
anything else. This means that the requirements restrictions are not
applied to it (neither are the constraints). So you get latest PBR from
easyinstall then later when something checks the requirements
(pkg
resources console script entrypoints?) they break because latest
PBR isn't allowed.

We need to stop pinning PBR and more generally stop pinning any
setup_requires (there are a few more now since setuptools itself is
starting to use that to list its deps rather than bundling them).

So I understand the culprit is probably pip installation scheme, and we
can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
avoid the entire issue.

Yes, a new release of PBR undoing the "pin" is the current sane step
forward for fixing this particular issue. Monty also suggested that we
gate global requirements changes on requiring changes not pin any
setup_requires.

But for the future, could we stop updating the requirements in oslo libs
for no good reason? just because some random OpenStack project hit a bug
somewhere?

For example, I've removed requirements update on tooz¹ for more than a
year now, which did not break anything in the meantime, proving that
this process is giving more problem than solutions. Oslo libs doing that
automatic update introduce more pain for all consumers than anything (at
least not in OpenStack).

You are likely largely shielded by the constraints list here which is
derivative of the global requirements list. Basically by using
constraints you get distilled global requirements and even without being
part of the requirements updates you'd be shielded from breakages when
installed via something like devstack or other deployment method using
constraints.

So if we care about Oslo users outside OpenStack, I beg us to stop this
crazyness. If we don't, we'll just spend time getting rid of Oslo over
the long term…

I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

At the meeting in Austin, the requirements team accepted my proposal
to stop syncing requirements updates into projects, as described
in https://etherpad.openstack.org/p/ocata-requirements-notes

We haven't been able to find anyone to work on the implementation,
though. I is my understanding that Tony did contact the Telemetry
and Swift teams, who are most interested in this area of change,
about devoting some resources to the tasks outlined in the proposal.

Doug

My 2c,

Cheers,

¹ Unless some API changed in a dep and we needed to raise the dep,
obviously.

--
Julien Danjou

Free Software hacker

https://julien.danjou.info

I don't have all the answers, but am fairly certain the situation we
have today is better than the one from several years ago. It is just not
perfect. I think we are better served by refining the current setup or
replacing it with something better but not by reverting.

Clark


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 19, 2017 by Doug_Hellmann (87,520 points)   3 4 8
0 votes

We have started this work. I've been working on:
https://review.openstack.org/#/c/444718/

Which will do requirement checks, as specified in the Pike PTG ehterpad for
Tuesday morning:
https://etherpad.openstack.org/p/relmgt-stable-requirements-ptg-pike (line
40+).

Once done, Tony and I were going to start testing it on the experimental
pipeline for Swift and Nova.

Regards,
Matt

On Thu, Apr 20, 2017 at 2:34 AM, Doug Hellmann doug@doughellmann.com
wrote:

Excerpts from Clark Boylan's message of 2017-04-19 08:10:43 -0700:

On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:

Hoy,

So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr"
and
some new release of oslo.* depends on pbr!=2.1.0.

Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
that got in banished by requirements Gods. It does not prevent it to be
used e.g. to install the software or get version information. But it
does break anything that is not in OpenStack because well, pip installs
the latest pbr (2.1.0) and then oslo.* is unhappy about it.

It actually breaks everything, including OpenStack. Shade and others are
affected by this as well. The specific problem here is that PBR is a
setuprequires which means it gets installed by easyinstall before
anything else. This means that the requirements restrictions are not
applied to it (neither are the constraints). So you get latest PBR from
easyinstall then later when something checks the requirements
(pkg
resources console script entrypoints?) they break because latest
PBR isn't allowed.

We need to stop pinning PBR and more generally stop pinning any
setup_requires (there are a few more now since setuptools itself is
starting to use that to list its deps rather than bundling them).

So I understand the culprit is probably pip installation scheme, and we
can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
avoid the entire issue.

Yes, a new release of PBR undoing the "pin" is the current sane step
forward for fixing this particular issue. Monty also suggested that we
gate global requirements changes on requiring changes not pin any
setup_requires.

But for the future, could we stop updating the requirements in oslo
libs
for no good reason? just because some random OpenStack project hit a
bug
somewhere?

For example, I've removed requirements update on tooz¹ for more than a
year now, which did not break anything in the meantime, proving that
this process is giving more problem than solutions. Oslo libs doing
that
automatic update introduce more pain for all consumers than anything
(at
least not in OpenStack).

You are likely largely shielded by the constraints list here which is
derivative of the global requirements list. Basically by using
constraints you get distilled global requirements and even without being
part of the requirements updates you'd be shielded from breakages when
installed via something like devstack or other deployment method using
constraints.

So if we care about Oslo users outside OpenStack, I beg us to stop this
crazyness. If we don't, we'll just spend time getting rid of Oslo over
the long term…

I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

At the meeting in Austin, the requirements team accepted my proposal
to stop syncing requirements updates into projects, as described
in https://etherpad.openstack.org/p/ocata-requirements-notes

We haven't been able to find anyone to work on the implementation,
though. I is my understanding that Tony did contact the Telemetry
and Swift teams, who are most interested in this area of change,
about devoting some resources to the tasks outlined in the proposal.

Doug

My 2c,

Cheers,

¹ Unless some API changed in a dep and we needed to raise the dep,
obviously.

--
Julien Danjou

Free Software hacker

https://julien.danjou.info

I don't have all the answers, but am fairly certain the situation we
have today is better than the one from several years ago. It is just not
perfect. I think we are better served by refining the current setup or
replacing it with something better but not by reverting.

Clark


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 20, 2017 by Matthew_Oliver (900 points)   1
0 votes

Doug Hellmann wrote:
Excerpts from Clark Boylan's message of 2017-04-19 08:10:43 -0700:

On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:

Hoy,

So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr" and
some new release of oslo.* depends on pbr!=2.1.0.

Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
that got in banished by requirements Gods. It does not prevent it to be
used e.g. to install the software or get version information. But it
does break anything that is not in OpenStack because well, pip installs
the latest pbr (2.1.0) and then oslo.* is unhappy about it.
It actually breaks everything, including OpenStack. Shade and others are
affected by this as well. The specific problem here is that PBR is a
setuprequires which means it gets installed by easyinstall before
anything else. This means that the requirements restrictions are not
applied to it (neither are the constraints). So you get latest PBR from
easyinstall then later when something checks the requirements
(pkg
resources console script entrypoints?) they break because latest
PBR isn't allowed.

We need to stop pinning PBR and more generally stop pinning any
setup_requires (there are a few more now since setuptools itself is
starting to use that to list its deps rather than bundling them).

So I understand the culprit is probably pip installation scheme, and we
can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
avoid the entire issue.
Yes, a new release of PBR undoing the "pin" is the current sane step
forward for fixing this particular issue. Monty also suggested that we
gate global requirements changes on requiring changes not pin any
setup_requires.

But for the future, could we stop updating the requirements in oslo libs
for no good reason? just because some random OpenStack project hit a bug
somewhere?

For example, I've removed requirements update on tooz¹ for more than a
year now, which did not break anything in the meantime, proving that
this process is giving more problem than solutions. Oslo libs doing that
automatic update introduce more pain for all consumers than anything (at
least not in OpenStack).
You are likely largely shielded by the constraints list here which is
derivative of the global requirements list. Basically by using
constraints you get distilled global requirements and even without being
part of the requirements updates you'd be shielded from breakages when
installed via something like devstack or other deployment method using
constraints.

So if we care about Oslo users outside OpenStack, I beg us to stop this
crazyness. If we don't, we'll just spend time getting rid of Oslo over
the long term…
I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

At the meeting in Austin, the requirements team accepted my proposal
to stop syncing requirements updates into projects, as described
in https://etherpad.openstack.org/p/ocata-requirements-notes

We haven't been able to find anyone to work on the implementation,
though. I is my understanding that Tony did contact the Telemetry
and Swift teams, who are most interested in this area of change,
about devoting some resources to the tasks outlined in the proposal.

Doug

My 2c,

Cheers,

Wasn't there also some decision made in austin (?) about how we as a
group stated something along the line of co-installability isn't as
important as it once was (and may not even be practical or what people
care about anymore anyway)?

With kolla becoming more popular (tripleo I think is using it, and ...)
and the containers it creates making isolated per-application
environments it makes me wonder what of global-requirements is still
valid (as a concept) and what isn't.

I do remember the days of free for all requirements (or requirements
sometimes just put/stashed in devstack vs elsewhere), which I don't
really want to go back to; but if we finally all agree that
co-installability isn't what people actually do and/or care about
(anymore?) then maybe we can re-think some things?

I personally still like having an ability to know some set of
requirements works for certain project X for a given release Z (as
tested by the gate); though I am not really concerned about if the same
set of requirements works for certain project Y (also in release Z). If
this is something others agree with then perhaps we just need to store
those requirements and the last constraints tested inside each project
(instead of storing it in a requirements repository)?

Sadly idk if that summit meeting (I think we had an etherpad?) ever went
anywhere. Probably because IMHO it's overwhelming to take into account
the amount of things that must be undone or changed to accomplish such a
goal; due to (but perhaps it's time we just tried).

My 4 cents,

-Josh


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 20, 2017 by harlowja_at_fastmail (16,200 points)   2 5 7
0 votes

Excerpts from Matthew Oliver's message of 2017-04-20 14:41:38 +1000:

We have started this work. I've been working on:
https://review.openstack.org/#/c/444718/

Wonderful! I'm sorry I didn't realize you were working on it. Thank you!

Which will do requirement checks, as specified in the Pike PTG ehterpad for
Tuesday morning:
https://etherpad.openstack.org/p/relmgt-stable-requirements-ptg-pike (line
40+).

Once done, Tony and I were going to start testing it on the experimental
pipeline for Swift and Nova.

That sounds like a good approach. I'll subscribe to the review and
follow along.

Doug

Regards,
Matt

On Thu, Apr 20, 2017 at 2:34 AM, Doug Hellmann doug@doughellmann.com
wrote:

Excerpts from Clark Boylan's message of 2017-04-19 08:10:43 -0700:

On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:

Hoy,

So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr"
and
some new release of oslo.* depends on pbr!=2.1.0.

Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
that got in banished by requirements Gods. It does not prevent it to be
used e.g. to install the software or get version information. But it
does break anything that is not in OpenStack because well, pip installs
the latest pbr (2.1.0) and then oslo.* is unhappy about it.

It actually breaks everything, including OpenStack. Shade and others are
affected by this as well. The specific problem here is that PBR is a
setuprequires which means it gets installed by easyinstall before
anything else. This means that the requirements restrictions are not
applied to it (neither are the constraints). So you get latest PBR from
easyinstall then later when something checks the requirements
(pkg
resources console script entrypoints?) they break because latest
PBR isn't allowed.

We need to stop pinning PBR and more generally stop pinning any
setup_requires (there are a few more now since setuptools itself is
starting to use that to list its deps rather than bundling them).

So I understand the culprit is probably pip installation scheme, and we
can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
avoid the entire issue.

Yes, a new release of PBR undoing the "pin" is the current sane step
forward for fixing this particular issue. Monty also suggested that we
gate global requirements changes on requiring changes not pin any
setup_requires.

But for the future, could we stop updating the requirements in oslo
libs
for no good reason? just because some random OpenStack project hit a
bug
somewhere?

For example, I've removed requirements update on tooz¹ for more than a
year now, which did not break anything in the meantime, proving that
this process is giving more problem than solutions. Oslo libs doing
that
automatic update introduce more pain for all consumers than anything
(at
least not in OpenStack).

You are likely largely shielded by the constraints list here which is
derivative of the global requirements list. Basically by using
constraints you get distilled global requirements and even without being
part of the requirements updates you'd be shielded from breakages when
installed via something like devstack or other deployment method using
constraints.

So if we care about Oslo users outside OpenStack, I beg us to stop this
crazyness. If we don't, we'll just spend time getting rid of Oslo over
the long term…

I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

At the meeting in Austin, the requirements team accepted my proposal
to stop syncing requirements updates into projects, as described
in https://etherpad.openstack.org/p/ocata-requirements-notes

We haven't been able to find anyone to work on the implementation,
though. I is my understanding that Tony did contact the Telemetry
and Swift teams, who are most interested in this area of change,
about devoting some resources to the tasks outlined in the proposal.

Doug

My 2c,

Cheers,

¹ Unless some API changed in a dep and we needed to raise the dep,
obviously.

--
Julien Danjou

Free Software hacker

https://julien.danjou.info

I don't have all the answers, but am fairly certain the situation we
have today is better than the one from several years ago. It is just not
perfect. I think we are better served by refining the current setup or
replacing it with something better but not by reverting.

Clark


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 20, 2017 by Doug_Hellmann (87,520 points)   3 4 8
0 votes

On 20/04/17 01:32 AM, Joshua Harlow wrote:
Wasn't there also some decision made in austin (?) about how we as a
group stated something along the line of co-installability isn't as
important as it once was (and may not even be practical or what people
care about anymore anyway)?

With kolla becoming more popular (tripleo I think is using it, and ...)
and the containers it creates making isolated per-application
environments it makes me wonder what of global-requirements is still
valid (as a concept) and what isn't.

I do remember the days of free for all requirements (or requirements
sometimes just put/stashed in devstack vs elsewhere), which I don't
really want to go back to; but if we finally all agree that
co-installability isn't what people actually do and/or care about
(anymore?) then maybe we can re-think some things?

agree with all of ^... but i imagine to make progress on this, we'd have
to change/drop devstack usage in gate and that will take forever and a
lifetime (is that a chick flick title?) given how embedded devstack is
in everything. it seems like the solution starts with devstack.

cheers,

--
gord


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 20, 2017 by gordon_chung (19,300 points)   2 3 8
0 votes

Excerpts from gordon chung's message of 2017-04-20 17:12:26 +0000:

On 20/04/17 01:32 AM, Joshua Harlow wrote:

Wasn't there also some decision made in austin (?) about how we as a
group stated something along the line of co-installability isn't as
important as it once was (and may not even be practical or what people
care about anymore anyway)?

I don't remember that, but I may not have been in the room at the
time. In the past when we've discussed that idea, we've continued
to maintain that co-installability is still needed for distributors
who have packaging constraints that require it and for use cases
like single-node deployments for POCs.

With kolla becoming more popular (tripleo I think is using it, and ...)
and the containers it creates making isolated per-application
environments it makes me wonder what of global-requirements is still
valid (as a concept) and what isn't.

We still need to review dependencies for license compatibility, to
minimize redundancy, and to ensure that we're not adding things to
the list that are not being maintained upstream. Even if we stop syncing
versions, official projects need to be those reviews, and having the
global list is a way to ensure that the reviews are done.

I do remember the days of free for all requirements (or requirements
sometimes just put/stashed in devstack vs elsewhere), which I don't
really want to go back to; but if we finally all agree that
co-installability isn't what people actually do and/or care about
(anymore?) then maybe we can re-think some things?

agree with all of ^... but i imagine to make progress on this, we'd have
to change/drop devstack usage in gate and that will take forever and a
lifetime (is that a chick flick title?) given how embedded devstack is
in everything. it seems like the solution starts with devstack.

cheers,


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 20, 2017 by Doug_Hellmann (87,520 points)   3 4 8
0 votes

Doug Hellmann wrote:
Excerpts from gordon chung's message of 2017-04-20 17:12:26 +0000:

On 20/04/17 01:32 AM, Joshua Harlow wrote:

Wasn't there also some decision made in austin (?) about how we as a
group stated something along the line of co-installability isn't as
important as it once was (and may not even be practical or what people
care about anymore anyway)?

I don't remember that, but I may not have been in the room at the
time. In the past when we've discussed that idea, we've continued
to maintain that co-installability is still needed for distributors
who have packaging constraints that require it and for use cases
like single-node deployments for POCs.

Ya, looking back I think it was:

https://etherpad.openstack.org/p/newton-global-requirements

I think that was robert that lead that session, but I might be incorrect
there.

With kolla becoming more popular (tripleo I think is using it, and ...)
and the containers it creates making isolated per-application
environments it makes me wonder what of global-requirements is still
valid (as a concept) and what isn't.

We still need to review dependencies for license compatibility, to
minimize redundancy, and to ensure that we're not adding things to
the list that are not being maintained upstream. Even if we stop syncing
versions, official projects need to be those reviews, and having the
global list is a way to ensure that the reviews are done.

I do remember the days of free for all requirements (or requirements
sometimes just put/stashed in devstack vs elsewhere), which I don't
really want to go back to; but if we finally all agree that
co-installability isn't what people actually do and/or care about
(anymore?) then maybe we can re-think some things?
agree with all of ^... but i imagine to make progress on this, we'd have
to change/drop devstack usage in gate and that will take forever and a
lifetime (is that a chick flick title?) given how embedded devstack is
in everything. it seems like the solution starts with devstack.

cheers,


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 21, 2017 by harlowja_at_fastmail (16,200 points)   2 5 7
0 votes

2017-04-19 23:10 GMT+08:00 Clark Boylan cboylan@sapwetik.org:

On Wed, Apr 19, 2017, at 05:54 AM, Julien Danjou wrote:

Hoy,

So Gnocchi gate is all broken (agaiiiin) because it depends on "pbr" and
some new release of oslo.* depends on pbr!=2.1.0.

Neither Gnocchi nor Oslo cares about whatever bug there is in pbr 2.1.0
that got in banished by requirements Gods. It does not prevent it to be
used e.g. to install the software or get version information. But it
does break anything that is not in OpenStack because well, pip installs
the latest pbr (2.1.0) and then oslo.* is unhappy about it.

It actually breaks everything, including OpenStack. Shade and others are
affected by this as well. The specific problem here is that PBR is a
setuprequires which means it gets installed by easyinstall before
anything else. This means that the requirements restrictions are not
applied to it (neither are the constraints). So you get latest PBR from
easyinstall then later when something checks the requirements
(pkg
resources console script entrypoints?) they break because latest
PBR isn't allowed.

What we disscuss here is the way to avoid break things, not sure we
add pbr into periodic-**-with-oslo-master works in
https://review.openstack.org/458753

We need to stop pinning PBR and more generally stop pinning any
setup_requires (there are a few more now since setuptools itself is
starting to use that to list its deps rather than bundling them).

So I understand the culprit is probably pip installation scheme, and we
can blame him until we fix it. I'm also trying to push pbr 2.2.0 to
avoid the entire issue.

Yes, a new release of PBR undoing the "pin" is the current sane step
forward for fixing this particular issue. Monty also suggested that we
gate global requirements changes on requiring changes not pin any
setup_requires.

But for the future, could we stop updating the requirements in oslo libs
for no good reason? just because some random OpenStack project hit a bug
somewhere?

For example, I've removed requirements update on tooz¹ for more than a
year now, which did not break anything in the meantime, proving that
this process is giving more problem than solutions. Oslo libs doing that
automatic update introduce more pain for all consumers than anything (at
least not in OpenStack).

You are likely largely shielded by the constraints list here which is
derivative of the global requirements list. Basically by using
constraints you get distilled global requirements and even without being
part of the requirements updates you'd be shielded from breakages when
installed via something like devstack or other deployment method using
constraints.

So if we care about Oslo users outside OpenStack, I beg us to stop this
crazyness. If we don't, we'll just spend time getting rid of Oslo over
the long term…

I think we know from experience that just stopping (eg reverting to the
situation we had before requirements and constraints) would lead to
sadness. Installations would frequently be impossible due to some
unresolvable error in dependency resolution. Do you have some
alternative in mind? Perhaps we loosen the in project requirements and
explicitly state that constraints are known to work due to testing and
users should use constraints? That would give users control to manage
their own constraints list too if they wish. Maybe we do this in
libraries while continuing to be more specific in applications?

My 2c,

Cheers,

¹ Unless some API changed in a dep and we needed to raise the dep,
obviously.

--
Julien Danjou

Free Software hacker

https://julien.danjou.info

I don't have all the answers, but am fairly certain the situation we
have today is better than the one from several years ago. It is just not
perfect. I think we are better served by refining the current setup or
replacing it with something better but not by reverting.

Clark


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 21, 2017 by ChangBo_Guo (4,540 points)   3 5
...