settingsLogin | Registersettings

Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno "alive" for longer.

0 votes

On a personal level, supporting the same release of an open source project for 5 years is something you should pay for...dearly. If operators have customers that are pinned to Juno for some reason I couldn't imagine right now, and they're willing to pay us to support it, then great!

But I think we need to very tightly scope what support means- Absolutely no back or forward porting. The features you have now are frozen in time. Also, they need to be tightly pinned to the OS distro repo versions of packages so we don't have to care about fixing critical vulns in stuff we don't maintain and can't control. This basically means they'll be paying us to make sure they can upgrade distro packages for security reasons and that OpenStack will keep functioning, and to file & patch upstream OpenStack bugs.

Effectively this means they're settling for less value for their money if they remain on Juno for the full 5 years, whereas customers using newer versions of operators' OpenStack offerings will be getting new development and features for the same support dollars (which is a good way to market new versions to them, BTW).

My $0.02

--
Tom Cameron


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
asked Nov 9, 2015 in openstack-operators by Tom_Cameron (320 points)  
retagged Feb 4, 2016 by admin

30 Responses

0 votes

BTW, what's the EOL date of Juno?

On Mon, Nov 9, 2015 at 11:50 PM, Tom Cameron Tom.Cameron@rackspace.com
wrote:

On a personal level, supporting the same release of an open source project
for 5 years is something you should pay for...dearly. If operators have
customers that are pinned to Juno for some reason I couldn't imagine right
now, and they're willing to pay us to support it, then great!

But I think we need to very tightly scope what support means- Absolutely
no back or forward porting. The features you have now are frozen in time.
Also, they need to be tightly pinned to the OS distro repo versions of
packages so we don't have to care about fixing critical vulns in stuff we
don't maintain and can't control. This basically means they'll be paying us
to make sure they can upgrade distro packages for security reasons and that
OpenStack will keep functioning, and to file & patch upstream OpenStack
bugs.

Effectively this means they're settling for less value for their money if
they remain on Juno for the full 5 years, whereas customers using newer
versions of operators' OpenStack offerings will be getting new development
and features for the same support dollars (which is a good way to market
new versions to them, BTW).

My $0.02

--
Tom Cameron


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Gareth

Cloud Computing, OpenStack, Distributed Storage, Fitness, Basketball
OpenStack contributor, kun_huang@freenode
My promise: if you find any spelling or grammar mistakes in my email from
Mar 1 2013, notify me *
*and I'll donate $1 or ¥1 to an open organization you specify.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Gareth (1,800 points)   6
0 votes

On 2015-11-10 00:18:34 +0800 (+0800), Gareth wrote:
BTW, what's the EOL date of Juno?

"2014.2.4 (eol) early November, 2015."

This was decided at the stable branch management session for the
Liberty Design Summit in Vancouver, BC back in May of this year.

So basically any time after 2014.2.4 is tagged on all the
release-managed stable-branch deliverables for Juno, the final state
of the corresponding stable/juno branches will be tagged as juno-eol
and then the stable/juno branches themselves deleted, per our usual
EOL process. After which time the Project Infrastructure team will
delete stable/juno CI jobs and dismantle any Juno-specific support
infrastructure (for example, CentOS 6.x workers which were kept to
support contemporary Python 2.6 compatibility testing).
--
Jeremy Stanley


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

responded Nov 9, 2015 by Jeremy_Stanley (56,700 points)   3 5 7
0 votes

disclaimer: I’ve never worked in a software auditing department or on in a company with one

What about risk-averse organizations with strict policy compliance guidelines? Can we expect them to audit a new distribution of Openstack every 6 months? Some sort of community-supported LTS system would at least give these consulting firms a base on which to build such a compliant Openstack distribution for industry X.

If we’re only talking about patches to support minor updates to system packages what’s the cost to the community?

I’m not against Tom’s idea and would be satisfied with it but it would be better, I think, to at least give the community an option of a solid base on which to build a compliant Openstack distribution that isn’t going to move out from underneath them in six months.

Unless of course that should be the job of some distribution maintainer… in which case how to we work with them?

On Nov 9, 2015, at 10:50 AM, Tom Cameron Tom.Cameron@rackspace.com wrote:

On a personal level, supporting the same release of an open source project for 5 years is something you should pay for...dearly. If operators have customers that are pinned to Juno for some reason I couldn't imagine right now, and they're willing to pay us to support it, then great!

But I think we need to very tightly scope what support means- Absolutely no back or forward porting. The features you have now are frozen in time. Also, they need to be tightly pinned to the OS distro repo versions of packages so we don't have to care about fixing critical vulns in stuff we don't maintain and can't control. This basically means they'll be paying us to make sure they can upgrade distro packages for security reasons and that OpenStack will keep functioning, and to file & patch upstream OpenStack bugs.

Effectively this means they're settling for less value for their money if they remain on Juno for the full 5 years, whereas customers using newer versions of operators' OpenStack offerings will be getting new development and features for the same support dollars (which is a good way to market new versions to them, BTW).

My $0.02

--
Tom Cameron


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by James_King (240 points)  
0 votes

Thanks Jeremy,

I hope it come earlier personally. Our company is designing next could
product release, based on Juno. Now I have powerful talking points ;)

On Tue, Nov 10, 2015 at 12:43 AM, Jeremy Stanley fungi@yuggoth.org wrote:
On 2015-11-10 00:18:34 +0800 (+0800), Gareth wrote:

BTW, what's the EOL date of Juno?

"2014.2.4 (eol) early November, 2015."

#Planned_stable.2Fjuno_releases_.2812_months.29 >

This was decided at the stable branch management session for the
Liberty Design Summit in Vancouver, BC back in May of this year.

So basically any time after 2014.2.4 is tagged on all the
release-managed stable-branch deliverables for Juno, the final state
of the corresponding stable/juno branches will be tagged as juno-eol
and then the stable/juno branches themselves deleted, per our usual
EOL process. After which time the Project Infrastructure team will
delete stable/juno CI jobs and dismantle any Juno-specific support
infrastructure (for example, CentOS 6.x workers which were kept to
support contemporary Python 2.6 compatibility testing).
--
Jeremy Stanley


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Gareth

Cloud Computing, OpenStack, Distributed Storage, Fitness, Basketball
OpenStack contributor, kun_huang@freenode
My promise: if you find any spelling or grammar mistakes in my email
from Mar 1 2013, notify me
and I'll donate $1 or ¥1 to an open organization you specify.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Gareth (1,800 points)   6
0 votes

James,

What about risk-averse organizations with strict policy compliance guidelines?

I strongly suspect most operators don't have customers (internal or otherwise) clamouring to upgrade every 6 months. But 5 years is, frankly, absurd. But, to the point about auditing, many organizations that have requirements around the auditing of software as relates to regulations (this is a pretty small set of users). Many of them can rely on external audits of software, so perhaps this would be an opportunity for the Openstack foundation to have long-term supported releases audited?

If we’re only talking about patches to support minor updates to system packages what’s the cost to the community?

Minor patches to distribution supplied packages isn't the actual problem, though there is some cost to the community in making sure that the minor updates (we're talking only security patches here at some point) doesn't break some depending component within Openstack. That should be a rare problem, but I could see something like an OpenSSL vulnerability found in TLS1.x requiring a fix like disabling TLS1.x, which would break a python library that doesn't yet support anything >TLS1.x. Sometimes these fixes get very cumbersome to carry forward for half a decade.

The real cost, though, is in the Openstack foundation and the community developers maintaining the actual release of the Openstack software components. At some point in 5 years, your pool of experts on that version of the software dwindles rapidly. Because most of the development is done by groups pushing the leading edge, the trailing tail gets less and less attention until nobody's working on it at all. Obviously that isn't the case for Juno right now, but in 5 years time, I can't imagine any volunteer wanting to support a very outdated, infrequently used version of an open source project.

Anyway, I suspect we're in violent agreement here. I support an LTS release strategy because it will allow more adoption for more sectors by offering that stability everyone's talking about. But, it shouldn't be a super-super long support offering. Maybe steal some of Ubuntu's game and do an LTS every 4 releases or so (24 months), but then maybe Openstack only supports them for 24 months time? Again, my concern is that this is free, open source software and you're probably not going to get many community members to volunteer to offer their precious time fixing bugs in a 2-year-old codebase that have been fixed for 18 months in a newer version. Sometimes backporting those fixes is more difficult than the actual fix itself, which makes the offer even less appealing.

It's good to see the discussion, though!

--
Tom Cameron


From: James King j.kenneth.king@gmail.com on behalf of James King james@agentultra.com
Sent: Monday, November 9, 2015 11:47
To: Tom Cameron
Cc: OpenStack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno "alive" for longer.

disclaimer: I’ve never worked in a software auditing department or on in a company with one

What about risk-averse organizations with strict policy compliance guidelines? Can we expect them to audit a new distribution of Openstack every 6 months? Some sort of community-supported LTS system would at least give these consulting firms a base on which to build such a compliant Openstack distribution for industry X.

If we’re only talking about patches to support minor updates to system packages what’s the cost to the community?

I’m not against Tom’s idea and would be satisfied with it but it would be better, I think, to at least give the community an option of a solid base on which to build a compliant Openstack distribution that isn’t going to move out from underneath them in six months.

Unless of course that should be the job of some distribution maintainer… in which case how to we work with them?

On Nov 9, 2015, at 10:50 AM, Tom Cameron Tom.Cameron@rackspace.com wrote:

On a personal level, supporting the same release of an open source project for 5 years is something you should pay for...dearly. If operators have customers that are pinned to Juno for some reason I couldn't imagine right now, and they're willing to pay us to support it, then great!

But I think we need to very tightly scope what support means- Absolutely no back or forward porting. The features you have now are frozen in time. Also, they need to be tightly pinned to the OS distro repo versions of packages so we don't have to care about fixing critical vulns in stuff we don't maintain and can't control. This basically means they'll be paying us to make sure they can upgrade distro packages for security reasons and that OpenStack will keep functioning, and to file & patch upstream OpenStack bugs.

Effectively this means they're settling for less value for their money if they remain on Juno for the full 5 years, whereas customers using newer versions of operators' OpenStack offerings will be getting new development and features for the same support dollars (which is a good way to market new versions to them, BTW).

My $0.02

--
Tom Cameron


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Tom_Cameron (320 points)  
0 votes

On 2015-11-09 17:11:35 +0000 (+0000), Tom Cameron wrote:
[...]
I support an LTS release strategy because it will allow more
adoption for more sectors by offering that stability everyone's
talking about. But, it shouldn't be a super-super long support
offering. Maybe steal some of Ubuntu's game and do an LTS every 4
releases or so (24 months), but then maybe Openstack only supports
them for 24 months time? Again, my concern is that this is free,
open source software and you're probably not going to get many
community members to volunteer to offer their precious time fixing
bugs in a 2-year-old codebase that have been fixed for 18 months
in a newer version.
[...]

Because we want people to be able upgrade their deployments, the
problem runs deeper than just backporting some fixes to a particular
branch for longer periods of time. Unfortunately the original poster
cross-posted this thread to multiple mailing lists so the discussion
has rapidly bifurcated, but I addressed this particular topic in my
http://lists.openstack.org/pipermail/openstack-dev/2015-November/078735.html
reply.
--
Jeremy Stanley


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Jeremy_Stanley (56,700 points)   3 5 7
0 votes

Excerpts from James King's message of 2015-11-09 08:47:08 -0800:

disclaimer: I’ve never worked in a software auditing department or on in a company with one

What about risk-averse organizations with strict policy compliance guidelines? Can we expect them to audit a new distribution of Openstack every 6 months? Some sort of community-supported LTS system would at least give these consulting firms a base on which to build such a compliant Openstack distribution for industry X.

Nobody has said that the idea of an LTS is bad. The realities are
simply challenging. Upgrades from release to release are already
painful, which I suspect is at least part of the reason many are still
on older releases. Upgrading across 2 years of development would
possibly be a herculean effort that so far nobody has even tried to
tackle without stepping through all intermediary releases.

If we’re only talking about patches to support minor updates to system packages what’s the cost to the community?

The two biggest are the testing infrastructure and maintaining upgrade
support. The first is already stretched thin, and the latter has
managed to scare away anyone who brings up LTS releases without them
even attempting it.

I’m not against Tom’s idea and would be satisfied with it but it would be better, I think, to at least give the community an option of a solid base on which to build a compliant Openstack distribution that isn’t going to move out from underneath them in six months.

Unless of course that should be the job of some distribution maintainer… in which case how to we work with them?

We do work with distro maintainers. They do a ton of work in the stable
branches.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Clint_Byrum (40,940 points)   4 6 10
0 votes

From your other thread...

Or else you're saying you intend to fix the current inability of our projects to skip intermediate releases entirely during upgrades

I think without knowing it, that's what most would be suggesting, yeah. Of course, like you mentioned, the real work is in how upgrades get refactored to skip intermediate releases (two or three of them).

DB schema changes can basically be rolled up and kept around for a while, so that's not too be a problem. Config files OTOH have no schema or schema validator, so that would require tooling and all kinds of fun (bug prone) wizardry.

This is all solvable, but it adds complexity for the sake of what I can only imagine are the extreme minority of users. What do the user/operator surveys say about the usage of older releases? What portion of the user base is actually on releases prior to Havana?

--
Tom Cameron


From: Jeremy Stanley fungi@yuggoth.org
Sent: Monday, November 9, 2015 12:35
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno "alive" for longer.

On 2015-11-09 17:11:35 +0000 (+0000), Tom Cameron wrote:
[...]
I support an LTS release strategy because it will allow more
adoption for more sectors by offering that stability everyone's
talking about. But, it shouldn't be a super-super long support
offering. Maybe steal some of Ubuntu's game and do an LTS every 4
releases or so (24 months), but then maybe Openstack only supports
them for 24 months time? Again, my concern is that this is free,
open source software and you're probably not going to get many
community members to volunteer to offer their precious time fixing
bugs in a 2-year-old codebase that have been fixed for 18 months
in a newer version.
[...]

Because we want people to be able upgrade their deployments, the
problem runs deeper than just backporting some fixes to a particular
branch for longer periods of time. Unfortunately the original poster
cross-posted this thread to multiple mailing lists so the discussion
has rapidly bifurcated, but I addressed this particular topic in my
http://lists.openstack.org/pipermail/openstack-dev/2015-November/078735.html
reply.
--
Jeremy Stanley


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Tom_Cameron (320 points)  
0 votes

One thing more to consider, live upgrades still arn't a thing yet, but getting closer. Being able to do it with single version upgrades is a pretty hard thing. Doing it across LTS style releases wouldn't work without a huge amount of effort all on its own.

We may be better off waiting until we get a core OpenStack release that allows live upgrades to work smoothly before calling that thing the first LTS release.

Thanks,
Kevin


From: Tom Cameron [Tom.Cameron@rackspace.com]
Sent: Monday, November 09, 2015 11:01 AM
To: Jeremy Stanley; openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno "alive" for longer.

From your other thread...

Or else you're saying you intend to fix the current inability of our projects to skip intermediate releases entirely during upgrades

I think without knowing it, that's what most would be suggesting, yeah. Of course, like you mentioned, the real work is in how upgrades get refactored to skip intermediate releases (two or three of them).

DB schema changes can basically be rolled up and kept around for a while, so that's not too be a problem. Config files OTOH have no schema or schema validator, so that would require tooling and all kinds of fun (bug prone) wizardry.

This is all solvable, but it adds complexity for the sake of what I can only imagine are the extreme minority of users. What do the user/operator surveys say about the usage of older releases? What portion of the user base is actually on releases prior to Havana?

--
Tom Cameron


From: Jeremy Stanley fungi@yuggoth.org
Sent: Monday, November 9, 2015 12:35
To: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-dev] [stable][all] Keeping Juno "alive" for longer.

On 2015-11-09 17:11:35 +0000 (+0000), Tom Cameron wrote:
[...]
I support an LTS release strategy because it will allow more
adoption for more sectors by offering that stability everyone's
talking about. But, it shouldn't be a super-super long support
offering. Maybe steal some of Ubuntu's game and do an LTS every 4
releases or so (24 months), but then maybe Openstack only supports
them for 24 months time? Again, my concern is that this is free,
open source software and you're probably not going to get many
community members to volunteer to offer their precious time fixing
bugs in a 2-year-old codebase that have been fixed for 18 months
in a newer version.
[...]

Because we want people to be able upgrade their deployments, the
problem runs deeper than just backporting some fixes to a particular
branch for longer periods of time. Unfortunately the original poster
cross-posted this thread to multiple mailing lists so the discussion
has rapidly bifurcated, but I addressed this particular topic in my
http://lists.openstack.org/pipermail/openstack-dev/2015-November/078735.html
reply.
--
Jeremy Stanley


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Fox,_Kevin_M (29,360 points)   1 3 4
0 votes

On 2015-11-09 19:01:36 +0000 (+0000), Tom Cameron wrote:
[...]
What do the user/operator surveys say about the usage of older
releases? What portion of the user base is actually on releases
prior to Havana?

The most recent OpenStack User Survey Report has an awesome trending
analysis answering this exact question. See page 22 (labeled 21) of
https://www.openstack.org/assets/survey/Public-User-Survey-Report.pdf
for a very appropriate graph.
--
Jeremy Stanley


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Nov 9, 2015 by Jeremy_Stanley (56,700 points)   3 5 7
...