settingsLogin | Registersettings

Re: [Openstack-operators] [nova] Should we allow passing new user_data during rebuild?

0 votes

Excerpts from Matt Riedemann's message of 2017-10-03 10:53:44 -0500:

We plan on deprecating personality files from the compute API in a new
microversion. The spec for that is here:

https://review.openstack.org/#/c/509013/

Today you can pass new personality files to inject during rebuild, and
at the PTG we said we'd allow passing new user_data to rebuild as a
replacement for the personality files.

However, if the only reason one would need to pass personality files
during rebuild is because we don't persist them during the initial
server create, do we really need to also allow passing userdata for
rebuild? The initial user
data is stored with the instance during
create, and re-used during rebuild, so do we need to allow updating it
during rebuild?

My personal opinion is that rebuild is an anti-pattern for cloud, and
should be frozen and deprecated. It does nothing but complicate Nova
and present challenges for scaling.

That said, if it must stay as a feature, I don't think updating the
user_data should be a priority. At that point, you've basically created an
entirely new server, and you can already do that by creating an entirely
new server.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
asked Oct 6, 2017 in openstack-operators by Clint_Byrum (40,940 points)   4 5 9

15 Responses

0 votes

We use rebuild when reverting with snapshots. Keeping the same IP and hostname avoids some issues with Active Directory and Kerberos.

Tim

-----Original Message-----
From: Clint Byrum clint@fewbar.com
Date: Tuesday, 3 October 2017 at 19:17
To: openstack-operators openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [nova] Should we allow passing new user_data during rebuild?

Excerpts from Matt Riedemann's message of 2017-10-03 10:53:44 -0500:

We plan on deprecating personality files from the compute API in a new
microversion. The spec for that is here:

https://review.openstack.org/#/c/509013/

Today you can pass new personality files to inject during rebuild, and
at the PTG we said we'd allow passing new user_data to rebuild as a
replacement for the personality files.

However, if the only reason one would need to pass personality files
during rebuild is because we don't persist them during the initial
server create, do we really need to also allow passing userdata for
rebuild? The initial user
data is stored with the instance during
create, and re-used during rebuild, so do we need to allow updating it
during rebuild?

My personal opinion is that rebuild is an anti-pattern for cloud, and
should be frozen and deprecated. It does nothing but complicate Nova
and present challenges for scaling.

That said, if it must stay as a feature, I don't think updating the
user_data should be a priority. At that point, you've basically created an
entirely new server, and you can already do that by creating an entirely
new server.

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 3, 2017 by Tim_Bell (16,440 points)   1 5 10
0 votes

I fully appreciate that there are users of it today, and that it is
a thing that will likely live for years.

Long lived VMs can use all sorts of features to make VMs work more like
precious long lived servers. However, supporting these cases directly
doesn't make OpenStack scalable or simple. Quite the opposite.

It's worth noting that AD and Kerberos were definitely not designed
for clouds that have short lived VMs, so it does not surprise me that
treating VMs as cattle and then putting them in AD would confuse it.

Excerpts from Tim Bell's message of 2017-10-03 18:46:31 +0000:

We use rebuild when reverting with snapshots. Keeping the same IP and hostname avoids some issues with Active Directory and Kerberos.

Tim

-----Original Message-----
From: Clint Byrum clint@fewbar.com
Date: Tuesday, 3 October 2017 at 19:17
To: openstack-operators openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [nova] Should we allow passing new user_data during rebuild?

Excerpts from Matt Riedemann's message of 2017-10-03 10:53:44 -0500:
> We plan on deprecating personality files from the compute API in a new 
> microversion. The spec for that is here:
> 
> https://review.openstack.org/#/c/509013/
> 
> Today you can pass new personality files to inject during rebuild, and 
> at the PTG we said we'd allow passing new user_data to rebuild as a 
> replacement for the personality files.
> 
> However, if the only reason one would need to pass personality files 
> during rebuild is because we don't persist them during the initial 
> server create, do we really need to also allow passing user_data for 
> rebuild? The initial user_data is stored with the instance during 
> create, and re-used during rebuild, so do we need to allow updating it 
> during rebuild?
> 

My personal opinion is that rebuild is an anti-pattern for cloud, and
should be frozen and deprecated. It does nothing but complicate Nova
and present challenges for scaling.

That said, if it must stay as a feature, I don't think updating the
user_data should be a priority. At that point, you've basically created an
entirely new server, and you can already do that by creating an entirely
new server.

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 3, 2017 by Clint_Byrum (40,940 points)   4 5 9
0 votes

On Tue, Oct 03, 2017 at 01:00:13PM -0700, Clint Byrum wrote:

:It's worth noting that AD and Kerberos were definitely not designed
:for clouds that have short lived VMs, so it does not surprise me that
:treating VMs as cattle and then putting them in AD would confuse it.

For instances we have that need Kerberos keytabs we specify the fixed
IP. This works in our OpenStack where it's our IP space so PTR record also
matches, not so well in public cloud where we can reserve an IP and
set forward DNS but not control its reverse mapping.

-Jon

:Excerpts from Tim Bell's message of 2017-10-03 18:46:31 +0000:
:> We use rebuild when reverting with snapshots. Keeping the same IP and hostname avoids some issues with Active Directory and Kerberos.
:>
:> Tim
:>
:> -----Original Message-----
:> From: Clint Byrum clint@fewbar.com
:> Date: Tuesday, 3 October 2017 at 19:17
:> To: openstack-operators openstack-operators@lists.openstack.org
:> Subject: Re: [Openstack-operators] [nova] Should we allow passing new userdata during rebuild?
:>
:>
:> Excerpts from Matt Riedemann's message of 2017-10-03 10:53:44 -0500:
:> > We plan on deprecating personality files from the compute API in a new
:> > microversion. The spec for that is here:
:> >
:> > https://review.openstack.org/#/c/509013/
:> >
:> > Today you can pass new personality files to inject during rebuild, and
:> > at the PTG we said we'd allow passing new user
data to rebuild as a
:> > replacement for the personality files.
:> >
:> > However, if the only reason one would need to pass personality files
:> > during rebuild is because we don't persist them during the initial
:> > server create, do we really need to also allow passing userdata for
:> > rebuild? The initial user
data is stored with the instance during
:> > create, and re-used during rebuild, so do we need to allow updating it
:> > during rebuild?
:> >
:>
:> My personal opinion is that rebuild is an anti-pattern for cloud, and
:> should be frozen and deprecated. It does nothing but complicate Nova
:> and present challenges for scaling.
:>
:> That said, if it must stay as a feature, I don't think updating the
:> user_data should be a priority. At that point, you've basically created an
:> entirely new server, and you can already do that by creating an entirely
:> new server.
:>
:> _______________________________________________
:> OpenStack-operators mailing list
:> OpenStack-operators@lists.openstack.org
:> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
:>
:
:_______________________________________________
:OpenStack-operators mailing list
:OpenStack-operators@lists.openstack.org
:http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 3, 2017 by jon_at_csail.mit.edu (4,720 points)   1 4 6
0 votes

On 2017-10-03 16:19:27 -0400 (-0400), Jonathan Proulx wrote:
[...]
This works in our OpenStack where it's our IP space so PTR record also
matches, not so well in public cloud where we can reserve an IP and
set forward DNS but not control its reverse mapping.
[...]

Not that it probably helps, but I consider any public cloud which
doesn't give you some means of automatically setting reverse DNS
(either through an API or delegation to your own nameservers) to be
thoroughly broken, at least for Internet-facing use cases.
--
Jeremy Stanley


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

responded Oct 3, 2017 by Jeremy_Stanley (56,700 points)   3 5 7
0 votes

On Tue, Oct 03, 2017 at 08:29:45PM +0000, Jeremy Stanley wrote:
:On 2017-10-03 16:19:27 -0400 (-0400), Jonathan Proulx wrote:
:[...]
:> This works in our OpenStack where it's our IP space so PTR record also
:> matches, not so well in public cloud where we can reserve an IP and
:> set forward DNS but not control its reverse mapping.
:[...]
:
:Not that it probably helps, but I consider any public cloud which
:doesn't give you some means of automatically setting reverse DNS
:(either through an API or delegation to your own nameservers) to be
:thoroughly broken, at least for Internet-facing use cases.

we wander off topic...and I wander waaay off topic below...

but I have exactly 1 instance in AWS where I
care about this, perhaps I just don't care enough to have found the
answer or perhaps for count of 1 it's not worth solving.

Then again perhaps AWS is just actually the trash is seem to be to
me. I've been trying to like it since before there was an OpenStack,
but the more I try the less I can stand it. People who use AWS and
complain about OpenStack UX baffle me, there's a lot OpenStack can do
to impove UX but it's waaay better than my AWS experices. I mean it
was fewer steps to enable ipv6 on my OpenStack provider networks than
it was to get it working in my AWS VPC and neutron isn't the poster
child for simplicity.

:--
:Jeremy Stanley

:_______________________________________________
:OpenStack-operators mailing list
:OpenStack-operators@lists.openstack.org
:http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

responded Oct 3, 2017 by jon_at_csail.mit.edu (4,720 points)   1 4 6
0 votes

On 10/03/2017 11:12 AM, Clint Byrum wrote:

My personal opinion is that rebuild is an anti-pattern for cloud, and
should be frozen and deprecated. It does nothing but complicate Nova
and present challenges for scaling.

That said, if it must stay as a feature, I don't think updating the
user_data should be a priority. At that point, you've basically created an
entirely new server, and you can already do that by creating an entirely
new server.

If you've got a whole heat stack with multiple resources, and you realize that
you messed up one thing in the template and one of your servers has the wrong
personality/user_data, it can be useful to be able to rebuild that one server
without affecting anything else in the stack. That's just a convenience though.

Chris


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 4, 2017 by Chris_Friesen (20,420 points)   3 16 24
0 votes

In our cloud rebuild is the only way for a user to keep the same IP.
Unfortunately, we don't offer floating IPs, yet.
Also, we use the userdata to bootstrap some actions in new instances
(puppet, ...).
Considering all the use-cases for rebuild it would be great if the
user
data can be updated at rebuild time.

On Wed, Oct 4, 2017 at 5:15 PM, Chris Friesen chris.friesen@windriver.com
wrote:

On 10/03/2017 11:12 AM, Clint Byrum wrote:

My personal opinion is that rebuild is an anti-pattern for cloud, and

should be frozen and deprecated. It does nothing but complicate Nova
and present challenges for scaling.

That said, if it must stay as a feature, I don't think updating the
user_data should be a priority. At that point, you've basically created an
entirely new server, and you can already do that by creating an entirely
new server.

If you've got a whole heat stack with multiple resources, and you realize
that you messed up one thing in the template and one of your servers has
the wrong personality/user_data, it can be useful to be able to rebuild
that one server without affecting anything else in the stack. That's just
a convenience though.

Chris


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 4, 2017 by Belmiro_Moreira (2,620 points)   2 4
0 votes

In our cloud, we offer the possibility to reinstall the same or another OS on a VPS (Virtual Private Server). Unfortunately, we couldn’t use the rebuild function because of the VPS‘s use of Cinder for root disk. We create a new instance and inject the same User Data so that the new instance has the same password and key as the last one. It also has the same name, and the same floating IP is attached. I believe it even has the same IPv6 through some Neutron port magic.

BTW, you wouldn’t believe how often people use the Reinstall feature.

Tomas from Homeatcloud

From: Belmiro Moreira [mailto:moreira.belmiro.email.lists@gmail.com]
Sent: Wednesday, October 04, 2017 5:34 PM
To: Chris Friesen
Cc: openstack-operators@lists.openstack.org
Subject: Re: [Openstack-operators] [nova] Should we allow passing new user_data during rebuild?

In our cloud rebuild is the only way for a user to keep the same IP. Unfortunately, we don't offer floating IPs, yet.

Also, we use the user_data to bootstrap some actions in new instances (puppet, ...).

Considering all the use-cases for rebuild it would be great if the user_data can be updated at rebuild time.

On Wed, Oct 4, 2017 at 5:15 PM, Chris Friesen chris.friesen@windriver.com wrote:

On 10/03/2017 11:12 AM, Clint Byrum wrote:

My personal opinion is that rebuild is an anti-pattern for cloud, and
should be frozen and deprecated. It does nothing but complicate Nova
and present challenges for scaling.

That said, if it must stay as a feature, I don't think updating the
user_data should be a priority. At that point, you've basically created an
entirely new server, and you can already do that by creating an entirely
new server.

If you've got a whole heat stack with multiple resources, and you realize that you messed up one thing in the template and one of your servers has the wrong personality/user_data, it can be useful to be able to rebuild that one server without affecting anything else in the stack. That's just a convenience though.

Chris


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 5, 2017 by vondra_at_homeatclou (780 points)  
0 votes

Excerpts from Chris Friesen's message of 2017-10-04 09:15:28 -0600:

On 10/03/2017 11:12 AM, Clint Byrum wrote:

My personal opinion is that rebuild is an anti-pattern for cloud, and
should be frozen and deprecated. It does nothing but complicate Nova
and present challenges for scaling.

That said, if it must stay as a feature, I don't think updating the
user_data should be a priority. At that point, you've basically created an
entirely new server, and you can already do that by creating an entirely
new server.

If you've got a whole heat stack with multiple resources, and you realize that
you messed up one thing in the template and one of your servers has the wrong
personality/user_data, it can be useful to be able to rebuild that one server
without affecting anything else in the stack. That's just a convenience though.

If you just changed that personality/user_data in the template, Heat
would spin up a new one, change all the references to it, wait for any
wait conditions to fire, allowing dependent servers to reconfigure with
the new one and acknowledge that, and then delete the old one for you.

Making your app work like this means being able to replace failed or
undersized servers with less downtime. You can do other things too,
like spin up a replacement in a different AZ to deal with maintenance
issues on your side or the cloud's side. Or you can deploy a new image,
without any downtime.

My point remains: rebuild (and resize) train users to see a server as
precious, instead of training users to write automation that expects
cloud servers to come and go often.

This, btw, is one reason I like that EC2 calls them instances and not
servers. They're not servers. We call them servers, but they're just
little regions of memory on actual servers, and as such, they're not
precious, and should be discarded as necessary.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 5, 2017 by Clint_Byrum (40,940 points)   4 5 9
0 votes

Excerpts from Belmiro Moreira's message of 2017-10-04 17:33:40 +0200:

In our cloud rebuild is the only way for a user to keep the same IP.
Unfortunately, we don't offer floating IPs, yet.
Also, we use the userdata to bootstrap some actions in new instances
(puppet, ...).
Considering all the use-cases for rebuild it would be great if the
user
data can be updated at rebuild time.

Indeed, it sounds like we're too far down the rabbit hole with rebuild to
stop digging.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 5, 2017 by Clint_Byrum (40,940 points)   4 5 9
...