settingsLogin | Registersettings

[OpenStack-DefCore] Image APIs in Glance and Nova

0 votes

Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just released, not really deployed anywhere)

For the


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
asked Jun 17, 2015 in defcore-committee by John_Garbutt (15,460 points)   3 4 5

35 Responses

0 votes

For the... ?

Is there more? :)

On Wed, Jun 17, 2015 at 10:56 AM, John Garbutt john@johngarbutt.com wrote:
Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just released, not really deployed anywhere)

For the


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 17, 2015 by Chris_Lee (180 points)   1
0 votes

oops, continued...

On 17 June 2015 at 17:56, John Garbutt john@johngarbutt.com wrote:
Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just fully released (kilo?), not deployed everywhere, yet)
* glance is building a new v3 API (user could help implement that, I suppose)

Now I think the best way forward here is to pick the Nova list images,
for the moment.

Problem 2:
Upload an image

I would argue there is no truly interoperable way to do this right
now. Glance currently has three ways to upload an image, and no way to
know which the deployer has picked.

For details see (20 mins in):
https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/liberty-product-management-and-openstack-technology

Now there should be a way, long term, but that needs fixing before
defcore can test that. I would chose fixing the glance v2 upload, over
the glance v2 tasks or glance v1)

Problem 3:
Pick the Ubuntu 12.04 base image

We don't have a standard way to pick these that all clouds have
adopted. We should create one of these.

So I am going to stop there, for the moment. The defcore effort
highlights some difficulties that the project need to go and work on,
and I think thats awesome.

There was talk about the current state of Nova and image APIs, here is
a summary...

Nova currently talks to an internal glance v1 API, and if thats not
available you can't boot from a glance image. It could be a public
glance v1, but that is not how glance v1 was designed, it was meant to
be an internal API (although not sure everyone got the memo, or it
never got written, I am unsure)

Users are able to run both Glance v1 and Glance v2 talking to the same
datastore, thats how Rackspace has Glance v1 for Nova (internal only)
and Glance v2 for public use.

Nova has a glance v1 proxy API. That API will always implement the
glance v1 API, currently it does that by talking to the glance v1 API,
it may do that in the future by talking to the glance v2 API. Nova has
no plans to add a glance v2 proxy API, because proxy APIs suck (with
timeouts), and the glance v2 API was built to be exposed to end users.

There were plans to make Nova talk to glance v2, but the kilo effort
never got complete. Largely because of some features that are still
missing in glance v2 (and the patch never got reviewed, largely as
everyone though it was still a work in progress). I am told there are
plans to get those features added into glance v2 during liberty.
However that means Nova will require a liberty glance v2, if
configured to use glance v2.

I don't see how any of this Nova work should really block the defcore
effort in terms of picking which image API to depend on. But its very
possible I am missing something.

Thanks,
John


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 17, 2015 by John_Garbutt (15,460 points)   3 4 5
0 votes

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just fully released (kilo?), not deployed everywhere, yet)
* glance is building a new v3 API (user could help implement that, I suppose)

Now I think the best way forward here is to pick the Nova list images,
for the moment.

Problem 2:
Upload an image

I would argue there is no truly interoperable way to do this right
now. Glance currently has three ways to upload an image, and no way to
know which the deployer has picked.

I think this where the "let things that aren't implemented fail" comes into play. If we can't know which paths/workflows a deployer has used, try the approaches from an approved/accepted scenarios list and report which ones work and which ones don't. Even if only one of those scenarios is the approved or blessed path, I think it would also be helpful for identifying and gathering data around which scenarios deployers support/use the most.

Daryl


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 17, 2015 by Daryl_Walleck (760 points)   1
0 votes

I don't see how any of this Nova work should really block the defcore
effort in terms of picking which image API to depend on. But its very
possible I am missing something.

I don’t think it does, but it does bring up a point that folks really need to be aware of (and I’m not sure everyone is). Namely: DefCore in it’s current form is intentionally a lagging indicator. E.g. something can’t become DefCore-required until (among other things) it shows proven usage out in the wild [1] and has a level of API stability [2]. DefCore Guidelines typically cover three OpenStack releases [3]. DefCore can’t just “pick a winner” among the available methods that exist today. The impact of that on the current image upload predicament goes something like this:

  • use glance v2 (only just fully released (kilo?), not deployed everywhere, yet)

Assuming you’re correct about Kilo, this would be grounds for v2 image upload to not be DefCore-required for now. The next DefCore guideline would cover Juno, Kilo, and Liberty, and Glance v2 image upload has neither met the >2 releases bar yet nor was it available in Juno. I’ll note that it’s also been pointed out that the Glance task API for importing images is incomplete even in Kilo, so it can’t really be DefCore-required either (and can’t be until 2017 at the earliest).

  • glance is building a new v3 API (user could help implement that, I suppose)

Assuming this lands in Liberty, we wouldn’t be able to DefCore-require it until 2017 (the first time it could conceivably meet the >2 releases bar and the first time Liberty would be the eldest release covered by a Guideline).

  • use glance v1 (should never be exposed to end users, was designed as internal only API)

Assuming a lot of other operators have come to the same conclusion and if we assume for a moment that Glance is thinking of deprecating it soonish (since they’re already working on v3), then this fails to meet the bars for future direction and wide deployment, so we can’t DefCore-require it either.

I would argue there is no truly interoperable way to do this right now.

That sure seems to be the case, which stinks. =/

The good news is, the process is working to the degree that we’re spotlighting major interoperability pain points like this now. The next test is to prove that we as a community aren’t just paying lip service to interoperability by figuring out how to solve these problems sooner rather than later, because it’s going to be a while before users of OpenStack Powered (TM) clouds can depend on the solutions even if we wizarded up a magical set of patches that solved this today.

IMHO hashing out a solution will likely require some discussion and agreement among a couple of different groups within the community (Nova, Glance, & DefCore all seem to have skin in the game here). What do you think is the best way to start tackling these sorts of things? Cross-project specs? Start nailing items to the TC meeting agendas? Let ML discussions like this start driving priority lists [4] for the relevant projects? Have inter-project liaisons [5] take it up?

[1] http://git.openstack.org/cgit/openstack/defcore/tree/process/CoreCriteria.rst#n36
[2] http://git.openstack.org/cgit/openstack/defcore/tree/process/CoreCriteria.rst#n50
[3] http://git.openstack.org/cgit/openstack/defcore/tree/HACKING.rst#n38
[4] Nova has nicely laid out http://specs.openstack.org/openstack/nova-specs/priorities/liberty-priorities.html but most projects unfortunately don’t.
[5] https://wiki.openstack.org/wiki/CrossProjectLiaisons#Inter-project_Liaisons

At Your Service,

Mark T. Voelker

On Jun 17, 2015, at 1:15 PM, John Garbutt john@johngarbutt.com wrote:

oops, continued...

On 17 June 2015 at 17:56, John Garbutt john@johngarbutt.com wrote:

Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just fully released (kilo?), not deployed everywhere, yet)
* glance is building a new v3 API (user could help implement that, I suppose)

Now I think the best way forward here is to pick the Nova list images,
for the moment.

Problem 2:
Upload an image

I would argue there is no truly interoperable way to do this right
now. Glance currently has three ways to upload an image, and no way to
know which the deployer has picked.

For details see (20 mins in):
https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/liberty-product-management-and-openstack-technology

Now there should be a way, long term, but that needs fixing before
defcore can test that. I would chose fixing the glance v2 upload, over
the glance v2 tasks or glance v1)

Problem 3:
Pick the Ubuntu 12.04 base image

We don't have a standard way to pick these that all clouds have
adopted. We should create one of these.

So I am going to stop there, for the moment. The defcore effort
highlights some difficulties that the project need to go and work on,
and I think thats awesome.

There was talk about the current state of Nova and image APIs, here is
a summary...

Nova currently talks to an internal glance v1 API, and if thats not
available you can't boot from a glance image. It could be a public
glance v1, but that is not how glance v1 was designed, it was meant to
be an internal API (although not sure everyone got the memo, or it
never got written, I am unsure)

Users are able to run both Glance v1 and Glance v2 talking to the same
datastore, thats how Rackspace has Glance v1 for Nova (internal only)
and Glance v2 for public use.

Nova has a glance v1 proxy API. That API will always implement the
glance v1 API, currently it does that by talking to the glance v1 API,
it may do that in the future by talking to the glance v2 API. Nova has
no plans to add a glance v2 proxy API, because proxy APIs suck (with
timeouts), and the glance v2 API was built to be exposed to end users.

There were plans to make Nova talk to glance v2, but the kilo effort
never got complete. Largely because of some features that are still
missing in glance v2 (and the patch never got reviewed, largely as
everyone though it was still a work in progress). I am told there are
plans to get those features added into glance v2 during liberty.
However that means Nova will require a liberty glance v2, if
configured to use glance v2.

I don't see how any of this Nova work should really block the defcore
effort in terms of picking which image API to depend on. But its very
possible I am missing something.

Thanks,
John


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 17, 2015 by Mark_Voelker (4,000 points)   2 3
0 votes

try the approaches from an approved/accepted scenarios list and report which ones work and which ones don’t.

So that’s baaaasically what developers have to do today, and it stinks for them. Monty has elaborated on this at length, but in a nutshell you end up writing all manner of if/else loops into your code to deal with all the peculiarities of different clouds you might want to use. If DefCore is successful, it means a lot of these if/else situations will go away because anytime you’re using an OpenStack Powered Platform, you can depend on a certain set of capabilities to be available to you.

Interoperability by if/else loop isn’t interoperability at all.

If we can't know which paths/workflows a deployer has used

RefStack should eventually help us understand exactly that, since it’ll provide data on all the tests that pass/fail (not just what’s currently DefCore-required). But it’s early days...

At Your Service,

Mark T. Voelker

On Jun 17, 2015, at 2:52 PM, Daryl Walleck daryl.walleck@RACKSPACE.COM wrote:

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just fully released (kilo?), not deployed everywhere, yet)
* glance is building a new v3 API (user could help implement that, I suppose)

Now I think the best way forward here is to pick the Nova list images,
for the moment.

Problem 2:
Upload an image

I would argue there is no truly interoperable way to do this right
now. Glance currently has three ways to upload an image, and no way to
know which the deployer has picked.

I think this where the "let things that aren't implemented fail" comes into play. If we can't know which paths/workflows a deployer has used, try the approaches from an approved/accepted scenarios list and report which ones work and which ones don't. Even if only one of those scenarios is the approved or blessed path, I think it would also be helpful for identifying and gathering data around which scenarios deployers support/use the most.

Daryl


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 17, 2015 by Mark_Voelker (4,000 points)   2 3
0 votes

On 06/17/2015 03:27 PM, Mark Voelker wrote:

I don't see how any of this Nova work should really block the
defcore effort in terms of picking which image API to depend on.
But its very possible I am missing something.

I don’t think it does, but it does bring up a point that folks really
need to be aware of (and I’m not sure everyone is). Namely: DefCore
in it’s current form is intentionally a lagging indicator. E.g.
something can’t become DefCore-required until (among other things) it
shows proven usage out in the wild [1] and has a level of API
stability [2]. DefCore Guidelines typically cover three OpenStack
releases [3]. DefCore can’t just “pick a winner” among the available
methods that exist today. The impact of that on the current image
upload predicament goes something like this:

  • use glance v2 (only just fully released (kilo?), not deployed
    everywhere, yet)

Assuming you’re correct about Kilo, this would be grounds for v2
image upload to not be DefCore-required for now. The next DefCore
guideline would cover Juno, Kilo, and Liberty, and Glance v2 image
upload has neither met the >2 releases bar yet nor was it available
in Juno. I’ll note that it’s also been pointed out that the Glance
task API for importing images is incomplete even in Kilo, so it can’t
really be DefCore-required either (and can’t be until 2017 at the
earliest).

  • glance is building a new v3 API (user could help implement that,
    I suppose)

Assuming this lands in Liberty, we wouldn’t be able to
DefCore-require it until 2017 (the first time it could conceivably
meet the >2 releases bar and the first time Liberty would be the
eldest release covered by a Guideline).

  • use glance v1 (should never be exposed to end users, was designed
    as internal only API)

Assuming a lot of other operators have come to the same conclusion
and if we assume for a moment that Glance is thinking of deprecating
it soonish (since they’re already working on v3), then this fails to
meet the bars for future direction and wide deployment, so we can’t
DefCore-require it either.

FWIW, all of the public clouds with I think 2 exceptions use glance v1
and expose it to the end users. However, the other cloud that uses v2
still uses PUT as the upload, so the v2-ness of the API exposed is minimal.

I'm not suggesting that it be defcore-required - you're right, it's not
appropriate given the current state ... but image upload needs to be
required and there needs to be a fix to the absolute mess we have right
now where one cloud does it one way and is the only one that does, and
all of the others do it the other way, even if people sometimes say "you
shouldn't do that"

I would argue there is no truly interoperable way to do this right
now.

That sure seems to be the case, which stinks. =/

It's very painful - although we do have a library now in openstack that
knows how to do the right thing with each cloud. It's sad we had to
write it, but if you use python and/or ansible, you've got a workaround
for a while.

The good news is, the process is working to the degree that we’re
spotlighting major interoperability pain points like this now.

Totally agree!

The
next test is to prove that we as a community aren’t just paying lip
service to interoperability by figuring out how to solve these
problems sooner rather than later, because it’s going to be a while
before users of OpenStack Powered (TM) clouds can depend on the
solutions even if we wizarded up a magical set of patches that solved
this today.

Agree even more.

IMHO hashing out a solution will likely require some discussion and
agreement among a couple of different groups within the community
(Nova, Glance, & DefCore all seem to have skin in the game here).
What do you think is the best way to start tackling these sorts of
things? Cross-project specs? Start nailing items to the TC meeting
agendas? Let ML discussions like this start driving priority lists
[4] for the relevant projects? Have inter-project liaisons [5] take
it up?

I know there are several of these problems we talked about on the
technical side of the fence at the summit - specifically around the need
to at the very least provide clearer direction outbound from the tech
community as to what the intent is. As you say, it won't fix existing
deploys tomorrow - but yeah, I think starting to write some
cross-project specs here is a great choice.

[2]
http://git.openstack.org/cgit/openstack/defcore/tree/process/CoreCriteria.rst#n50

[3]
http://git.openstack.org/cgit/openstack/defcore/tree/HACKING.rst#n38
[4] Nova has nicely laid out
http://specs.openstack.org/openstack/nova-specs/priorities/liberty-priorities.html
but most projects unfortunately don’t. [5]
https://wiki.openstack.org/wiki/CrossProjectLiaisons#Inter-project_Liaisons

At Your Service,

Mark T. Voelker

On Jun 17, 2015, at 1:15 PM, John Garbutt john@johngarbutt.com
wrote:

oops, continued...

On 17 June 2015 at 17:56, John Garbutt john@johngarbutt.com
wrote:

Hi,

So this was raised in the cross project meeting, and thank you
for that.

I keep mentioning use cases, so here we go... * create from cloud
base image for ubuntu 12.04 * above but with boot from volume *
create a snapshot * download snapshot * upload image into another
openstack cloud * boot server from uploaded image

Now, at a higher level: * user does above using custom script for
Cloud A and Cloud B * user keeps to just the APIs that are
defcore tested * user gets access to Cloud C and Cloud D * user
wants to point script at new clouds, and everything should just
work

So I think thats where we want to be. Now lets dig in...

To list images, the user could: * use nova to list images
(stable, but project wants to delete it) * use glance v1 (should
never be exposed to end users, was designed as internal only
API) * use glance v2 (only just fully released (kilo?), not
deployed everywhere, yet) * glance is building a new v3 API (user
could help implement that, I suppose)

Now I think the best way forward here is to pick the Nova list
images, for the moment.

Problem 2: Upload an image

I would argue there is no truly interoperable way to do this right
now. Glance currently has three ways to upload an image, and no way
to know which the deployer has picked.

For details see (20 mins in):
https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/liberty-product-management-and-openstack-technology

Now there should be a way, long term, but that needs fixing before

defcore can test that. I would chose fixing the glance v2 upload,
over the glance v2 tasks or glance v1)

Problem 3: Pick the Ubuntu 12.04 base image

We don't have a standard way to pick these that all clouds have
adopted. We should create one of these.

So I am going to stop there, for the moment. The defcore effort
highlights some difficulties that the project need to go and work
on, and I think thats awesome.

There was talk about the current state of Nova and image APIs, here
is a summary...

Nova currently talks to an internal glance v1 API, and if thats
not available you can't boot from a glance image. It could be a
public glance v1, but that is not how glance v1 was designed, it
was meant to be an internal API (although not sure everyone got the
memo, or it never got written, I am unsure)

Users are able to run both Glance v1 and Glance v2 talking to the
same datastore, thats how Rackspace has Glance v1 for Nova
(internal only) and Glance v2 for public use.

Nova has a glance v1 proxy API. That API will always implement the
glance v1 API, currently it does that by talking to the glance v1
API, it may do that in the future by talking to the glance v2 API.
Nova has no plans to add a glance v2 proxy API, because proxy APIs
suck (with timeouts), and the glance v2 API was built to be exposed
to end users.

There were plans to make Nova talk to glance v2, but the kilo
effort never got complete. Largely because of some features that
are still missing in glance v2 (and the patch never got reviewed,
largely as everyone though it was still a work in progress). I am
told there are plans to get those features added into glance v2
during liberty. However that means Nova will require a liberty
glance v2, if configured to use glance v2.

I don't see how any of this Nova work should really block the
defcore effort in terms of picking which image API to depend on.
But its very possible I am missing something.

Thanks, John

_______________________________________________ Defcore-committee
mailing list Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee

_______________________________________________ Defcore-committee
mailing list Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 17, 2015 by Monty_Taylor (22,780 points)   2 5 8
0 votes

On 06/17/2015 01:15 PM, John Garbutt wrote:
oops, continued...

On 17 June 2015 at 17:56, John Garbutt john@johngarbutt.com wrote:

Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just fully released (kilo?), not deployed everywhere, yet)
* glance is building a new v3 API (user could help implement that, I suppose)

Now I think the best way forward here is to pick the Nova list images,
for the moment.

Problem 2:
Upload an image

I would argue there is no truly interoperable way to do this right
now. Glance currently has three ways to upload an image, and no way to
know which the deployer has picked.

For details see (20 mins in):
https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/liberty-product-management-and-openstack-technology

Omg. A video to one of my talks is linked.

Slides are here:

http://inaugust.com/talks/product-management.html#/

The image-related stuff starts here:

http://inaugust.com/talks/product-management.html#/17

BTW - if you want more fun, you can enjoy the part in a follow up talk
about networking issues:

http://inaugust.com/talks/now-what.html#/31

Now there should be a way, long term, but that needs fixing before
defcore can test that. I would chose fixing the glance v2 upload, over
the glance v2 tasks or glance v1)

Yes please. It's the sanest of the three.

Problem 3:
Pick the Ubuntu 12.04 base image

We don't have a standard way to pick these that all clouds have
adopted. We should create one of these.

So I am going to stop there, for the moment. The defcore effort
highlights some difficulties that the project need to go and work on,
and I think thats awesome.

There was talk about the current state of Nova and image APIs, here is
a summary...

Nova currently talks to an internal glance v1 API, and if thats not
available you can't boot from a glance image. It could be a public
glance v1, but that is not how glance v1 was designed, it was meant to
be an internal API (although not sure everyone got the memo, or it
never got written, I am unsure)

Users are able to run both Glance v1 and Glance v2 talking to the same
datastore, thats how Rackspace has Glance v1 for Nova (internal only)
and Glance v2 for public use.

Nova has a glance v1 proxy API. That API will always implement the
glance v1 API, currently it does that by talking to the glance v1 API,
it may do that in the future by talking to the glance v2 API. Nova has
no plans to add a glance v2 proxy API, because proxy APIs suck (with
timeouts), and the glance v2 API was built to be exposed to end users.

There were plans to make Nova talk to glance v2, but the kilo effort
never got complete. Largely because of some features that are still
missing in glance v2 (and the patch never got reviewed, largely as
everyone though it was still a work in progress). I am told there are
plans to get those features added into glance v2 during liberty.
However that means Nova will require a liberty glance v2, if
configured to use glance v2.

I don't see how any of this Nova work should really block the defcore
effort in terms of picking which image API to depend on. But its very
possible I am missing something.

Thanks,
John


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 17, 2015 by Monty_Taylor (22,780 points)   2 5 8
0 votes

I'm replying to this version of John's mail to expand on his "use case" point.

What John calls a use case is, in this instance, the same as a high level "Test Case". Essentially, this is the documentation/comments that define what the test(s) for this case are doing. DefCore (or others) could create these high level test cases and use them as the framework into which new tests, specific for the capability, could be written. This, I think, would be an ideal way to start documenting missing capabilities tests. For some capabilities, there could be multiple use cases (or test cases) that define the capability.

This could also be used for existing tests, but instead of including code for each case or step in a case, the git repo link to the existing test could be included. This process could also be used with new tests being linked into the test case document as they are merged into the trunk.

This process provides the higher level test "specs" that developers implement into tests. It provides both a way to track what tests exist and what don't, and a description of the capability that is understandable by end users and other interested parties not concerned with the "guts," but just the behavior of the capability.

For the case of multiple ways of performing an action, the tests would need to implement all methods that meet DefCore criteria. If any of the methods result in the appropriate end result, then for DefCore purposes, the test has passed.

Does this make sense for DefCore? As OpenStack matures and acts more like a collection of products rather than a collection of code bases, the processes and collateral of software process that exist to turn code into product come more into play. Hence, the QA as not just verification gate, but a measurement and metrics repository.

Sorry! Had to get this thought out. You don't need the QA that most SW companies have until you have product. We are getting close to one and so are now in need of expanding OpenStack QA to encompass the other.

--Rocky

-----Original Message-----
From: John Garbutt [mailto:john@johngarbutt.com]
Sent: Wednesday, June 17, 2015 09:57
To: defcore-commit.
Cc: Nikhil Komawar
Subject: [OpenStack-DefCore] Image APIs in Glance and Nova

Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just released, not really deployed anywhere)

For the


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 18, 2015 by Rochelle_Grober (7,040 points)   1 3 3
0 votes

On Jun 17, 2015, at 8:51 PM, Rochelle Grober rochelle.grober@huawei.com wrote:

I'm replying to this version of John's mail to expand on his "use case" point.

What John calls a use case is, in this instance, the same as a high level "Test Case". Essentially, this is the documentation/comments that define what the test(s) for this case are doing. DefCore (or others) could create these high level test cases and use them as the framework into which new tests, specific for the capability, could be written. This, I think, would be an ideal way to start documenting missing capabilities tests. For some capabilities, there could be multiple use cases (or test cases) that define the capability.

This could also be used for existing tests, but instead of including code for each case or step in a case, the git repo link to the existing test could be included. This process could also be used with new tests being linked into the test case document as they are merged into the trunk.

This process provides the higher level test "specs" that developers implement into tests. It provides both a way to track what tests exist and what don't, and a description of the capability that is understandable by end users and other interested parties not concerned with the "guts," but just the behavior of the capability.

For the case of multiple ways of performing an action, the tests would need to implement all methods that meet DefCore criteria. If any of the methods result in the appropriate end result, then for DefCore purposes, the test has passed.

Maybe I’m misunderstanding, but this seems hugely problematic for interoperability. If we say there’s more than one way to do a thing but you pass if you implement any one of those ways, then app developers still don’t know what methods they can depend on being available. Instead of interoperable clouds we have clouds that can do the same things but for which I still have to write a bunch of this**:

def listimages():
“”” A function to list images. Because all OpenStack Powered Platforms can do that…somehow.””"
if $cloud == ‘vendorA’:
# TODO: this also works for vendorX
list
imagesvianovaimageapi()
elif $cloud == ‘vendorB’:
# TODO: this also worked for vendorY last week but now, um?
listimagesviaglancev1()
elif $cloud == ‘vendorC’:
listimagesviaglancev2()
else:
# I dunno what cloud this is, but it’s OpenStack Powered! So something must work.
try:
listimagesvianovaimageapi()
except NopeError:
# D’oh, guess that wasn’t it…
try:
list
imagesviaglancev1()
except StillNopeError:
# Aww…well third time’s the charm?
try:
list
imagesviaglancev2()
except NopeNopeNopeError:
rage
quit()

into any app that I want to be able to run on OpenStack Powered Platforms. Which is a Bad Thing and basically where we’re at now. How you expose a capability matters tremendously. IMHO, interoperability by if/else loop isn’t interoperability at all.

** I’m strawmanning slightly here in that I’m pretending these three methods all somehow met DefCore criteria, but you get the picture.

At Your Service,

Mark T. Voelker

Does this make sense for DefCore? As OpenStack matures and acts more like a collection of products rather than a collection of code bases, the processes and collateral of software process that exist to turn code into product come more into play. Hence, the QA as not just verification gate, but a measurement and metrics repository.

Sorry! Had to get this thought out. You don't need the QA that most SW companies have until you have product. We are getting close to one and so are now in need of expanding OpenStack QA to encompass the other.

--Rocky

-----Original Message-----
From: John Garbutt [mailto:john@johngarbutt.com]
Sent: Wednesday, June 17, 2015 09:57
To: defcore-commit.
Cc: Nikhil Komawar
Subject: [OpenStack-DefCore] Image APIs in Glance and Nova

Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just released, not really deployed anywhere)

For the


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 18, 2015 by Mark_Voelker (4,000 points)   2 3
0 votes

I think what Rocky is speaking to here is similar to the line of thinking I had earlier, which I may not have communicated as well.

As a tester and a test developer, I want to identify high level use cases or scenarios, and exercise all possible paths so that I have a thorough picture of the behaviors of the system under test. This can conflict with the notion of a DefCore capability, which speaks to having a single, guaranteed paths to perform an operation. This is where I see the DefCore test selection process kicks in and the workflows that define interop can be selected, given that the tests are designed in an atomic fashion such that individual tests are tightly scoped and do not creep into steps that may not relate directly to the capability under test. Tests for distinct functionality can still exist to satisfy test coverage, which then can then be used as data about what functionality deployers are exposing since one of the aspects of the process is to report all Tempest test results and not just the selected ones.

While I've seen the Glance example bounced around a bit, it sounds like there must be other similar areas of concern, which understanding would help fill in some of the knowledge gaps around these concerns. Is there an existing list of more examples of inconsistencies that are specifically a concern for interop?

Thanks,

Daryl

Sent from my Windows Phone


From: Mark Voelkermvoelker@vmware.com
Sent: ‎6/‎17/‎2015 10:08 PM
To: Rochelle Groberrochelle.grober@huawei.com
Cc: defcore-committee@lists.openstack.org
Subject: Re: [OpenStack-DefCore] Identifying and defining missing/new tests for capabilities :Was Image APIs in Glance and Nova

On Jun 17, 2015, at 8:51 PM, Rochelle Grober rochelle.grober@huawei.com wrote:

I'm replying to this version of John's mail to expand on his "use case" point.

What John calls a use case is, in this instance, the same as a high level "Test Case". Essentially, this is the documentation/comments that define what the test(s) for this case are doing. DefCore (or others) could create these high level test cases and use them as the framework into which new tests, specific for the capability, could be written. This, I think, would be an ideal way to start documenting missing capabilities tests. For some capabilities, there could be multiple use cases (or test cases) that define the capability.

This could also be used for existing tests, but instead of including code for each case or step in a case, the git repo link to the existing test could be included. This process could also be used with new tests being linked into the test case document as they are merged into the trunk.

This process provides the higher level test "specs" that developers implement into tests. It provides both a way to track what tests exist and what don't, and a description of the capability that is understandable by end users and other interested parties not concerned with the "guts," but just the behavior of the capability.

For the case of multiple ways of performing an action, the tests would need to implement all methods that meet DefCore criteria. If any of the methods result in the appropriate end result, then for DefCore purposes, the test has passed.

Maybe I’m misunderstanding, but this seems hugely problematic for interoperability. If we say there’s more than one way to do a thing but you pass if you implement any one of those ways, then app developers still don’t know what methods they can depend on being available. Instead of interoperable clouds we have clouds that can do the same things but for which I still have to write a bunch of this**:

def listimages():
“”” A function to list images. Because all OpenStack Powered Platforms can do that…somehow.””"
if $cloud == ‘vendorA’:
# TODO: this also works for vendorX
list
imagesvianovaimageapi()
elif $cloud == ‘vendorB’:
# TODO: this also worked for vendorY last week but now, um?
listimagesviaglancev1()
elif $cloud == ‘vendorC’:
listimagesviaglancev2()
else:
# I dunno what cloud this is, but it’s OpenStack Powered! So something must work.
try:
listimagesvianovaimageapi()
except NopeError:
# D’oh, guess that wasn’t it…
try:
list
imagesviaglancev1()
except StillNopeError:
# Aww…well third time’s the charm?
try:
list
imagesviaglancev2()
except NopeNopeNopeError:
rage
quit()

into any app that I want to be able to run on OpenStack Powered Platforms. Which is a Bad Thing and basically where we’re at now. How you expose a capability matters tremendously. IMHO, interoperability by if/else loop isn’t interoperability at all.

** I’m strawmanning slightly here in that I’m pretending these three methods all somehow met DefCore criteria, but you get the picture.

At Your Service,

Mark T. Voelker

Does this make sense for DefCore? As OpenStack matures and acts more like a collection of products rather than a collection of code bases, the processes and collateral of software process that exist to turn code into product come more into play. Hence, the QA as not just verification gate, but a measurement and metrics repository.

Sorry! Had to get this thought out. You don't need the QA that most SW companies have until you have product. We are getting close to one and so are now in need of expanding OpenStack QA to encompass the other.

--Rocky

-----Original Message-----
From: John Garbutt [mailto:john@johngarbutt.com]
Sent: Wednesday, June 17, 2015 09:57
To: defcore-commit.
Cc: Nikhil Komawar
Subject: [OpenStack-DefCore] Image APIs in Glance and Nova

Hi,

So this was raised in the cross project meeting, and thank you for that.

I keep mentioning use cases, so here we go...
* create from cloud base image for ubuntu 12.04
* above but with boot from volume
* create a snapshot
* download snapshot
* upload image into another openstack cloud
* boot server from uploaded image

Now, at a higher level:
* user does above using custom script for Cloud A and Cloud B
* user keeps to just the APIs that are defcore tested
* user gets access to Cloud C and Cloud D
* user wants to point script at new clouds, and everything should just work

So I think thats where we want to be. Now lets dig in...

To list images, the user could:
* use nova to list images (stable, but project wants to delete it)
* use glance v1 (should never be exposed to end users, was designed as
internal only API)
* use glance v2 (only just released, not really deployed anywhere)

For the


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee

Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Jun 18, 2015 by Daryl_Walleck (760 points)   1
...