settingsLogin | Registersettings

[openstack-dev] [nova] placement/resource providers update 27

0 votes

After 40 days in the desert I've returned with placement update 27.

Unfortunately, as far as I can tell, no one did any updates while I
was gone so I don't have anything to crib from to have the full
story on what's going on. I suspect I will miss some relevant
reviews when making this list. If I have, please let me know.
Otherwise, let's begin:

What Matters Most

Claims in the scheduler remains the key feature we'd like to get in
before feature freeze. After some hiccups on how to do it, making
requests of the new /allocation_candidates (of which more, below) is
the way to go. Changes for that are starting at

 https://review.openstack.org/#/c/476631/

What's Changed

As mentioned, there's now a new URL in the placement API:
GET /allocationcandidates. It has a similar interface to GET
/resource
providers (in that you can filter the results by the kind
of resources required) but the information is formatted as a
two-tuple of lists of allocation requests and a dictionary of
resource provider information. The latter will provide the initial
list of available resource providers and augment the process of
filtering and weighing those providers. The former provides a
collection of correctly formed JSON bodies that can be sent in a PUT
to /allocations/{consumer_uuid} when making a claim.

I'm still a bit confused about where the concept of "alternatives"
that are going to be passed to the cell conductors fits into this,
but I guess that will become more clear soon.

It also seems like this model creates a pretty strong conceptual
coupling between a thing which behaves like a nova-scheduler
(request, process, then claim resources). As placement becomes
useful to other services it will be important to revisit some of
these decisions and make sure the HTTP API is not imposing too many
behaviuor requirements on the client side (otherwise why bother
having an HTTP API?). But that's for later. Right now we're on a
tight schedule trying to make sure that claims get in in Ocata.

Because there's a bit of a dependency hierarchy with the various
threads of work going on in placement, the work on claims may punt
traits and/or nested resource providers further down the timeline.
Work continues on all three concurrently.

Another change is that allocations now include project id and user
id information and usages by those id can be retrieved.

Help Wanted

Areas where volunteers are needed.

Main Themes

Claims in the Scheduler

As described above there's been a change in direction. That probably
means some or all of the code now at

 https://review.openstack.org/#/q/status:open+topic:bp/placement-claims

can be abandoned in favor of the work at

 https://review.openstack.org/#/q/topic:bp/placement-allocation-requests+status:open

The main starting point for that is

 https://review.openstack.org/#/c/476631/

Traits

The concept of traits now exists in the placement service, but
filtering resource providers on traits is in flux. With the advent
of /allocation_candidates as the primary scheduling interface, that
needs to support traits. Work for that is in a stack starting at

 https://review.openstack.org/#/c/478464/

It's not yet clear if we'll want to support traits at both
/allocationcandidates and /resourceproviders. I think we should,
but the immediate need is on /allocation_candidates.

There's some proposed code to get the latter started:

 https://review.openstack.org/#/c/474602/

Shared Resource Providers

Support for shared resource providers is "built in" to the
/allocation_candidates concept and one of the drivers for having it.

Nested Resource Providers

Work continues on nested resource providers.

   https://review.openstack.org/#/q/status:open+topic:bp/nested-resource-providers

The need with these is simply more review, but they are behind
claims in priority.

Docs

Lots of placement-related api docs have merged or are in progress:

 https://review.openstack.org/#/q/status:open+topic:cd/placement-api-ref

Shortly there will be a real publishing job:

 https://review.openstack.org/#/c/480991/

and the tooling which tests that new handlers are documented
will be turned on:

 https://review.openstack.org/#/c/480924/

Some changes have been proposed to document the scheduler's
workflow, including visual aids, starting at:

 https://review.openstack.org/#/c/475810/

Other Code/Specs

End

That's all I've got this week, next week I should be a bit more
caught up and aware of any bits I've missed. No prize this week, but
maybe next week.

--
Chris Dent ┬──┬◡ノ(° -°ノ) https://anticdent.org/
freenode: cdent tw: @anticdent__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

asked Jul 8, 2017 in openstack-dev by cdent_plus_os_at_ant (12,800 points)   2 2 5

3 Responses

0 votes

Thanks for resuming these, Chris, much appreciated. Comments inline.

On 07/07/2017 07:44 AM, Chris Dent wrote:

After 40 days in the desert I've returned with placement update 27.

Unfortunately, as far as I can tell, no one did any updates while I
was gone so I don't have anything to crib from to have the full
story on what's going on. I suspect I will miss some relevant
reviews when making this list. If I have, please let me know.
Otherwise, let's begin:

Actually, you did a great job catching up, thank you!

What Matters Most

Claims in the scheduler remains the key feature we'd like to get in
before feature freeze. After some hiccups on how to do it, making
requests of the new /allocation_candidates (of which more, below) is
the way to go. Changes for that are starting at

 https://review.openstack.org/#/c/476631/

Well, the above is the only remaining non-WIP patch that hasn't already
merged. The remainder of the series:

https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic:bp/placement-allocation-requests

has merged in the last couple weeks. We've made a lot of progress on it.

The patch above is currently approved and on its way through the gate.

The patch after it:

https://review.openstack.org/#/c/476632/

Is what I'm currently actively working on and performs the resource
claims within the scheduler by picking one of the allocation candidates
returned from the call to GET /allocationcandidates and using that
candidate as the HTTP request body to PUT /allocations/{consumer
uuid}.

I'm hoping to have that patch un-WIP'd and ready for review by the end
of day today.

What's Changed

As mentioned, there's now a new URL in the placement API:
GET /allocationcandidates. It has a similar interface to GET
/resource
providers (in that you can filter the results by the kind
of resources required) but the information is formatted as a
two-tuple of lists of allocation requests and a dictionary of
resource provider information. The latter will provide the initial
list of available resource providers and augment the process of
filtering and weighing those providers. The former provides a
collection of correctly formed JSON bodies that can be sent in a PUT
to /allocations/{consumer_uuid} when making a claim.

Yup, exactly. This was Dan Smith's idea and I really like it. It makes
the returned allocation request body opaque (to systems other than the
scheduler and placement service) so that things like the cell-level
conductor and nova-compute service can literally pass a different
allocation request JSON object to PUT /allocations/{consumer_uuid} and
we get a clean, consistent "retry" mechanism that operators wanted to keep.

I'm still a bit confused about where the concept of "alternatives"
that are going to be passed to the cell conductors fits into this,
but I guess that will become more clear soon.

Yes, this should become clearer soon, but essentially we'll be changing
the scheduler's select_destinations() RPC method to return not just the
destination hosts selected by the scheduler for a request but also one
or more allocation candidates that matched the original launch request
for resources. These JSON objects will be opaque to everything outside
of the scheduler/placement services and, as noted above, will serve as
the clean, consistent retry mechanism within the cells. This retry
mechanism is missing in cellsv2 architecture due to the way that cells
cannot do "upcalls" to the API layer.

It also seems like this model creates a pretty strong conceptual
coupling between a thing which behaves like a nova-scheduler
(request, process, then claim resources).

Conceptual coupling between nova-scheduler and what? It looks like you
cut off your question above mid-stream :)

As placement becomes useful to other services it will be important to
revisit some of these decisions and make sure the HTTP API is not
imposing too many behaviuor requirements on the client side
(otherwise why bother having an HTTP API?). But that's for later.
Right now we're on a tight schedule trying to make sure that claims
get in in Ocata.

The concept of a claim is simply an atomic creation of related
allocation records for a consumer against one or more resource
providers. What are the behaviour requirements on the client side that
worry you? Let's discuss those concerns here and work through them.

Because there's a bit of a dependency hierarchy with the various
threads of work going on in placement, the work on claims may punt
traits and/or nested resource providers further down the timeline.
Work continues on all three concurrently.

Right, all three continue concurrently.

Another change is that allocations now include project id and user
id information and usages by those id can be retrieved.

Yup, this is A Good Thing :) I look forward to the day we can replace
the old os-simple-tenant-usage Compute API with something that's more
efficient!

Best,
-jay

Help Wanted

Areas where volunteers are needed.

Main Themes

Claims in the Scheduler

As described above there's been a change in direction. That probably
means some or all of the code now at

 https://review.openstack.org/#/q/status:open+topic:bp/placement-claims

can be abandoned in favor of the work at

 

https://review.openstack.org/#/q/topic:bp/placement-allocation-requests+status:open

The main starting point for that is

 https://review.openstack.org/#/c/476631/

Traits

The concept of traits now exists in the placement service, but
filtering resource providers on traits is in flux. With the advent
of /allocation_candidates as the primary scheduling interface, that
needs to support traits. Work for that is in a stack starting at

 https://review.openstack.org/#/c/478464/

It's not yet clear if we'll want to support traits at both
/allocationcandidates and /resourceproviders. I think we should,
but the immediate need is on /allocation_candidates.

There's some proposed code to get the latter started:

 https://review.openstack.org/#/c/474602/

Shared Resource Providers

Support for shared resource providers is "built in" to the
/allocation_candidates concept and one of the drivers for having it.

Nested Resource Providers

Work continues on nested resource providers.

   

https://review.openstack.org/#/q/status:open+topic:bp/nested-resource-providers

The need with these is simply more review, but they are behind
claims in priority.

Docs

Lots of placement-related api docs have merged or are in progress:

 

https://review.openstack.org/#/q/status:open+topic:cd/placement-api-ref

Shortly there will be a real publishing job:

 https://review.openstack.org/#/c/480991/

and the tooling which tests that new handlers are documented
will be turned on:

 https://review.openstack.org/#/c/480924/

Some changes have been proposed to document the scheduler's
workflow, including visual aids, starting at:

 https://review.openstack.org/#/c/475810/

Other Code/Specs

End

That's all I've got this week, next week I should be a bit more
caught up and aware of any bits I've missed. No prize this week, but
maybe next week.


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 7, 2017 by Jay_Pipes (59,760 points)   3 10 14
0 votes

On 7/7/2017 6:44 AM, Chris Dent wrote:

After 40 days in the desert I've returned with placement update 27.

Unfortunately, as far as I can tell, no one did any updates while I
was gone so I don't have anything to crib from to have the full
story on what's going on. I suspect I will miss some relevant
reviews when making this list. If I have, please let me know.
Otherwise, let's begin:

What Matters Most

Claims in the scheduler remains the key feature we'd like to get in
before feature freeze. After some hiccups on how to do it, making
requests of the new /allocation_candidates (of which more, below) is
the way to go. Changes for that are starting at

 https://review.openstack.org/#/c/476631/

What's Changed

As mentioned, there's now a new URL in the placement API:
GET /allocationcandidates. It has a similar interface to GET
/resource
providers (in that you can filter the results by the kind
of resources required) but the information is formatted as a
two-tuple of lists of allocation requests and a dictionary of
resource provider information. The latter will provide the initial
list of available resource providers and augment the process of
filtering and weighing those providers. The former provides a
collection of correctly formed JSON bodies that can be sent in a PUT
to /allocations/{consumer_uuid} when making a claim.

I'm still a bit confused about where the concept of "alternatives"
that are going to be passed to the cell conductors fits into this,
but I guess that will become more clear soon.

It also seems like this model creates a pretty strong conceptual
coupling between a thing which behaves like a nova-scheduler
(request, process, then claim resources). As placement becomes
useful to other services it will be important to revisit some of
these decisions and make sure the HTTP API is not imposing too many
behaviuor requirements on the client side (otherwise why bother
having an HTTP API?). But that's for later. Right now we're on a
tight schedule trying to make sure that claims get in in Ocata.

Because there's a bit of a dependency hierarchy with the various
threads of work going on in placement, the work on claims may punt
traits and/or nested resource providers further down the timeline.
Work continues on all three concurrently.

Another change is that allocations now include project id and user
id information and usages by those id can be retrieved.

Help Wanted

Areas where volunteers are needed.

Main Themes

Claims in the Scheduler

As described above there's been a change in direction. That probably
means some or all of the code now at

 https://review.openstack.org/#/q/status:open+topic:bp/placement-claims

can be abandoned in favor of the work at

 

https://review.openstack.org/#/q/topic:bp/placement-allocation-requests+status:open

The main starting point for that is

 https://review.openstack.org/#/c/476631/

Traits

The concept of traits now exists in the placement service, but
filtering resource providers on traits is in flux. With the advent
of /allocation_candidates as the primary scheduling interface, that
needs to support traits. Work for that is in a stack starting at

 https://review.openstack.org/#/c/478464/

It's not yet clear if we'll want to support traits at both
/allocationcandidates and /resourceproviders. I think we should,
but the immediate need is on /allocation_candidates.

There's some proposed code to get the latter started:

 https://review.openstack.org/#/c/474602/

Shared Resource Providers

Support for shared resource providers is "built in" to the
/allocation_candidates concept and one of the drivers for having it.

Nested Resource Providers

Work continues on nested resource providers.

   

https://review.openstack.org/#/q/status:open+topic:bp/nested-resource-providers

The need with these is simply more review, but they are behind
claims in priority.

Docs

Lots of placement-related api docs have merged or are in progress:

 

https://review.openstack.org/#/q/status:open+topic:cd/placement-api-ref

Shortly there will be a real publishing job:

 https://review.openstack.org/#/c/480991/

and the tooling which tests that new handlers are documented
will be turned on:

 https://review.openstack.org/#/c/480924/

Some changes have been proposed to document the scheduler's
workflow, including visual aids, starting at:

 https://review.openstack.org/#/c/475810/

Other Code/Specs

End

That's all I've got this week, next week I should be a bit more
caught up and aware of any bits I've missed. No prize this week, but
maybe next week.


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Another thing to mention that wasn't in this list:

https://blueprints.launchpad.net/nova/+spec/custom-resource-classes-in-flavors

Ed got the part for the scheduler done, but there are two other changes
needed for that blueprint. I also have a change up to amend the spec
since it was missing some details:

https://review.openstack.org/#/c/481748/

--

Thanks,

Matt


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 7, 2017 by mriedemos_at_gmail.c (15,720 points)   2 4 10
0 votes

2017-07-07 19:44 GMT+08:00 Chris Dent cdent+os@anticdent.org:

After 40 days in the desert I've returned with placement update 27.

Unfortunately, as far as I can tell, no one did any updates while I
was gone so I don't have anything to crib from to have the full
story on what's going on. I suspect I will miss some relevant
reviews when making this list. If I have, please let me know.
Otherwise, let's begin:

What Matters Most

Claims in the scheduler remains the key feature we'd like to get in
before feature freeze. After some hiccups on how to do it, making
requests of the new /allocation_candidates (of which more, below) is
the way to go. Changes for that are starting at

https://review.openstack.org/#/c/476631/

What's Changed

As mentioned, there's now a new URL in the placement API:
GET /allocationcandidates. It has a similar interface to GET
/resource
providers (in that you can filter the results by the kind
of resources required) but the information is formatted as a
two-tuple of lists of allocation requests and a dictionary of
resource provider information. The latter will provide the initial
list of available resource providers and augment the process of
filtering and weighing those providers. The former provides a
collection of correctly formed JSON bodies that can be sent in a PUT
to /allocations/{consumer_uuid} when making a claim.

I'm still a bit confused about where the concept of "alternatives"
that are going to be passed to the cell conductors fits into this,
but I guess that will become more clear soon.

It also seems like this model creates a pretty strong conceptual
coupling between a thing which behaves like a nova-scheduler
(request, process, then claim resources). As placement becomes
useful to other services it will be important to revisit some of
these decisions and make sure the HTTP API is not imposing too many
behaviuor requirements on the client side (otherwise why bother
having an HTTP API?). But that's for later. Right now we're on a
tight schedule trying to make sure that claims get in in Ocata.

Because there's a bit of a dependency hierarchy with the various
threads of work going on in placement, the work on claims may punt
traits and/or nested resource providers further down the timeline.
Work continues on all three concurrently.

Another change is that allocations now include project id and user
id information and usages by those id can be retrieved.

Help Wanted

Areas where volunteers are needed.

Main Themes

Claims in the Scheduler

As described above there's been a change in direction. That probably
means some or all of the code now at

https://review.openstack.org/#/q/status:open+topic:bp/placement-claims

can be abandoned in favor of the work at

https://review.openstack.org/#/q/topic:bp/placement-allocati

on-requests+status:open

The main starting point for that is

https://review.openstack.org/#/c/476631/

Traits

The concept of traits now exists in the placement service, but
filtering resource providers on traits is in flux. With the advent
of /allocation_candidates as the primary scheduling interface, that
needs to support traits. Work for that is in a stack starting at

https://review.openstack.org/#/c/478464/

It's not yet clear if we'll want to support traits at both
/allocationcandidates and /resourceproviders. I think we should,
but the immediate need is on /allocation_candidates.

For traits support in /allocation_candidates, I started some patches:
https://review.openstack.org/478464
https://review.openstack.org/479766
https://review.openstack.org/479776

There's some proposed code to get the latter started:

https://review.openstack.org/#/c/474602/

Shared Resource Providers

Support for shared resource providers is "built in" to the
/allocation_candidates concept and one of the drivers for having it.

Nested Resource Providers

Work continues on nested resource providers.

  https://review.openstack.org/#/q/status:open+topic:bp/nested

-resource-providers

The need with these is simply more review, but they are behind
claims in priority.

Docs

Lots of placement-related api docs have merged or are in progress:

https://review.openstack.org/#/q/status:open+topic:cd/placem

ent-api-ref

Shortly there will be a real publishing job:

https://review.openstack.org/#/c/480991/

and the tooling which tests that new handlers are documented
will be turned on:

https://review.openstack.org/#/c/480924/

Some changes have been proposed to document the scheduler's
workflow, including visual aids, starting at:

https://review.openstack.org/#/c/475810/

Other Code/Specs

End

That's all I've got this week, next week I should be a bit more
caught up and aware of any bits I've missed. No prize this week, but
maybe next week.

--
Chris Dent ┬──┬◡ノ(° -°ノ) https://anticdent.org/
freenode: cdent tw: @anticdent


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 8, 2017 by Alex_Xu (8,960 points)   1 3 3
...