settingsLogin | Registersettings

[openstack-dev] [nova] schedule instance based on CPU frequency ?

0 votes

CPU frequency is an import performance parameter, currently nova drivers
just report cpuinfo without frequency. we stored the compute node
cpu
info in database with colum computenodes.cpuinfo, we can add the
frequency easily.

The usage of cpu frequency I can think is used to schedule to meet
applications which need high frequency. add a frequency based filter ?
if we need this , I would like to propose a spec for this .

There are two steps to leverage cpu frequency:
1. report cpu frequency and record the value, nova hypervisor-show will
include the value .

  1. filter compute nodes based cpu frequency.
    add a new scheduler filter to do that

before I start to do these stuff. I would like to your input .

Do we need leverage CPU frequency in Nova ?
if yes, do we need a new filter or leverage existing filter to use
frequency ?

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Jun 30, 2015 in openstack-dev by ChangBo_Guo (4,540 points)   3 5

15 Responses

0 votes

On 06/30/2015 07:42 AM, ChangBo Guo wrote:
CPU frequency is an import performance parameter, currently nova
drivers just report cpuinfo without frequency. we stored the compute
node cpu
info in database with colum computenodes.cpuinfo, we can add
the frequency easily.

The usage of cpu frequency I can think is used to schedule to meet
applications which need high frequency. add a frequency based filter ?
if we need this , I would like to propose a spec for this .

Would it be possible to give more details on the type of app that will
have this specific requirement.

I don't think I have all the details in my head, but it seems to me that
the frequency of the hypervisor CPU is just not something that carries
enough information for users about how most applications will perform. I
would imagine they would either want "the fastest" or some specialized
HW for specific applications.

There are two steps to leverage cpu frequency:
1. report cpu frequency and record the value, nova hypervisor-show
will include the value .

  1. filter compute nodes based cpu frequency.
    add a new scheduler filter to do that

before I start to do these stuff. I would like to your input .

Do we need leverage CPU frequency in Nova ?
if yes, do we need a new filter or leverage existing filter to use
frequency ?

I don't think we do personally - but I may not understand what problem
this is trying to solve.

But even if we do - the most important thing IMHO would be how to
expose it to users (do we allow them to request a minimum frequency, or
a specific one or something else). API contract is extremely important
here because we want to make sure that we are exposing the right
semantics for users - as we would want this to be usable to as big a
group of people as possible.

If it's just about having a high performance tier - can we do this with
host aggregates and flavors? These are the questions we want to answer
first IMHO.

N.

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jun 30, 2015 by Nikola_Đipanov (5,400 points)   2 3 4
0 votes

On Tue, Jun 30, 2015 at 02:42:01PM +0800, ChangBo Guo wrote:
CPU frequency is an import performance parameter, currently nova drivers
just report cpuinfo without frequency. we stored the compute node
cpu
info in database with colum computenodes.cpuinfo, we can add the
frequency easily.

Is CPU frequency really an accurate metric for determining relative
performance of different hardware ? It seems maximum CPU frequency of
CPUs has rather plateaued, and chip vendors have been following
different avenues to improve performance of their chips such as multicore,
multhreads, and various other architectural changes. So I'm not sure that
just having a filter that compares CPU frequency is neccessarily going to
give useful results. ie faster frequency no longer neccessarily implies
faster performance.

Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jun 30, 2015 by Daniel_P._Berrange (27,920 points)   2 4 9
0 votes

2015-06-30 16:38 GMT+08:00 Nikola Đipanov ndipanov@redhat.com:

On 06/30/2015 07:42 AM, ChangBo Guo wrote:

CPU frequency is an import performance parameter, currently nova
drivers just report cpuinfo without frequency. we stored the compute
node cpu
info in database with colum computenodes.cpuinfo, we can add
the frequency easily.

The usage of cpu frequency I can think is used to schedule to meet
applications which need high frequency. add a frequency based filter ?
if we need this , I would like to propose a spec for this .

Would it be possible to give more details on the type of app that will
have this specific requirement.

I don't think I have all the details in my head, but it seems to me that
the frequency of the hypervisor CPU is just not something that carries
enough information for users about how most applications will perform. I
would imagine they would either want "the fastest" or some specialized
HW for specific applications.

There are two steps to leverage cpu frequency:
1. report cpu frequency and record the value, nova hypervisor-show
will include the value .

  1. filter compute nodes based cpu frequency.
    add a new scheduler filter to do that

before I start to do these stuff. I would like to your input .

Do we need leverage CPU frequency in Nova ?
if yes, do we need a new filter or leverage existing filter to use
frequency ?

I don't think we do personally - but I may not understand what problem
this is trying to solve.

But even if we do - the most important thing IMHO would be how to
expose it to users (do we allow them to request a minimum frequency, or
a specific one or something else). API contract is extremely important
here because we want to make sure that we are exposing the right
semantics for users - as we would want this to be usable to as big a
group of people as possible.

The use case: user want to integrate hardware management with api
/os-hypervisors,
they want to know more details about hardware, so cpu frequency is good
to have.
so I think if we can provide it to user, we don't change database schema,
just change colume
computenodes.cpuinfo, is it that ok ?

If it's just about having a high performance tier - can we do this with
host aggregates and flavors? These are the questions we want to answer
first IMHO.

yes we can host aggregates to solve the schedule.

N.

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jun 30, 2015 by ChangBo_Guo (4,540 points)   3 5
0 votes

On 06/30/2015 02:42 AM, ChangBo Guo wrote:
CPU frequency is an import performance parameter, currently nova
drivers just report cpuinfo without frequency. we stored the compute
node cpu
info in database with colum computenodes.cpuinfo, we can add
the frequency easily.

The usage of cpu frequency I can think is used to schedule to meet
applications which need high frequency. add a frequency based filter ?
if we need this , I would like to propose a spec for this .

There are two steps to leverage cpu frequency:
1. report cpu frequency and record the value, nova hypervisor-show
will include the value .

  1. filter compute nodes based cpu frequency.
    add a new scheduler filter to do that

before I start to do these stuff. I would like to your input .

Do we need leverage CPU frequency in Nova ?
if yes, do we need a new filter or leverage existing filter to use
frequency ?

Like Dan B, I question whether CPU frequency really is a useful metric
for scheduling decisions.

That said, it is already possible to use CPU frequency in the
MetricsWeigher scheduler weigher. The compute monitor plugin system is
currently being overhauled [1], but the functionality to monitor
CPU-related metrics already exists in Nova and can be enabled by doing
the following in your nova-compute nova.conf:

compute_monitors = ComputeDriverCPUMonitor

Note that with the refactoring of the monitoring plugin interface, the
above option will change due to using stevedore to load monitor extensions:

computemonitors = nova.compute.monitors.cpu.virtdriver:Monitor

In your Nova scheduler nova.conf, you will need to add the following in
the [metrics] section of the file:

weights_setting = cpu.frequency=10.0

Again, I'm not saying that the above will result in any appreciable
enhancement to the scheduler's decision-making, but it will do what
you're trying to accomplish :)

Best,
-jay

[1]
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bug/1468012,n,z


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jun 30, 2015 by Jay_Pipes (59,760 points)   3 10 14
0 votes

thanks Dan and Jay, we don't need add new scheduler for that :-),
what about provide cpu frequency to api /os-hypervisors, that means we
can
report this value automatically, the value can be used in high level mange
tools.

2015-07-01 2:58 GMT+08:00 Jay Pipes jaypipes@gmail.com:

On 06/30/2015 02:42 AM, ChangBo Guo wrote:

CPU frequency is an import performance parameter, currently nova
drivers just report cpuinfo without frequency. we stored the compute
node cpu
info in database with colum computenodes.cpuinfo, we can add
the frequency easily.

The usage of cpu frequency I can think is used to schedule to meet
applications which need high frequency. add a frequency based filter ?
if we need this , I would like to propose a spec for this .

There are two steps to leverage cpu frequency:
1. report cpu frequency and record the value, nova hypervisor-show
will include the value .

  1. filter compute nodes based cpu frequency.
    add a new scheduler filter to do that

before I start to do these stuff. I would like to your input .

Do we need leverage CPU frequency in Nova ?
if yes, do we need a new filter or leverage existing filter to use
frequency ?

Like Dan B, I question whether CPU frequency really is a useful metric for
scheduling decisions.

That said, it is already possible to use CPU frequency in the
MetricsWeigher scheduler weigher. The compute monitor plugin system is
currently being overhauled [1], but the functionality to monitor
CPU-related metrics already exists in Nova and can be enabled by doing the
following in your nova-compute nova.conf:

compute_monitors = ComputeDriverCPUMonitor

Note that with the refactoring of the monitoring plugin interface, the
above option will change due to using stevedore to load monitor extensions:

computemonitors = nova.compute.monitors.cpu.virtdriver:Monitor

In your Nova scheduler nova.conf, you will need to add the following in
the [metrics] section of the file:

weights_setting = cpu.frequency=10.0

Again, I'm not saying that the above will result in any appreciable
enhancement to the scheduler's decision-making, but it will do what you're
trying to accomplish :)

Best,
-jay

[1]
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bug/1468012,n,z


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 1, 2015 by ChangBo_Guo (4,540 points)   3 5
0 votes

On 07/01/2015 12:23 AM, ChangBo Guo wrote:
thanks Dan and Jay, we don't need add new scheduler for that :-),
what about provide cpu frequency to api /os-hypervisors, that means
we can
report this value automatically, the value can be used in high level
mange tools.

Meh, I'm not too big of a fan of the os-hypervisors extension. Actually,
one might say I despise that extension :)

That said, I suppose it should be possible to include the output of the
CPU frequency in the cpu_info field there...

-jay

2015-07-01 2:58 GMT+08:00 Jay Pipes <jaypipes@gmail.com
jaypipes@gmail.com>:

On 06/30/2015 02:42 AM, ChangBo Guo wrote:

    CPU frequency  is an import performance parameter,  currently  nova
    drivers just  report cpu_info without frequency.   we stored the
    compute
    node cpu_info in database with colum compute_nodes.cpu_info,  we
    can add
    the frequency  easily.

    The usage of  cpu frequency  I  can think is used to schedule to
    meet
    applications which need high frequency.  add a frequency based
    filter ?
    if we need this , I would like to propose  a spec for this .


    There are two steps to leverage cpu frequency:
    1.  report cpu frequency  and record the value,  nova
    hypervisor-show
    will include the value .

    2.  filter compute nodes based  cpu frequency.
          add a new scheduler filter to do that

    before I start to do these stuff.  I would like to your  input .

    Do we need leverage CPU frequency  in Nova ?
    if yes, do we need a new filter  or  leverage existing filter to use
    frequency ?


Like Dan B, I question whether CPU frequency really is a useful
metric for scheduling decisions.

That said, it is already possible to use CPU frequency in the
MetricsWeigher scheduler weigher. The compute monitor plugin system
is currently being overhauled [1], but the functionality to monitor
CPU-related metrics already exists in Nova and can be enabled by
doing the following in your nova-compute nova.conf:

compute_monitors = ComputeDriverCPUMonitor

Note that with the refactoring of the monitoring plugin interface,
the above option will change due to using stevedore to load monitor
extensions:

compute_monitors = nova.compute.monitors.cpu.virt_driver:Monitor

In your Nova scheduler nova.conf, you will need to add the following
in the [metrics] section of the file:

weights_setting = cpu.frequency=10.0

Again, I'm not saying that the above will result in any appreciable
enhancement to the scheduler's decision-making, but it will do what
you're trying to accomplish :)

Best,
-jay

[1]
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bug/1468012,n,z


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 2, 2015 by Jay_Pipes (59,760 points)   3 10 14
0 votes

Le 02/07/2015 21:40, Jay Pipes a écrit :

On 07/01/2015 12:23 AM, ChangBo Guo wrote:

thanks Dan and Jay, we don't need add new scheduler for that :-),
what about provide cpu frequency to api /os-hypervisors, that means
we can
report this value automatically, the value can be used in high level
mange tools.

Meh, I'm not too big of a fan of the os-hypervisors extension.
Actually, one might say I despise that extension :)

That said, I suppose it should be possible to include the output of
the CPU frequency in the cpu_info field there...

Well, IMHO I don't like to have the Hypervisors API to be a Nagios-like
view of the hypervisors world and I don't really much benefits of pusing
the metrics up to the API.

On the other hand, those monitor metrics are already sent as
notifications on the bus [1] so a 3rd party tool can easily fetch them
without necessarly needing to extend the API.

HTH,
-Sylvain

[1]
http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/resource_tracker.py?id=49873d8f6dff651cd83ff10ad5491a04286783d9#n364

-jay

2015-07-01 2:58 GMT+08:00 Jay Pipes <jaypipes@gmail.com
jaypipes@gmail.com>:

On 06/30/2015 02:42 AM, ChangBo Guo wrote:

    CPU frequency  is an import performance parameter, currently  

nova
drivers just report cpuinfo without frequency. we stored the
compute
node cpu
info in database with colum computenodes.cpuinfo, we
can add
the frequency easily.

    The usage of  cpu frequency  I  can think is used to schedule to
    meet
    applications which need high frequency.  add a frequency based
    filter ?
    if we need this , I would like to propose  a spec for this .


    There are two steps to leverage cpu frequency:
    1.  report cpu frequency  and record the value,  nova
    hypervisor-show
    will include the value .

    2.  filter compute nodes based  cpu frequency.
          add a new scheduler filter to do that

    before I start to do these stuff.  I would like to your input .

    Do we need leverage CPU frequency  in Nova ?
    if yes, do we need a new filter  or  leverage existing filter 

to use
frequency ?

Like Dan B, I question whether CPU frequency really is a useful
metric for scheduling decisions.

That said, it is already possible to use CPU frequency in the
MetricsWeigher scheduler weigher. The compute monitor plugin system
is currently being overhauled [1], but the functionality to monitor
CPU-related metrics already exists in Nova and can be enabled by
doing the following in your nova-compute nova.conf:

compute_monitors = ComputeDriverCPUMonitor

Note that with the refactoring of the monitoring plugin interface,
the above option will change due to using stevedore to load monitor
extensions:

compute_monitors = nova.compute.monitors.cpu.virt_driver:Monitor

In your Nova scheduler nova.conf, you will need to add the following
in the [metrics] section of the file:

weights_setting = cpu.frequency=10.0

Again, I'm not saying that the above will result in any appreciable
enhancement to the scheduler's decision-making, but it will do what
you're trying to accomplish :)

Best,
-jay

[1]

https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bug/1468012,n,z


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 3, 2015 by Sylvain_Bauza (14,100 points)   1 3 5
0 votes

On 07/03/2015 06:32 AM, Sylvain Bauza wrote:
Le 02/07/2015 21:40, Jay Pipes a écrit :

On 07/01/2015 12:23 AM, ChangBo Guo wrote:

thanks Dan and Jay, we don't need add new scheduler for that :-),
what about provide cpu frequency to api /os-hypervisors, that means
we can
report this value automatically, the value can be used in high level
mange tools.

Meh, I'm not too big of a fan of the os-hypervisors extension.
Actually, one might say I despise that extension :)

That said, I suppose it should be possible to include the output of
the CPU frequency in the cpu_info field there...

Well, IMHO I don't like to have the Hypervisors API to be a Nagios-like
view of the hypervisors world and I don't really much benefits of pusing
the metrics up to the API.

On the other hand, those monitor metrics are already sent as
notifications on the bus [1] so a 3rd party tool can easily fetch them
without necessarly needing to extend the API.

Yeah, the difference here is that CPU frequency really isn't a metric...
it's a static thing that doesn't change over time. Which is why I think
it's OK to put it in cpu_info.

Best,
-jay


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 3, 2015 by Jay_Pipes (59,760 points)   3 10 14
0 votes

Le 03/07/2015 15:25, Jay Pipes a écrit :

On 07/03/2015 06:32 AM, Sylvain Bauza wrote:

Le 02/07/2015 21:40, Jay Pipes a écrit :

On 07/01/2015 12:23 AM, ChangBo Guo wrote:

thanks Dan and Jay, we don't need add new scheduler for that :-),
what about provide cpu frequency to api /os-hypervisors, that means
we can
report this value automatically, the value can be used in high level
mange tools.

Meh, I'm not too big of a fan of the os-hypervisors extension.
Actually, one might say I despise that extension :)

That said, I suppose it should be possible to include the output of
the CPU frequency in the cpu_info field there...

Well, IMHO I don't like to have the Hypervisors API to be a Nagios-like
view of the hypervisors world and I don't really much benefits of pusing
the metrics up to the API.

On the other hand, those monitor metrics are already sent as
notifications on the bus [1] so a 3rd party tool can easily fetch them
without necessarly needing to extend the API.

Yeah, the difference here is that CPU frequency really isn't a
metric... it's a static thing that doesn't change over time. Which is
why I think it's OK to put it in cpu_info.

Oh, you're right, it's a very static thing. If it's not a metric, then I
would like to see it removed from the CPU monitor and provided in
cpu_info then.

-Sylvain

Best,
-jay


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 3, 2015 by Sylvain_Bauza (14,100 points)   1 3 5
0 votes

I propose a review to add it in cpu_info :
https://review.openstack.org/#/c/197829/

2015-07-03 21:37 GMT+08:00 Sylvain Bauza sbauza@redhat.com:

Le 03/07/2015 15:25, Jay Pipes a écrit :

On 07/03/2015 06:32 AM, Sylvain Bauza wrote:

Le 02/07/2015 21:40, Jay Pipes a écrit :

On 07/01/2015 12:23 AM, ChangBo Guo wrote:

thanks Dan and Jay, we don't need add new scheduler for that :-),
what about provide cpu frequency to api /os-hypervisors, that means
we can
report this value automatically, the value can be used in high level
mange tools.

Meh, I'm not too big of a fan of the os-hypervisors extension.
Actually, one might say I despise that extension :)

That said, I suppose it should be possible to include the output of
the CPU frequency in the cpu_info field there...

Well, IMHO I don't like to have the Hypervisors API to be a Nagios-like
view of the hypervisors world and I don't really much benefits of pusing
the metrics up to the API.

On the other hand, those monitor metrics are already sent as
notifications on the bus [1] so a 3rd party tool can easily fetch them
without necessarly needing to extend the API.

Yeah, the difference here is that CPU frequency really isn't a metric...
it's a static thing that doesn't change over time. Which is why I think
it's OK to put it in cpu_info.

Oh, you're right, it's a very static thing. If it's not a metric, then I
would like to see it removed from the CPU monitor and provided in cpu_info
then.

-Sylvain

Best,

-jay


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
ChangBo Guo(gcb)


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Jul 6, 2015 by ChangBo_Guo (4,540 points)   3 5
...