settingsLogin | Registersettings

[openstack-dev] [nova] [neutron] Adding neutron VIF NUMA locality support

0 votes

Hey,

NUMA locality matters as much for NICs used e.g for Open vSwitch as for SR-IOV
devices. At the moment, nova support NUMA affinity for PCI passthrough devices
and SR-IOV devices, but it makes no attempt to do the same for other NICs. In
the name of NFV enablement, we should probably close this gap.

I have some ideas around how this could work, but they're fuzzy enough and
involve exchanging os-vif objects between nova and neutron. This is probably
the most difficult path as we've been trying to get os-vif objects over the
nova-neutron wire for a while now, to no success.

Anyone else keen on such a feature? Given that there are a significant amount
of people from nova, neutron, and general NFV backgrounds at the PTG next
week, we have a very good opportunity to talk about this (either in the nova-
neutron sync, if that's not already full, or in some hallway somewhere).

At this point in the day, this is probably very much a Rocky feature, but we
could definitely put in whatever groundwork is necessary this cycle to make the
work in Rocky as easy possible.

Cheers,
Stephen


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Sep 7, 2017 in openstack-dev by Stephen_Finucane (1,620 points)   2

2 Responses

0 votes

-----Original Message-----
From: Stephen Finucane [mailto:sfinucan@redhat.com]
Sent: Thursday, September 7, 2017 5:42 PM
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Cc: Jakub Libosvar jlibosva@redhat.com; Karthik Sundaravel
ksundara@redhat.com; Mooney, Sean K sean.k.mooney@intel.com
Subject: [nova] [neutron] Adding neutron VIF NUMA locality support

Hey,

NUMA locality matters as much for NICs used e.g for Open vSwitch as for
SR-IOV devices. At the moment, nova support NUMA affinity for PCI
passthrough devices and SR-IOV devices, but it makes no attempt to do
the same for other NICs. In the name of NFV enablement, we should
probably close this gap.
[Mooney, Sean K] I like this idea in general, that said in ovs-dpdk we modified
ovs to schedule the vhost-user port to be processed on a pmd that is on the same
Numa node as the vm and reallocate the vhsot user port memory where possible
To also have the same affinity.

I have some ideas around how this could work, but they're fuzzy enough
and involve exchanging os-vif objects between nova and neutron. This is
probably the most difficult path as we've been trying to get os-vif
objects over the nova-neutron wire for a while now, to no success.
[Mooney, Sean K] actually we have so poc code you should proably review
This topic.
https://blueprints.launchpad.net/os-vif/+spec/vif-port-profile
https://review.openstack.org/#/c/490829/
https://review.openstack.org/#/c/490819/
https://review.openstack.org/#/c/441590/
the first patch of the neutron side poc should be up before the ptg.

Anyone else keen on such a feature? Given that there are a significant
amount of people from nova, neutron, and general NFV backgrounds at the
PTG next week, we have a very good opportunity to talk about this
(either in the nova- neutron sync, if that's not already full, or in
some hallway somewhere).
[Mooney, Sean K] in terms of basic numa affinity this is not as important
With ovs-dpdk because we make best effort to fix it in ovs this is less pressing
Then it used to be. It is still important for other backbends but we need
Also have a mechanism to control numa affinity policy like
https://review.openstack.org/#/c/361140/ to not break existing deployments.

I have some taught about modeling network backbends
in placement and also passing traits requests for neutron that this would dove
tail with so would love to talk to anyone who is interested in this.
By modeling ovs and other network backend in placement and combining that
With traits and the nova-neutron negotiation protocol we support several
Advance usescase.

By the way ovs-dpdk allow you to specify vhost-port rx/tx queue mapping
to pmd which could give a nice performance boost if done correctly. It
might be worth extending os-vif to do that in the future though this could
equally be handeled by the neutron ovs l2 agent.

At this point in the day, this is probably very much a Rocky feature,
but we could definitely put in whatever groundwork is necessary this
cycle to make the work in Rocky as easy possible.
[Mooney, Sean K] I'm hoping we can get the nova neutron negotiation done in queens.

Cheers,
Stephen


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 7, 2017 by Mooney,_Sean_K (3,580 points)   3 8
0 votes

Stephen-

FYI, we're teeing up (anti-)affinity and neutron-nova interaction

topics for the "generic device management" discussions at the PTG.
Please jump on the etherpad [1] and expand/expound in the relevant
spots, which, as of this writing, you can find by searching for
"aggregate", "vf selection policy", and "interplay".

    Thanks,
    Eric

[1]
https://etherpad.openstack.org/p/nova-ptg-queens-generic-device-management

On 09/07/2017 01:07 PM, Mooney, Sean K wrote:

-----Original Message-----
From: Stephen Finucane [mailto:sfinucan@redhat.com]
Sent: Thursday, September 7, 2017 5:42 PM
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Cc: Jakub Libosvar jlibosva@redhat.com; Karthik Sundaravel
ksundara@redhat.com; Mooney, Sean K sean.k.mooney@intel.com
Subject: [nova] [neutron] Adding neutron VIF NUMA locality support

Hey,

NUMA locality matters as much for NICs used e.g for Open vSwitch as for
SR-IOV devices. At the moment, nova support NUMA affinity for PCI
passthrough devices and SR-IOV devices, but it makes no attempt to do
the same for other NICs. In the name of NFV enablement, we should
probably close this gap.
[Mooney, Sean K] I like this idea in general, that said in ovs-dpdk we modified
ovs to schedule the vhost-user port to be processed on a pmd that is on the same
Numa node as the vm and reallocate the vhsot user port memory where possible
To also have the same affinity.

I have some ideas around how this could work, but they're fuzzy enough
and involve exchanging os-vif objects between nova and neutron. This is
probably the most difficult path as we've been trying to get os-vif
objects over the nova-neutron wire for a while now, to no success.
[Mooney, Sean K] actually we have so poc code you should proably review
This topic.
https://blueprints.launchpad.net/os-vif/+spec/vif-port-profile
https://review.openstack.org/#/c/490829/
https://review.openstack.org/#/c/490819/
https://review.openstack.org/#/c/441590/
the first patch of the neutron side poc should be up before the ptg.

Anyone else keen on such a feature? Given that there are a significant
amount of people from nova, neutron, and general NFV backgrounds at the
PTG next week, we have a very good opportunity to talk about this
(either in the nova- neutron sync, if that's not already full, or in
some hallway somewhere).
[Mooney, Sean K] in terms of basic numa affinity this is not as important
With ovs-dpdk because we make best effort to fix it in ovs this is less pressing
Then it used to be. It is still important for other backbends but we need
Also have a mechanism to control numa affinity policy like
https://review.openstack.org/#/c/361140/ to not break existing deployments.

I have some taught about modeling network backbends
in placement and also passing traits requests for neutron that this would dove
tail with so would love to talk to anyone who is interested in this.
By modeling ovs and other network backend in placement and combining that
With traits and the nova-neutron negotiation protocol we support several
Advance usescase.

By the way ovs-dpdk allow you to specify vhost-port rx/tx queue mapping
to pmd which could give a nice performance boost if done correctly. It
might be worth extending os-vif to do that in the future though this could
equally be handeled by the neutron ovs l2 agent.

At this point in the day, this is probably very much a Rocky feature,
but we could definitely put in whatever groundwork is necessary this
cycle to make the work in Rocky as easy possible.
[Mooney, Sean K] I'm hoping we can get the nova neutron negotiation done in queens.

Cheers,
Stephen


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Sep 7, 2017 by Eric_Fried (1,900 points)   2
...