settingsLogin | Registersettings

[Openstack-operators] Live-migration CPU doesn't have compatibility

0 votes

Hi list,

I'm facing issues on Liberty/CentOS7 doing live migrations between to
hosts. The hosts are Haswell and Broadwell. However, there is not
feature specific running on my VMs

Haswell -> Broadwell works
Broadwell -> Haswell fails with the error below.

I have on both hosts configured
[libvirt]
cpu_mode=none

and restarted openstack-nova-compute on hosts, however that didn't
help, with the same error. there gotta be a way of ignoring this
check? pls advice. thx will

2016-10-26 19:36:29.025 1438627 INFO nova.virt.libvirt.driver
[req-XXXX] Instance launched has CPU info: {"vendor": "Intel",
"model": "Broadwell", "arch": "x8664", "features": ["smap", "avx",
"clflush", "sep", "rtm", "vme", "dtes64", "invpcid", "tsc",
"fsgsbase", "xsave", "pge", "vmx", "erms", "xtpr", "cmov", "hle",
"smep", "ssse3", "est", "pat", "monitor", "smx", "pbe", "lm", "msr",
"adx", "3dnowprefetch", "nx", "fxsr", "syscall", "tm", "sse4.1",
"pae", "sse4.2", "pclmuldq", "acpi", "fma", "tsc-deadline", "mmx",
"osxsave", "cx8", "mce", "de", "tm2", "ht", "dca", "lahf
lm", "abm",
"rdseed", "popcnt", "mca", "pdpe1gb", "apic", "sse", "f16c", "pse",
"ds", "invtsc", "pni", "rdtscp", "avx2", "aes", "sse2", "ss",
"ds_cpl", "bmi1", "bmi2", "pcid", "fpu", "cx16", "pse36", "mtrr",
"movbe", "pdcm", "rdrand", "x2apic"], "topology": {"cores": 10,
"cells": 2, "threads": 2, "sockets": 1}}
2016-10-26 19:36:29.028 1438627 ERROR nova.virt.libvirt.driver
[req-XXXX] CPU doesn't have compatibility.

0

Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult
2016-10-26 19:36:29.057 1438627 ERROR oslo_messaging.rpc.dispatcher
[req-XXXX] Exception during message handling: Unacceptable CPU info:
CPU doesn't have compatibility.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
asked Oct 26, 2016 in openstack-operators by William_Josefsson (1,780 points)   1 8 11

7 Responses

0 votes

The VMs have to be restarted so that the libvirt config is updated
with the new CPU model.

Good luck!

On Wed, Oct 26, 2016 at 8:07 AM, William Josefsson
william.josefson@gmail.com wrote:
Hi list,

I'm facing issues on Liberty/CentOS7 doing live migrations between to
hosts. The hosts are Haswell and Broadwell. However, there is not
feature specific running on my VMs

Haswell -> Broadwell works
Broadwell -> Haswell fails with the error below.

I have on both hosts configured
[libvirt]
cpu_mode=none

and restarted openstack-nova-compute on hosts, however that didn't
help, with the same error. there gotta be a way of ignoring this
check? pls advice. thx will

2016-10-26 19:36:29.025 1438627 INFO nova.virt.libvirt.driver
[req-XXXX] Instance launched has CPU info: {"vendor": "Intel",
"model": "Broadwell", "arch": "x8664", "features": ["smap", "avx",
"clflush", "sep", "rtm", "vme", "dtes64", "invpcid", "tsc",
"fsgsbase", "xsave", "pge", "vmx", "erms", "xtpr", "cmov", "hle",
"smep", "ssse3", "est", "pat", "monitor", "smx", "pbe", "lm", "msr",
"adx", "3dnowprefetch", "nx", "fxsr", "syscall", "tm", "sse4.1",
"pae", "sse4.2", "pclmuldq", "acpi", "fma", "tsc-deadline", "mmx",
"osxsave", "cx8", "mce", "de", "tm2", "ht", "dca", "lahf
lm", "abm",
"rdseed", "popcnt", "mca", "pdpe1gb", "apic", "sse", "f16c", "pse",
"ds", "invtsc", "pni", "rdtscp", "avx2", "aes", "sse2", "ss",
"ds_cpl", "bmi1", "bmi2", "pcid", "fpu", "cx16", "pse36", "mtrr",
"movbe", "pdcm", "rdrand", "x2apic"], "topology": {"cores": 10,
"cells": 2, "threads": 2, "sockets": 1}}
2016-10-26 19:36:29.028 1438627 ERROR nova.virt.libvirt.driver
[req-XXXX] CPU doesn't have compatibility.

0

Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult
2016-10-26 19:36:29.057 1438627 ERROR oslo_messaging.rpc.dispatcher
[req-XXXX] Exception during message handling: Unacceptable CPU info:
CPU doesn't have compatibility.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Mohammed Naser — vexxhost


D. 514-316-8872
D. 800-910-1726 ext. 200
E. mnaser@vexxhost.com
W. http://vexxhost.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 26, 2016 by Mohammed_Naser (3,860 points)   1 3
0 votes

Thank you Mohammed. So I just set cpu_mode=none in libvirt section on
both source and destination hosts, restart nova-compute, restart VMs
on source host, and finally do the live migration? pls let me know if
this is wrong. thx will

On Wed, Oct 26, 2016 at 11:45 PM, Mohammed Naser mnaser@vexxhost.com wrote:
The VMs have to be restarted so that the libvirt config is updated
with the new CPU model.

Good luck!

On Wed, Oct 26, 2016 at 8:07 AM, William Josefsson
william.josefson@gmail.com wrote:

Hi list,

I'm facing issues on Liberty/CentOS7 doing live migrations between to
hosts. The hosts are Haswell and Broadwell. However, there is not
feature specific running on my VMs

Haswell -> Broadwell works
Broadwell -> Haswell fails with the error below.

I have on both hosts configured
[libvirt]
cpu_mode=none

and restarted openstack-nova-compute on hosts, however that didn't
help, with the same error. there gotta be a way of ignoring this
check? pls advice. thx will

2016-10-26 19:36:29.025 1438627 INFO nova.virt.libvirt.driver
[req-XXXX] Instance launched has CPU info: {"vendor": "Intel",
"model": "Broadwell", "arch": "x8664", "features": ["smap", "avx",
"clflush", "sep", "rtm", "vme", "dtes64", "invpcid", "tsc",
"fsgsbase", "xsave", "pge", "vmx", "erms", "xtpr", "cmov", "hle",
"smep", "ssse3", "est", "pat", "monitor", "smx", "pbe", "lm", "msr",
"adx", "3dnowprefetch", "nx", "fxsr", "syscall", "tm", "sse4.1",
"pae", "sse4.2", "pclmuldq", "acpi", "fma", "tsc-deadline", "mmx",
"osxsave", "cx8", "mce", "de", "tm2", "ht", "dca", "lahf
lm", "abm",
"rdseed", "popcnt", "mca", "pdpe1gb", "apic", "sse", "f16c", "pse",
"ds", "invtsc", "pni", "rdtscp", "avx2", "aes", "sse2", "ss",
"ds_cpl", "bmi1", "bmi2", "pcid", "fpu", "cx16", "pse36", "mtrr",
"movbe", "pdcm", "rdrand", "x2apic"], "topology": {"cores": 10,
"cells": 2, "threads": 2, "sockets": 1}}
2016-10-26 19:36:29.028 1438627 ERROR nova.virt.libvirt.driver
[req-XXXX] CPU doesn't have compatibility.

0

Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult
2016-10-26 19:36:29.057 1438627 ERROR oslo_messaging.rpc.dispatcher
[req-XXXX] Exception during message handling: Unacceptable CPU info:
CPU doesn't have compatibility.


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Mohammed Naser — vexxhost


D. 514-316-8872
D. 800-910-1726 ext. 200
E. mnaser@vexxhost.com
W. http://vexxhost.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 26, 2016 by William_Josefsson (1,780 points)   1 8 11
0 votes

On 10/26/2016 06:07 AM, William Josefsson wrote:
Hi list,

I'm facing issues on Liberty/CentOS7 doing live migrations between to
hosts. The hosts are Haswell and Broadwell. However, there is not
feature specific running on my VMs

Haswell -> Broadwell works
Broadwell -> Haswell fails with the error below.

I have on both hosts configured
[libvirt]
cpu_mode=none

and restarted openstack-nova-compute on hosts, however that didn't
help, with the same error. there gotta be a way of ignoring this
check? pls advice. thx will

If you are using kvm/qemu and set cpu_mode=none, then it will use 'host-model',
and any instances started on Broadwell can't be live-migrated onto Haswell.

In your case you probably want to set both computes to have:

[libvirt]
cpumode = custom
cpu
model = Haswell

This will cause nova to start guests with the "Haswell" model on both nodes.

Chris


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 27, 2016 by Chris_Friesen (20,420 points)   3 16 24
0 votes

On Thu, Oct 27, 2016 at 5:20 PM, Chris Friesen
chris.friesen@windriver.com wrote:
In your case you probably want to set both computes to have:

[libvirt]
cpumode = custom
cpu
model = Haswell

Hi Chris, thanks! Yerps, I finally got it working. However, I set
cpumodel=kvm64 everywhere and it seems to work. It is listed here:
https://wiki.openstack.org/wiki/LibvirtXMLCPUModel hopefully 'kvm64'
has no performance impact what cpu
model is set to, or would 'kvm64'
as model negatively affect my VMs? thx will


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 27, 2016 by William_Josefsson (1,780 points)   1 8 11
0 votes

Depending on your workload, it will. If they depend on any custom CPU
extensions, they will miss out on them and performance will be
decreased. My personal suggestion is to read the docs for it and use
the "smallest common denominator" in terms of CPU usage.

On Thu, Oct 27, 2016 at 11:31 AM, William Josefsson
william.josefson@gmail.com wrote:
On Thu, Oct 27, 2016 at 5:20 PM, Chris Friesen
chris.friesen@windriver.com wrote:

In your case you probably want to set both computes to have:

[libvirt]
cpumode = custom
cpu
model = Haswell

Hi Chris, thanks! Yerps, I finally got it working. However, I set
cpumodel=kvm64 everywhere and it seems to work. It is listed here:
https://wiki.openstack.org/wiki/LibvirtXMLCPUModel hopefully 'kvm64'
has no performance impact what cpu
model is set to, or would 'kvm64'
as model negatively affect my VMs? thx will


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Mohammed Naser — vexxhost


D. 514-316-8872
D. 800-910-1726 ext. 200
E. mnaser@vexxhost.com
W. http://vexxhost.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 27, 2016 by Mohammed_Naser (3,860 points)   1 3
0 votes

hi, I did 'virsh capabilities' on the Haswell, which turned out to
list model: Haswell-noTSX. So I set in nova.conf
cpu_model=Haswell-noTSX on both Haswell and Broadwell hosts and it
seems to work. I believe this is my smallest common denominator. thx
will

On Fri, Oct 28, 2016 at 2:39 AM, Mohammed Naser mnaser@vexxhost.com wrote:
Depending on your workload, it will. If they depend on any custom CPU
extensions, they will miss out on them and performance will be
decreased. My personal suggestion is to read the docs for it and use
the "smallest common denominator" in terms of CPU usage.

On Thu, Oct 27, 2016 at 11:31 AM, William Josefsson
william.josefson@gmail.com wrote:

On Thu, Oct 27, 2016 at 5:20 PM, Chris Friesen
chris.friesen@windriver.com wrote:

In your case you probably want to set both computes to have:

[libvirt]
cpumode = custom
cpu
model = Haswell

Hi Chris, thanks! Yerps, I finally got it working. However, I set
cpumodel=kvm64 everywhere and it seems to work. It is listed here:
https://wiki.openstack.org/wiki/LibvirtXMLCPUModel hopefully 'kvm64'
has no performance impact what cpu
model is set to, or would 'kvm64'
as model negatively affect my VMs? thx will


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
Mohammed Naser — vexxhost


D. 514-316-8872
D. 800-910-1726 ext. 200
E. mnaser@vexxhost.com
W. http://vexxhost.com


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 28, 2016 by William_Josefsson (1,780 points)   1 8 11
0 votes

On 10/27/2016 11:09 PM, William Josefsson wrote:
hi, I did 'virsh capabilities' on the Haswell, which turned out to
list model: Haswell-noTSX. So I set in nova.conf
cpu_model=Haswell-noTSX on both Haswell and Broadwell hosts and it
seems to work. I believe this is my smallest common denominator.

Almost certainly, yes.

Chris


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Oct 31, 2016 by Chris_Friesen (20,420 points)   3 16 24
...