settingsLogin | Registersettings

[Openstack] libvirtd remote access security for Live migration

0 votes

Hi list:

I want to setup security for libvirtd.
I have read these articles:
https://wiki.openstack.org/wiki/OSSN/OSSN-0007
https://www.ibm.com/support/knowledgecenter/en/linuxonibm/liabp/liabpkvmsecsrmsasl.htm

My current conf is /etc/libvirt/libvirtd.conf:
listentls = 0
listen
tcp = 1
auth_tcp = "sasl"

And now I try :
nova live-migration --block-migrate 4309758f-2e66-4c66-bcbb-5f5ee33956ad ip-10-8-8-232.clouds.com

I found error on the src hv :
Live Migration failure: operation failed: Failed to connect to remote libvirt URI qemu+tcp://ip-10-8-8-232.clouds.com/system: authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found)

I try it with virsh ,it worked :
root@ip-10-8-8-243:/var/log/nova# virsh -c qemu+tcp://ip-10-8-8-232.clouds.com/system
Please enter your authentication name: admin
Please enter your password:
Welcome to virsh, the virtualization interactive terminal.

Type: 'help' for help with commands
'quit' to quit

virsh #
virsh #

My question is that :
How to make the SASL auth successfully when live-migration.

Thank you !

sosogh


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
asked Apr 10, 2017 in openstack by sosogh (180 points)   1 1

2 Responses

0 votes

On Fri, Apr 07, 2017 at 09:27:16AM +0800, sosogh wrote:
Hi list:

I want to setup security for libvirtd.
I have read these articles:
https://wiki.openstack.org/wiki/OSSN/OSSN-0007
https://www.ibm.com/support/knowledgecenter/en/linuxonibm/liabp/liabpkvmsecsrmsasl.htm

My current conf is /etc/libvirt/libvirtd.conf:
listentls = 0
listen
tcp = 1
auth_tcp = "sasl"

And now I try :
nova live-migration --block-migrate 4309758f-2e66-4c66-bcbb-5f5ee33956ad ip-10-8-8-232.clouds.com

I found error on the src hv :
Live Migration failure: operation failed: Failed to connect to remote libvirt URI
qemu+tcp://ip-10-8-8-232.clouds.com/system: authentication failed: Failed to start
SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found)

I try it with virsh ,it worked :
root@ip-10-8-8-243:/var/log/nova# virsh -c qemu+tcp://ip-10-8-8-232.clouds.com/system
Please enter your authentication name: admin
Please enter your password:
Welcome to virsh, the virtualization interactive terminal.

Type: 'help' for help with commands
'quit' to quit

virsh #
virsh #

Notice that when you use virsh it prompts for a username + password. There
is no way for nova to prompt for the username + password when doing live
migration. So the error you're getting is probably a result of Nova not
having provided any way for libvirt to query authentication credentials.
You could create a config file containing the credentials at

$HOME/.config/libvirt/auth.conf for the

nova user account. For this to be practical you would have to create the
same username+password on every single hypervisor host, as it would be
impractical to populate it with differnt creds for every possible host
in the cloud.

Second, please note that while historically the libvirt SASL configuration has
defaulted to the DIGEST-MD5 mechanism, this mechanism is no longer considered
to be secure by modern standards as it is vulnerable to MITM attacks and
brute force of the MD5 hash.

As such, from 3.2.0, libvirt has dropped DIGEST-MD5 by default and now
strongly recommends against its use. The only remaining secure SASL
mechanism that can be used on its own is Kerberos V5. That would not
suffer the problem with auth credentials, since the Kerberos ticket is
acquired out-of-band with no interactive prompts required.

That said, I'm guessing it is unlikely you have kerberos deployed and
probably wouldn't want to start deploying it either.

In the context of openstack my strong recommendation is to use TLS with
x509 certificates for libvirt remote access, since this aligns nicely
with the need to use TLS for almost every other service in OpenStack.

A future suggestion for interested Nova developers....

Once using TLS w/x509 you could also layer in SASL to get an extra
layer of protection. The ideal approach would be for libvirt to have
TLS w/x509 certs setup and a SASL SCRAM mechanism enabled over TLS
during initial deployment.

Then at the time Nova initiates a migration, the Nova scheduler would
create a one-time-password on the target host and give that to the
source host. This ensures that 2 libvirtd daemons cannot talk to each
other during normal operation - Nova scheduler would have to authorize
the connection between 2 nodes by setting up the auth credentials

Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|

responded Apr 7, 2017 by Daniel_P._Berrange (27,920 points)   2 4 9
0 votes

Hi Daniel

Thank you for your detailed replies.
You are exactly the frist one to point out this security issuse 3 years ago.

$HOME/.config/libvirt/auth.conf

I tried it , but still got the same error.
I also find another disadvantage of SASL :
The username+password will be shown using this cmd: strings /etc/libvirt/passwd.db
As your sugguestion , I think TLS is the best approach so far .

sosogh

From: Daniel P. Berrange
Date: 2017-04-07 17:48
To: sosogh
CC: openstack
Subject: Re: [Openstack] libvirtd remote access security for Live migration
On Fri, Apr 07, 2017 at 09:27:16AM +0800, sosogh wrote:
Hi list:

I want to setup security for libvirtd.
I have read these articles:
https://wiki.openstack.org/wiki/OSSN/OSSN-0007
https://www.ibm.com/support/knowledgecenter/en/linuxonibm/liabp/liabpkvmsecsrmsasl.htm

My current conf is /etc/libvirt/libvirtd.conf:
listentls = 0
listen
tcp = 1
auth_tcp = "sasl"

And now I try :
nova live-migration --block-migrate 4309758f-2e66-4c66-bcbb-5f5ee33956ad ip-10-8-8-232.clouds.com

I found error on the src hv :
Live Migration failure: operation failed: Failed to connect to remote libvirt URI
qemu+tcp://ip-10-8-8-232.clouds.com/system: authentication failed: Failed to start
SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found)

I try it with virsh ,it worked :
root@ip-10-8-8-243:/var/log/nova# virsh -c qemu+tcp://ip-10-8-8-232.clouds.com/system
Please enter your authentication name: admin
Please enter your password:
Welcome to virsh, the virtualization interactive terminal.

Type: 'help' for help with commands
'quit' to quit

virsh #
virsh #

Notice that when you use virsh it prompts for a username + password. There
is no way for nova to prompt for the username + password when doing live
migration. So the error you're getting is probably a result of Nova not
having provided any way for libvirt to query authentication credentials.
You could create a config file containing the credentials at

$HOME/.config/libvirt/auth.conf for the

nova user account. For this to be practical you would have to create the
same username+password on every single hypervisor host, as it would be
impractical to populate it with differnt creds for every possible host
in the cloud.

Second, please note that while historically the libvirt SASL configuration has
defaulted to the DIGEST-MD5 mechanism, this mechanism is no longer considered
to be secure by modern standards as it is vulnerable to MITM attacks and
brute force of the MD5 hash.

As such, from 3.2.0, libvirt has dropped DIGEST-MD5 by default and now
strongly recommends against its use. The only remaining secure SASL
mechanism that can be used on its own is Kerberos V5. That would not
suffer the problem with auth credentials, since the Kerberos ticket is
acquired out-of-band with no interactive prompts required.

That said, I'm guessing it is unlikely you have kerberos deployed and
probably wouldn't want to start deploying it either.

In the context of openstack my strong recommendation is to use TLS with
x509 certificates for libvirt remote access, since this aligns nicely
with the need to use TLS for almost every other service in OpenStack.

A future suggestion for interested Nova developers....

Once using TLS w/x509 you could also layer in SASL to get an extra
layer of protection. The ideal approach would be for libvirt to have
TLS w/x509 certs setup and a SASL SCRAM mechanism enabled over TLS
during initial deployment.

Then at the time Nova initiates a migration, the Nova scheduler would
create a one-time-password on the target host and give that to the
source host. This ensures that 2 libvirtd daemons cannot talk to each
other during normal operation - Nova scheduler would have to authorize
the connection between 2 nodes by setting up the auth credentials

Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|


Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
responded Apr 10, 2017 by sosogh (180 points)   1 1
...