settingsLogin | Registersettings

[openstack-dev] [nova] Encrypted Ephemeral Storage

0 votes

I've been looking into using encrypted ephemeral storage with LVM. With the
[ephemeralstorageencryption] and [keymgr] sections to nova.conf, I get an
LVM volume with "-dmcrypt" is appended to the volume name, but otherwise
see no difference; I can still grep for text inside the volume.

Upon reading the source, I don't see "cryptsetup luksFormat" being called
anywhere (nova/libvirt/storage/*).

I was expecting a new encrypted LVM volume when a new instance was created.
Are my expectations misplaced? How is this feature envisioned to work?

Thanks,

-Chris


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Apr 11, 2016 in openstack-dev by chris.buccella_at_ve (520 points)   2 3

4 Responses

0 votes

Upon reading the source, I don't see "cryptsetup luksFormat" being called anywhere (nova/libvirt/storage/*).
Check out imagebackend.py:Lvm.create_imagehttps://github.com/openstack/nova/blob/master/nova/virt/libvirt/imagebackend.py#L690 and dmcrypt.py:create_volumehttps://github.com/openstack/nova/blob/master/nova/virt/libvirt/storage/dmcrypt.py#L48.

How is this feature envisioned to work?
The LVM volume with the '-dmcrypt' suffix is the unencrypted device that is passed to the VM. From a DevStack machine with an encrypted instance:

$ sudo cryptsetup status /dev/mapper/065859b2-50d6-46d6-927a-2dfd07db3306_disk-dmcrypt

/dev/mapper/065859b2-50d6-46d6-927a-2dfd07db3306_disk-dmcrypt is active and is in use.

type: PLAIN

cipher: aes-xts-plain64

keysize: 256 bits

device: /dev/mapper/stack--volumes--default-065859b2--50d6--46d6--927a--2dfd07db3306_disk

offset: 0 sectors

size: 2097152 sectors

mode: read/write

$ sudo fuser -vam /dev/mapper/065859b2-50d6-46d6-927a-2dfd07db3306_disk-dmcrypt

                 USER        PID ACCESS COMMAND

/dev/dm-1: libvirt-qemu 8429 F.... qemu-system-x86

While information in the '-dmcrypt' device is visible to a root user on the compute host, the underlying device (stack--volumes--default- in the example above) is encrypted, and everything written to the underlying disk is also encrypted. Try searching for the text in the underlying device – you shouldn't be able to find it.

Joel

From: Chris Buccella chris.buccella@verilume.com
Reply-To: "openstack-dev@lists.openstack.org" openstack-dev@lists.openstack.org
Date: Monday, April 11, 2016 at 1:06 PM
To: "openstack-dev@lists.openstack.org" openstack-dev@lists.openstack.org
Subject: [openstack-dev] [nova] Encrypted Ephemeral Storage

I've been looking into using encrypted ephemeral storage with LVM. With the [ephemeralstorageencryption] and [keymgr] sections to nova.conf, I get an LVM volume with "-dmcrypt" is appended to the volume name, but otherwise see no difference; I can still grep for text inside the volume.

Upon reading the source, I don't see "cryptsetup luksFormat" being called anywhere (nova/libvirt/storage/*).

I was expecting a new encrypted LVM volume when a new instance was created. Are my expectations misplaced? How is this feature envisioned to work?

Thanks,

-Chris


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 14, 2016 by Coffman,_Joel_M. (480 points)   1
0 votes

Thanks Joel. I was able to try this again and can confirm it works as you
describe. Cool.

Based on the comments to the RBD encryption change [1], it looks like there
will be a new direction for ephemeral disk encryption (embedding it in QEMU
directly). I assume LVM will work the same way when the time comes. Will
there be a migration path for the existing ephemeral disk encryption
support for LVM to the new model?

-Chris

[1] https://review.openstack.org/#/c/239798/

On Thu, Apr 14, 2016 at 12:56 PM, Coffman, Joel M. Joel.Coffman@jhuapl.edu
wrote:

Upon reading the source, I don't see "cryptsetup luksFormat" being
called anywhere (nova/libvirt/storage/*).
Check out imagebackend.py:Lvm.create_image
https://github.com/openstack/nova/blob/master/nova/virt/libvirt/imagebackend.py#L690
and dmcrypt.py:create_volume
https://github.com/openstack/nova/blob/master/nova/virt/libvirt/storage/dmcrypt.py#L48
.

How is this feature envisioned to work?
The LVM volume with the '-dmcrypt' suffix is the unencrypted device
that is passed to the VM. From a DevStack machine with an encrypted
instance:

$ sudo cryptsetup status
/dev/mapper/065859b2-50d6-46d6-927a-2dfd07db3306_disk-dmcrypt

/dev/mapper/065859b2-50d6-46d6-927a-2dfd07db3306_disk-dmcrypt is active
and is in use.

type: PLAIN

cipher: aes-xts-plain64

keysize: 256 bits

device:
/dev/mapper/stack--volumes--default-065859b2--50d6--46d6--927a--2dfd07db3306_disk

offset: 0 sectors

size: 2097152 sectors

mode: read/write

$ sudo fuser -vam
/dev/mapper/065859b2-50d6-46d6-927a-2dfd07db3306_disk-dmcrypt

                 USER        PID ACCESS COMMAND

/dev/dm-1: libvirt-qemu 8429 F.... qemu-system-x86
While information in the '-dmcrypt' device is visible to a root user on
the compute host, the underlying device (stack--volumes--default-
in the
example above) is encrypted, and everything written to the underlying disk
is also encrypted. Try searching for the text in the underlying device –
you shouldn't be able to find it.

Joel

From: Chris Buccella chris.buccella@verilume.com
Reply-To: "openstack-dev@lists.openstack.org" <
openstack-dev@lists.openstack.org>
Date: Monday, April 11, 2016 at 1:06 PM
To: "openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org
>
Subject: [openstack-dev] [nova] Encrypted Ephemeral Storage

I've been looking into using encrypted ephemeral storage with LVM. With
the [ephemeralstorageencryption] and [keymgr] sections to nova.conf, I
get an LVM volume with "-dmcrypt" is appended to the volume name, but
otherwise see no difference; I can still grep for text inside the volume.

Upon reading the source, I don't see "cryptsetup luksFormat" being called
anywhere (nova/libvirt/storage/*).

I was expecting a new encrypted LVM volume when a new instance was
created. Are my expectations misplaced? How is this feature envisioned to
work?

Thanks,

-Chris


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 25, 2016 by Chris_Buccella (660 points)   1
0 votes

Based on the comments to the RBD encryption change [1], it looks like there will be a new direction for ephemeral disk encryption (embedding it in QEMU directly). I assume LVM will work the same way when the time comes. Will there be a migration path for the existing ephemeral disk encryption support for LVM to the new model?

[1] https://review.openstack.org/#/c/239798/

Yes, as I understand it, the long-term goal is to provide encryption support directly in QEMU and have a unified interface for LVM, RBD, and file-based backends. I do not yet know what the potential migration path will look like.


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 25, 2016 by Coffman,_Joel_M. (480 points)   1
0 votes

On Mon, Apr 25, 2016 at 04:28:17PM +0000, Coffman, Joel M. wrote:
Based on the comments to the RBD encryption change [1], it looks
like there will be a new direction for ephemeral disk encryption
(embedding it in QEMU directly). I assume LVM will work the same
way when the time comes. Will there be a migration path for the
existing ephemeral disk encryption support for LVM to the new
model?

[1] https://review.openstack.org/#/c/239798/

Yes, as I understand it, the long-term goal is to provide encryption
support directly in QEMU and have a unified interface for LVM, RBD,
and file-based backends. I do not yet know what the potential
migration path will look like.

The forthcoming QEMU 2.6 release will include native support for the
LUKS data format. There is a test suite with QEMU to prove that this
is interoperable with the kernel dm-crypt/cryptsetup tools. So there
will be no data migration required. Nova will merely need to change
the way it configures to point QEMU directly to the encrypted LVM
volume, instead of creating a dm-crypt volume wrapper. QEMU will
then directly decrypt the LVM volume.

Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Apr 25, 2016 by Daniel_P._Berrange (27,920 points)   2 4 10
...