settingsLogin | Registersettings

[Openstack-operators] Mixed env for nova (ceph for some compute nodes, local disk for the rest): qcow2 or raw images ?

0 votes

Hi

Currently in our Cloud we are using a gluster storage for cinder and glance.
For nova we are using a shared file system (implemented using gluster) for
part of the compute nodes; the rest of the compute nodes use the local disk.

We are now planning the replacement of gluster with ceph. The idea is
therefore to use ceph for cinder, glance. Ceph would be used for nova but
just for a set of compute nodes (the other compute nodes would keep using
the local disk).

In such configuration I see a problem with the choice of the best format
type
for images.

As far as I understand (please correct me if am wrong) the ideal setup
would be using raw images for VMs targeted to compute nodes using ceph, and
qcow2 images for VMs targeted to compute nodes using the local disk for
nova.
In fact starting a VM using a qcow2 image on a compute node using ceph for
nova works but it is quite inefficient since the qcow2 image must be first
downloaded in /var/lib/nova/instances/_base and then converted into raw.
This also means that some space is needed on the local disk.

And if you start a VM using a raw image on a a compute node using the local
disk for nova, the raw image (usually quite big) must be downloaded on the
compute node, and this is less efficient wrt a qcow2 image. It is true that
the qcow2 is then converted into raw, but I think that most of the time is
taken in downloading the image.

Did I get it right ?
Any advice ?

Thanks, Massimo


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
asked Apr 5, 2017 in openstack-operators by Massimo_Sgaravatto (780 points)   3 5

2 Responses

0 votes

Hi Massimo,

You can upload the images twice, in both qcow2 and raw format, then create
a host aggregate for your "local-disk" compute nodes and set its metadata
to match the property you'll set on your qcow2 images.

When somebody will start a qcow2 version of the image, it will be scheduled
on your compute nodes with local disk and pull the qcow2 image from Glance.

Does it make sense?

George

On Wed, Apr 5, 2017 at 10:05 AM, Massimo Sgaravatto <
massimo.sgaravatto@gmail.com> wrote:

Hi

Currently in our Cloud we are using a gluster storage for cinder and
glance.
For nova we are using a shared file system (implemented using gluster) for
part of the compute nodes; the rest of the compute nodes use the local disk.

We are now planning the replacement of gluster with ceph. The idea is
therefore to use ceph for cinder, glance. Ceph would be used for nova but
just for a set of compute nodes (the other compute nodes would keep using
the local disk).

In such configuration I see a problem with the choice of the best format
type
for images.

As far as I understand (please correct me if am wrong) the ideal setup
would be using raw images for VMs targeted to compute nodes using ceph, and
qcow2 images for VMs targeted to compute nodes using the local disk for
nova.
In fact starting a VM using a qcow2 image on a compute node using ceph for
nova works but it is quite inefficient since the qcow2 image must be first
downloaded in /var/lib/nova/instances/_base and then converted into raw.
This also means that some space is needed on the local disk.

And if you start a VM using a raw image on a a compute node using the
local disk for nova, the raw image (usually quite big) must be downloaded
on the compute node, and this is less efficient wrt a qcow2 image. It is
true that the qcow2 is then converted into raw, but I think that most of
the time is taken in downloading the image.

Did I get it right ?
Any advice ?

Thanks, Massimo


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Apr 5, 2017 by lmihaiescu_at_gmail. (1,620 points)   2
0 votes

Hi George

Thanks for your feedback

Yes, it makes sense, but most of our users upload their own images.
Educating them to upload the image twice and setting the metadata on these
images won't be easy at all

Thanks, Massimo

2017-04-05 16:23 GMT+02:00 George Mihaiescu lmihaiescu@gmail.com:

Hi Massimo,

You can upload the images twice, in both qcow2 and raw format, then create
a host aggregate for your "local-disk" compute nodes and set its metadata
to match the property you'll set on your qcow2 images.

When somebody will start a qcow2 version of the image, it will be
scheduled on your compute nodes with local disk and pull the qcow2 image
from Glance.

Does it make sense?

George

On Wed, Apr 5, 2017 at 10:05 AM, Massimo Sgaravatto <
massimo.sgaravatto@gmail.com> wrote:

Hi

Currently in our Cloud we are using a gluster storage for cinder and
glance.
For nova we are using a shared file system (implemented using gluster)
for part of the compute nodes; the rest of the compute nodes use the local
disk.

We are now planning the replacement of gluster with ceph. The idea is
therefore to use ceph for cinder, glance. Ceph would be used for nova but
just for a set of compute nodes (the other compute nodes would keep using
the local disk).

In such configuration I see a problem with the choice of the best format
type
for images.

As far as I understand (please correct me if am wrong) the ideal setup
would be using raw images for VMs targeted to compute nodes using ceph, and
qcow2 images for VMs targeted to compute nodes using the local disk for
nova.
In fact starting a VM using a qcow2 image on a compute node using ceph
for nova works but it is quite inefficient since the qcow2 image must be
first downloaded in /var/lib/nova/instances/_base and then converted into
raw. This also means that some space is needed on the local disk.

And if you start a VM using a raw image on a a compute node using the
local disk for nova, the raw image (usually quite big) must be downloaded
on the compute node, and this is less efficient wrt a qcow2 image. It is
true that the qcow2 is then converted into raw, but I think that most of
the time is taken in downloading the image.

Did I get it right ?
Any advice ?

Thanks, Massimo


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
responded Apr 5, 2017 by Massimo_Sgaravatto (780 points)   3 5
...