settingsLogin | Registersettings

[Openstack] Network speed issue

0 votes

Hi all!

In my OpenStack installation (Icehouse and use nova legacy networking)
the VMs are talking to each other over a 1Gbps network link.

My issue is that although file transfers between physical (hypervisor)
nodes can saturate that link transfers between VMs reach very lower
speeds e.g. 30MB/s (approx. 240Mbps).

My tests are performed by scp'ing a large image file (approx. 4GB)
between the nodes and between the VMs.

I have updated my images to use e1000 nic driver but the results remain
the same.

What are any other limiting factors?

Does it has to do with the disk driver I am using? Does it play
significant role the filesystem of the hypervisor node?

Any ideas on how to approach the saturation of the 1Gbps link?

Best regards,


Mailing list:
Post to :
Unsubscribe :
asked Dec 16, 2014 in openstack by Georgios_Dimitrakaki (5,080 points)   3 11 16
retagged Feb 25, 2015 by admin

21 Responses

0 votes

Both results show a really high bandwidth but I can't get that speed for
file transfers.

Well, at least we can more or less exonerate networking :) Though
frankly I've always felt that virtio was better than an emulated NIC.

How much memory to you have in the VM receiving the 4 GB file being
transferred? One hypothesis, and it is only a hypothesis, is that the
reason the file transfer goes quicker at the beginning (if I recall
correctly) is the receiving VM is still just writing to filecache within
the VM itself. Once the number of ditry pages in the filecache gets to
some limit (which I can never remember) a flushing daemon will kick-in
and start pushing pages out to "disc" (well, what the VM thinks of as
its disc, whatever you've setup there). At that point there may still
be something of a "race" between the arriving bytes of the file and the
flushing and if there ends-up being enough memory consumed by the
filecache, writes to the file will end-up getting blocked, waiting for
space to free-up. And that won't happen any faster than the flusher can
get them out.

And then there can be the similar thing happening between the VM and the
host, where in this part we can think of the VM as being akin to the scp
receiving process.

happy benchmarking,

rick jones

Mailing list:
Post to :
Unsubscribe :
responded Dec 16, 2014 by Rick_Jones (1,240 points)   1 2