We're experiencing issues with running large instances (~60GB RAM) on
fairly large NUMA nodes (4 CPUs, 256GB RAM) while using cpu pinning. The
problem is that it seems that in some extreme cases qemu/KVM can have
significant memory overhead (10-15%?) which nova-compute service doesn't
take in to the account when launching VMs. Using our configuration as an
example - imagine running two VMs with 30GB RAM on one NUMA node
(because we use cpu pinning) - therefore using 60GB out of 64GB for
given NUMA domain. When both VMs would consume their entire memory
(given 10% KVM overhead) OOM killer takes an action (despite having
plenty of free RAM in other NUMA nodes). (the numbers are just
arbitrary, the point is that nova-scheduler schedules the instance to
run on the node because the memory seems 'free enough', but specific
NUMA node can be lacking the memory reserve).
Our initial solution was to use ramallocationratio < 1 to ensure
having some reserved memory - this didn't work. Upon studying source of
nova, it turns out that ramallocationratio is ignored when using cpu
). We're running Mitaka, but this piece of code is implemented in Ocata
in a same way.
We're considering to create a patch for taking ramallocationratio in
My question is - is ramallocationratio ignored on purpose when using
cpu pinning? If yes, what is the reasoning behind it? And what would be
the right solution to ensure having reserved RAM on the NUMA nodes?
OpenStack Development Mailing List (not for usage questions)