settingsLogin | Registersettings

[openstack-dev] Concerns about Gnocchi's performance

0 votes

Hello,

I?m testing gnocchi to work with ceilometer these days and I have some concerns about its performance.

I noticed in the code, every time a measure gets updated, gnocchi first reads all the previous contents of that measure, updates the measure in memory and writes the updated contents as well as original contents back. IMHO, the overhead here could be very large. As an improvement, I propose we use some form of seek that only modifies the updated parts.

So I?d like to hear about your opinions, and if there?s already a solution, please point it to me.

Regards,
Can Zhang
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: http://lists.openstack.org/pipermail/openstack-dev/attachments/20150115/a44a2ad0/attachment.pgp

asked Jan 15, 2015 in openstack-dev by =?utf-8?B?5byg54G_sl (160 points)   2
retagged Jan 28, 2015 by admin

3 Responses

0 votes

On Thu, Jan 15 2015, ?? wrote:

Hi,

I noticed in the code, every time a measure gets updated, gnocchi first reads
all the previous contents of that measure, updates the measure in memory and
writes the updated contents as well as original contents back. IMHO, the
overhead here could be very large. As an improvement, I propose we use some
form of seek that only modifies the updated parts.

Yep, that's the main downside of the use of the Carbonara based backend
in Gnocchi. But honestly, since it's download/upload are parallel and
the file size is quite small, sending a metric should be pretty fast in
most circumstances. Though that's sure it's gonna be faster with
backends where you just have to send the actual metrics (OpenTSDB or
InfluxDB likely).

So I?d like to hear about your opinions, and if there?s already a
solution, please point it to me.

I'd love to improve the backends you describe with some sort of `seek'
solution, unfortunately I don't think it'll be possible to do so using
Swift. Though it might be possible to do so using Ceph and the file
based backends, so maybe it'd be worth looking at that. Any help
appreciated!

--
Julien Danjou

Free Software hacker

http://julien.danjou.info

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 826 bytes
Desc: not available
URL:

responded Jan 15, 2015 by Julien_Danjou (20,500 points)   2 4 6
0 votes

On Jan 15, 2015, at 18:46, Julien Danjou wrote:

Yep, that's the main downside of the use of the Carbonara based backend
in Gnocchi. But honestly, since it's download/upload are parallel and
the file size is quite small, sending a metric should be pretty fast in
most circumstances. Though that's sure it's gonna be faster with
backends where you just have to send the actual metrics (OpenTSDB or
InfluxDB likely).

As a further question, how do you estimate the storage impact of a vm instance?s metrics? I?m not very sure about the back_window and definition part, could you clarify that for me?

Regards,
Can Zhang
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL:

responded Jan 15, 2015 by =?utf-8?B?5byg54G_sl (160 points)   2
0 votes

On Thu, Jan 15 2015, ?? wrote:

As a further question, how do you estimate the storage impact of a vm
instance?s metrics? I?m not very sure about the back_window and definition
part, could you clarify that for me?

That defines the number of measure (data points) that are stored for
each metric. The size of an actual metric depends on these parameters.
So depending on the granularity and the number of points, the size of a
metric can go from a few kilobytes to a few megabytes.

--
Julien Danjou
-- Free Software hacker
-- http://julien.danjou.info
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 826 bytes
Desc: not available
URL:

responded Jan 15, 2015 by Julien_Danjou (20,500 points)   2 4 6
...