[Configuration] Limit the memory usage of influxdb

[FGRibreau]

Hi guys!

I don't see any configuration option to limit the memory usage of influxdb, how can I achieve that?

If it's currently not configurable, it's definitely my first concern right now (in order to use it as a shadow-slave in production).

[pauldix]

It's not configurable right now, but we should definitely support that.
It'll be a little tricky because there's the LevelDB cache and then there's
stuff in Go. At the very least we can make the LevelDB cache limited, which
is probably where the vast majority of the memory usage will be. We can
also add setrlimit, but that might cause things to crash once it goes over.
More on this soon.

On Tue, Nov 12, 2013 at 1:43 PM, Francois-Guillaume Ribreau <
notifications@github.com> wrote:

Hi guys!

I don't see any configuration option to limit the memory usage of
influxdb, how can I achieve that?

If it's currently not configurable, it's definitely my first concern right
now (in order to use it as a shadow-slave in production).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/38
.

[FGRibreau]

Thanks! Keep me posted!

[predictivematch]

Hi, Any update on this, is this still on the roadmap?

[pauldix]

It is, I just assigned it to v0.6.0 which we're targeting for release in mid to late February.

[schmurfy]

for me 0.5.0 seems to reduce the issue a bit, influxdb now use a lot less memory on startup and manage to stabilize itself around 580MB, it stayed that way the whole night but as soon as I queried the data back this morning to check how it went it started growing again.

The memory growth rate seems slower than with 0.4.0, I will keep a close look on it to see how it goes but if there is not real way to control how much memory it can uses it will be hard to use it in production.

is there any config options beside limiting the number of openened leveldb databases which could help ? (the number of leveldb databases open does not seem to have a great impact)

Edit: Here are the results for last night and this morning:

[pauldix]

The leveldb settings are all that exist at the moment. What queries are you
running? Does the memory only spike while it runs or does it stay high?

Specific limits is one of the priority items for 0.6.0, but I think having
reasonable memory usage is a blocker for 0.5.0 general release.

On Thu, Feb 27, 2014 at 4:47 AM, Julien Ammous notifications@github.comwrote:

for me 0.5.0 seems to reduce the issue a bit, influxdb now use a lot less
memory on startup and manage to stabilize itself around 580MB, it stayed
that way the whole night but as soon as I queried the data back this
morning to check how it went it started growing again.

The memory growth rate seems slower than with 0.4.0, I will keep a close
look on it to see how it goes but if there is not real way to control how
much memory it can uses it will be hard to use it in production.

is there any config options beside limiting the number of openened leveldb
databases which could help ? (the number of leveldb databases open does not
seem to have a great impact)

Reply to this email directly or view it on GitHubhttps://github.com//issues/38#issuecomment-36225917
.

[schmurfy]

since I reduced the number of fields in my records memory usage are lower but my concern is more than all I see is memory usage increasing, never decreasing.

[schmurfy]

What does influxdb use all this memory for btw ?
From what I understand from the different discussions leveldb itself only use a small amount of memory as cache for each openened database which would means that most if not all the memory is used by influxdb itself right ?

My influxdb instances (both the production and test servers) are currently sitting around 1GB which is quite a lot of memory.

As a comparison I can look at our production mysql server which uses 126MB and currently stores a lot more data than influxdb.

[pauldix]

The Go part of the process shouldn't be using much. The only stuff it keeps in memory when not running queries is the cluster metadata. When you run queries, things get pulled into memory. We're going to be optimizing some of that in the coming weeks.

We can probably reduce the profile by tweaking how we're working with LevelDB. We'll add some settings for this in rc.3 and see if we can get this to a more reasonable number. This is high priority for us :)

[jvshahid]

@schmurfy can you explain what metric are you monitoring, is it virtual memory or resident memory ? Also, do you know or can guess what happened around 9:00 that triggered the memory to increase.

[FGRibreau]

We can probably reduce the profile by tweaking how we're working with LevelDB. We'll add some settings for this in rc.3 and see if we can get this to a more reasonable number. This is high priority for us :)

Great news! 👍

[schmurfy]

what caused the memory to increase is me loading the dashboard (which issues select queries to gather the data), the memory reported here is measured by libsigar and from different testing this is the "physical" memory used (mac os x calls it real memory which is the best name I have seen).
This value is also the one shown by top in the RES column.

My experience so far with 0.5.0 is similar to 0.4.0 but the later used more memory from the start and its use increased dramatically when the data were queried, with 0.5.0 it stills shows similar behavior but all the numbers are lower but I have no run it for anough time to really be sure, the week end should be a good test :)

One of the thing I am particularly interested is: does this memory usage ever goes down ?
I will see how it goes during the week end.

[schmurfy]

Here is another one for the last three days:

they are downscaled values but it appears memory consumption does decrease but really slowly.
and like before as soon as I query the data the memory steps up: