Skip to content

Memory leak in CKMSQuantiles.java? #422

Closed
@mdedetrich

Description

@mdedetrich

At work we have noticed that one of our applications have started leaking memory. After diagnosing with VisualVM over a period of time, I have noticed that the heap size of io.prometheus.client.CKMSQuantiles$Item is gradually increasing over time without it ever lowering.

Using VisualVM, here is a screenshot of the first snaphot that we did
screen shot 2018-09-10 at 16 38 05
And here is the second, around an hour later
screen shot 2018-09-10 at 16 39 28
As you can see, the amounts of int's have also increased which seem to correspond to the members of the Quantiles Item class.

Although its expected that the number of items in the quantile increase, in our case the memory is never freed and our applications actually restart due to OOM roughly once a day. Our bucket size by default is 10 minutes, so it should be freeing up the memory shortly after that point.

We are btw using https://developer.lightbend.com/docs/cinnamon/current/plugins/prometheus/prometheus.html which uses client_java under the hood, you can also see the default configuration settings.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions