Skip to content

Statsd/Carbon backends drop metrics when creating more than 1024 items in short time #48

Open

Description

exportChan.open(maxItems = METRIC_EXPORT_BUFER_SIZE)
creates a queue of 1024 items - when performing metrics updates in a loop, this queue will easily overflow, creating an unreliable metrics flow.

There are several ways to improve the situation:

  • count dropped messages such that they can be discovered
  • allow configuring the queue size for applications with more fine-grained needs
  • don't post full metric on channel - instead, post a notification that a new value is available and send latest value to server at send time - in a burst, the current code will send the least recent value that is already on the queue and drop any newer values
  • drop the oldest values in the channel when it's full, instead of not posting the latest
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions