Skip to content

influxdb: Concurrent flush operations for PeriodicFlusher #2185

Closed
@codebien

Description

@codebien

The current PeriodicFlusher helper flushes samples synchronously and it delegates the eventual concurrency to the specific Output. This is something required because the out-of-order sometimes is not accepted (e.g. writing files). This approach has the downside for the flusher to make it very dependent on the output's rate achieved.

Instead, there are cases (e.g InfluxDB), where out-of-order is not relevant so we can add a safe concurrent flush for them and speed up things when few flush requests will get a bad response time.

Suggested implementation

Add a new field to the PeriodicFlusher struct for explicit the required concurrency limit, then spawn in run method the relative amount of goroutines.

A dedicated NewAsyncPeriodicFlusher(limit int) must be implemented to avoid a breaking change for the current API.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions