Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flush based on buffer size rather than time #699

Merged
merged 1 commit into from
Feb 17, 2016
Merged

Flush based on buffer size rather than time #699

merged 1 commit into from
Feb 17, 2016

Conversation

sparrc
Copy link
Contributor

@sparrc sparrc commented Feb 16, 2016

closes #666

@sparrc sparrc force-pushed the cs666 branch 2 times, most recently from b496d53 to 93106be Compare February 16, 2016 00:47
@netixen
Copy link
Contributor

netixen commented Feb 16, 2016

Awesome!

@sparrc sparrc force-pushed the cs666 branch 3 times, most recently from 91aaa99 to 18f77bb Compare February 16, 2016 22:52
@tripledes
Copy link

@sparrc nice! Thanks, I'll see when I can give it a try.

@sparrc sparrc force-pushed the cs666 branch 10 times, most recently from 3d571ff to f8077bc Compare February 17, 2016 01:56
@sparrc sparrc changed the title [WIP] Add Accumulator to the ServiceInput Start() function Flush based on buffer size rather than time Feb 17, 2016
@sparrc
Copy link
Contributor Author

sparrc commented Feb 17, 2016

@netixen @tripledes this is pretty much ready to go, if you have some time to put some eyes on it and review it would be great :)

The changes in the nats_consumer, mqtt_consumer, and kafka_consumer highlight the changes to the service plugins. Then the changes in running_output are the meat of the new flushing method.

this includes:
- Add Accumulator to the Start() function of service inputs
- For message consumer plugins, use the Accumulator to constantly add
  metrics and make Gather a dummy function
- rework unit tests to match this new behavior.
- make "flush_buffer_when_full" a config option that defaults to true

closes #666
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Telegraf should have a way of flushing based on buffer size rather than time
3 participants