Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to specify outputs as lists #370

Merged
merged 1 commit into from
Nov 16, 2015
Merged

Allow users to specify outputs as lists #370

merged 1 commit into from
Nov 16, 2015

Conversation

sparrc
Copy link
Contributor

@sparrc sparrc commented Nov 13, 2015

This will provide the ability to specify multiple outputs for a single
type of output.

In essence, allowing this:

[outputs]

[[outputs.influxdb]]
  urls = ["udp://localhost:8089"]
  database = "udp-telegraf"

[[outputs.influxdb]]
  urls = ["http://myhost:8086"]
  database = "telegraf"

[[outputs.kafka]]
  brokers = ["192.168.99.100:9092"]
  topic = "telegraf"

closes #335

@sparrc
Copy link
Contributor Author

sparrc commented Nov 13, 2015

This will not be a breaking change after all, with this change, Telegraf will assign identifiers to the names of the outputs, so the Telegraf output will look something like this (with the above config):

% ./telegraf -config ~/ws/telegraf.conf-test -filter cpu
2015/11/13 16:11:20 Starting Telegraf (version v0.2.0-58-gbf8e0f4)
2015/11/13 16:11:20 Loaded outputs: influxdb-0 influxdb-1 kafka-0
2015/11/13 16:11:20 Loaded plugins: cpu
2015/11/13 16:11:20 Tags enabled:
2015/11/13 16:11:30 Gathered metrics, (10s interval), from 1 plugins in 160.361µs
2015/11/13 16:11:40 Gathered metrics, (10s interval), from 1 plugins in 619.351µs
2015/11/13 16:11:40 Flushed 90 metrics to output influxdb-1 in 249.043µs
2015/11/13 16:11:40 Flushed 90 metrics to output influxdb-0 in 6.362077ms
2015/11/13 16:11:40 Flushed 90 metrics to output kafka-0 in 22.188595ms

The numbers are guaranteed to be based on the order they appear in the config file.

The previous config file convention of specifying a single output for each name will still be supported, each output will get an ID of 0.

cc @pauldix @dswarbrick @daviesalex @sebito91 please review

@sparrc
Copy link
Contributor Author

sparrc commented Nov 13, 2015

also note that the above IDs are pretty much just for logging & internal telegraf purposes, they don't show up in the database and all regular tags still apply

@sparrc
Copy link
Contributor Author

sparrc commented Nov 13, 2015

Also this config would still work:

[outputs]

[outputs.influxdb]
  urls = ["http://localhost:8086"]
  database = "udp-telegraf"

[outputs.kafka]
  brokers = ["192.168.99.100:9092"]
  topic = "telegraf"

both outputs would get id==0

% ./telegraf -config ~/ws/telegraf.conf-test -filter cpu
2015/11/13 16:11:20 Starting Telegraf (version v0.2.0-58-gbf8e0f4)
2015/11/13 16:11:20 Loaded outputs: influxdb-0 kafka-0
2015/11/13 16:11:20 Loaded plugins: cpu
2015/11/13 16:11:20 Tags enabled:
2015/11/13 16:11:30 Gathered metrics, (10s interval), from 1 plugins in 160.361µs
2015/11/13 16:11:40 Gathered metrics, (10s interval), from 1 plugins in 619.351µs
2015/11/13 16:11:40 Flushed 90 metrics to output influxdb-0 in 6.362077ms
2015/11/13 16:11:40 Flushed 90 metrics to output kafka-0 in 22.188595ms

err = c.parseOutput(outputName, outputSubtbl)
if err != nil {
return nil, err
switch outputSubtbl := outputVal.(type) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit, outputSubtbl feels awkward (same with tbl in the rest of this function, but that's out of scope). outputSubTbl or even outputSubTable would be more readable.

@gunnaraasen
Copy link
Member

LGTM, excited for this feature!

for id, t := range outputSubtbl {
err = c.parseOutput(outputName, t, id)
if err != nil {
return nil, err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here re: unable to parse messaging.

@sebito91
Copy link
Contributor

My comments are out of order above, but basically all LGTM save adding some debug code to understand which output failed to parse.

This will provide the ability to specify multiple outputs for a single
type of output.

In essence, allowing this:

[outputs]

[[outputs.influxdb]]
  urls = ["udp://localhost:8089"]
  database = "udp-telegraf"

[[outputs.influxdb]]
  urls = ["http://myhost:8086"]
  database = "telegraf"

[[outputs.kafka]]
  brokers = ["192.168.99.100:9092"]
  topic = "telegraf"

closes #335
@sparrc
Copy link
Contributor Author

sparrc commented Nov 16, 2015

@sebito91 @gunnaraasen Thanks for your input, this is now in master and will be available in 0.2.1

@sparrc sparrc deleted the multi-output branch November 16, 2015 17:33
@sebito91
Copy link
Contributor

@sparrc, thank YOU!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants