Skip to content

Commit

Permalink
Use sarama built in support for consumer groups (influxdata#6172)
Browse files Browse the repository at this point in the history
  • Loading branch information
danielnelson authored and Mathieu Lecarme committed Apr 17, 2020
1 parent d5be13d commit 077ba24
Show file tree
Hide file tree
Showing 8 changed files with 643 additions and 433 deletions.
108 changes: 96 additions & 12 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 0 additions & 4 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,6 @@
name = "github.com/aws/aws-sdk-go"
version = "1.15.54"

[[constraint]]
name = "github.com/bsm/sarama-cluster"
version = "2.1.13"

[[constraint]]
name = "github.com/couchbase/go-couchbase"
branch = "master"
Expand Down
8 changes: 3 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
ifeq ($(SHELL), cmd)
VERSION := $(shell git describe --exact-match --tags 2>nil)
HOME := $(HOMEPATH)
else ifeq ($(SHELL), sh.exe)
ifeq ($(OS), Windows_NT)
VERSION := $(shell git describe --exact-match --tags 2>nil)
HOME := $(HOMEPATH)
CGO_ENABLED ?= 0
export CGO_ENABLED
else
VERSION := $(shell git describe --exact-match --tags 2>/dev/null)
endif
Expand Down Expand Up @@ -48,7 +47,6 @@ install: telegraf
mkdir -p $(DESTDIR)$(PREFIX)/bin/
cp telegraf $(DESTDIR)$(PREFIX)/bin/


.PHONY: test
test:
go test -short ./...
Expand Down
3 changes: 1 addition & 2 deletions docs/LICENSE_OF_DEPENDENCIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ following works:
- github.com/aws/aws-sdk-go [Apache License 2.0](https://github.com/aws/aws-sdk-go/blob/master/LICENSE.txt)
- github.com/Azure/go-autorest [Apache License 2.0](https://github.com/Azure/go-autorest/blob/master/LICENSE)
- github.com/beorn7/perks [MIT License](https://github.com/beorn7/perks/blob/master/LICENSE)
- github.com/bsm/sarama-cluster [MIT License](https://github.com/bsm/sarama-cluster/blob/master/LICENSE)
- github.com/cenkalti/backoff [MIT License](https://github.com/cenkalti/backoff/blob/master/LICENSE)
- github.com/cisco-ie/nx-telemetry-proto [Apache License 2.0](https://github.com/cisco-ie/nx-telemetry-proto/blob/master/LICENSE)
- github.com/couchbase/go-couchbase [MIT License](https://github.com/couchbase/go-couchbase/blob/master/LICENSE)
Expand Down Expand Up @@ -139,4 +138,4 @@ following works:
- gopkg.in/yaml.v2 [Apache License 2.0](https://github.com/go-yaml/yaml/blob/v2.2.2/LICENSE)

## telegraf used and modified code from these projects
- github.com/DataDog/datadog-agent [Apache License 2.0](https://github.com/DataDog/datadog-agent/LICENSE)
- github.com/DataDog/datadog-agent [Apache License 2.0](https://github.com/DataDog/datadog-agent/LICENSE)
17 changes: 10 additions & 7 deletions plugins/inputs/kafka_consumer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,13 @@ and use the old zookeeper connection method.

```toml
[[inputs.kafka_consumer]]
## kafka servers
## Kafka brokers.
brokers = ["localhost:9092"]
## topic(s) to consume

## Topics to consume.
topics = ["telegraf"]
## Add topic as tag if topic_tag is not empty

## When set this tag will be added to all metrics with the topic as the value.
# topic_tag = ""

## Optional Client id
Expand All @@ -37,10 +39,11 @@ and use the old zookeeper connection method.
# sasl_username = "kafka"
# sasl_password = "secret"

## the name of the consumer group
consumer_group = "telegraf_metrics_consumers"
## Offset (must be either "oldest" or "newest")
offset = "oldest"
## Name of the consumer group.
# consumer_group = "telegraf_metrics_consumers"

## Initial offset position; one of "oldest" or "newest".
# offset = "oldest"

## Maximum length of a message to consume, in bytes (default 0/unlimited);
## larger messages are dropped
Expand Down
Loading

0 comments on commit 077ba24

Please sign in to comment.