Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent label cardinality #10

Closed
panchiz opened this issue Oct 9, 2017 · 2 comments · Fixed by #16
Closed

Inconsistent label cardinality #10

panchiz opened this issue Oct 9, 2017 · 2 comments · Fixed by #16
Assignees

Comments

@panchiz
Copy link

panchiz commented Oct 9, 2017

Once the service is runnig after 2 or 4 requests to the /metrics endpoint I get:

Attaching to beanstalkdexporter_beanstalkd_1, beanstalkdexporter_beanstalkd_exporter_1
beanstalkd_exporter_1  | 2017/10/09 20:53:17 main.go:216: Listening on port :8080 .
beanstalkd_exporter_1  | 2017/10/09 20:53:17 main.go:217: Polling beanstalkd:11300 for stats every 30 seconds
beanstalkd_exporter_1  | panic: inconsistent label cardinality
beanstalkd_exporter_1  |
beanstalkd_exporter_1  | goroutine 16 [running]:
beanstalkd_exporter_1  | github.com/prometheus/client_golang/prometheus.(*MetricVec).With(0xc42017bc20, 0xc42017b890, 0xc42017bc20, 0x0)
beanstalkd_exporter_1  | 	/go/src/github.com/messagebird/beanstalkd_exporter/vendor/src/github.com/prometheus/client_golang/prometheus/vec.go:143 +0x76
beanstalkd_exporter_1  | github.com/prometheus/client_golang/prometheus.(*GaugeVec).With(0xc42017bc20, 0xc42017b890, 0x88f740, 0xc42017bc20)
beanstalkd_exporter_1  | 	/go/src/github.com/messagebird/beanstalkd_exporter/vendor/src/github.com/prometheus/client_golang/prometheus/gauge.go:119 +0x37
beanstalkd_exporter_1  | main.statTube(0xc4200ec140, 0x7ffcac2dff27, 0x10, 0xc42014f860, 0x7)
beanstalkd_exporter_1  | 	/go/src/github.com/messagebird/beanstalkd_exporter/src/beanstalkd_exporter/main.go:162 +0x7c5
beanstalkd_exporter_1  | main.poll(0x7ffcac2dff27, 0x10)
beanstalkd_exporter_1  | 	/go/src/github.com/messagebird/beanstalkd_exporter/src/beanstalkd_exporter/main.go:97 +0x44e
beanstalkd_exporter_1  | created by main.main.func1
beanstalkd_exporter_1  | 	/go/src/github.com/messagebird/beanstalkd_exporter/src/beanstalkd_exporter/main.go:222 +0xba
beanstalkdexporter_beanstalkd_exporter_1 exited with code 2

I'm running this using docker-compose (see #9)

@dbendelman
Copy link

@samwierema @marcelcorso I'm hitting this too. Launched beanstalkd_exporter next to a freshly launched and totally empty beanstalkd v1.10:

/ # time beanstalkd_exporter -poll 5 -log.level debug
2017/10/17 19:40:44 main.go:216: Listening on port :8080 .
2017/10/17 19:40:44 main.go:217: Polling localhost:11300 for stats every 5 seconds
2017/10/17 19:40:49 main.go:44: Debug: Calling localhost:11300 stats()
2017/10/17 19:40:49 main.go:85: Debug: Calling localhost:11300 ListTubes()
2017/10/17 19:40:49 main.go:104: Debug: Calling localhost:11300 Tube{name: default}.Stats()
panic: inconsistent label cardinality

goroutine 27 [running]:
github.com/prometheus/client_golang/prometheus.(*MetricVec).With(0xc4201cf170, 0xc4201cede0, 0xc4201cf170, 0x0)
	/go/src/github.com/messagebird/beanstalkd_exporter/vendor/src/github.com/prometheus/client_golang/prometheus/vec.go:143 +0x76
github.com/prometheus/client_golang/prometheus.(*GaugeVec).With(0xc4201cf170, 0xc4201cede0, 0x8c4a80, 0xc4201cf170)
	/go/src/github.com/messagebird/beanstalkd_exporter/vendor/src/github.com/prometheus/client_golang/prometheus/gauge.go:119 +0x37
main.statTube(0xc42014a640, 0x74b64c, 0xf, 0xc42013ad60, 0x7)
	/go/src/github.com/messagebird/beanstalkd_exporter/src/beanstalkd_exporter/main.go:162 +0x793
main.poll(0x74b64c, 0xf)
	/go/src/github.com/messagebird/beanstalkd_exporter/src/beanstalkd_exporter/main.go:97 +0x7bc
created by main.main.func1
	/go/src/github.com/messagebird/beanstalkd_exporter/src/beanstalkd_exporter/main.go:222 +0x96
Command exited with non-zero status 2
real	0m 5.09s
user	0m 0.00s
sys	0m 0.00s

As you can see from the above, no need to request the /metrics endpoint as the OP mentioned; the panic happens right after the first poll.

Based on this comment, it seems beanstalkd_exporter is giving the Prometheus client library the same metric multiple times but with a different number of labels, which is not allowed.

@marcelcorso
Copy link
Contributor

@panchiz @dbendelman thanks for the report! I'll take a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants