v0.41.0 http panic on scrape

[Deshke]

mongodb-exporter-1  | 2024/09/18 08:19:00 http: panic serving 10.0.23.73:39496: descriptor Desc{fqName: "collector_scrape_time_ms", help: "Time taken for scrape by collector", constLabels: {collector="profile"}, variableLabels: {exporter}} already exists with the same fully-qualified name and const label values
mongodb-exporter-1  | goroutine 9518 [running]:
mongodb-exporter-1  | net/http.(*conn).serve.func1()
mongodb-exporter-1  | 	/opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:1903 +0xbe
mongodb-exporter-1  | panic({0xd1c980?, 0xc000bf6bd0?})
mongodb-exporter-1  | 	/opt/hostedtoolcache/go/1.22.7/x64/src/runtime/panic.go:770 +0x132
mongodb-exporter-1  | github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister(...)
mongodb-exporter-1  | 	/home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.19.1/prometheus/registry.go:405
mongodb-exporter-1  | github.com/percona/mongodb_exporter/exporter.(*Exporter).makeRegistry(0xc0006b26c0, {0x13934f8, 0xc00023c000}, 0xc0001b0620, {0x13901b8, 0xc000f02990}, {{0xc00050fc80, 0x1, 0x1}, 0x0, ...})
mongodb-exporter-1  | 	/home/runner/work/mongodb_exporter/mongodb_exporter/exporter/exporter.go:247 +0xc19
mongodb-exporter-1  | github.com/percona/mongodb_exporter/exporter.RunWebServer.(*Exporter).Handler.func2({0x1392720, 0xc0001b00e0}, 0xc0000ea240)
mongodb-exporter-1  | 	/home/runner/work/mongodb_exporter/mongodb_exporter/exporter/exporter.go:376 +0x6cf
mongodb-exporter-1  | net/http.HandlerFunc.ServeHTTP(0x19c6f40?, {0x1392720?, 0xc0001b00e0?}, 0x7fa63a?)
mongodb-exporter-1  | 	/opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2171 +0x29
mongodb-exporter-1  | net/http.(*ServeMux).ServeHTTP(0x471859?, {0x1392720, 0xc0001b00e0}, 0xc0000ea240)
mongodb-exporter-1  | 	/opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2688 +0x1ad
mongodb-exporter-1  | net/http.serverHandler.ServeHTTP({0xc000dd8690?}, {0x1392720?, 0xc0001b00e0?}, 0x6?)
mongodb-exporter-1  | 	/opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:3142 +0x8e
mongodb-exporter-1  | net/http.(*conn).serve(0xc0004ba120, {0x1393450, 0xc0004b0390})
mongodb-exporter-1  | 	/opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2044 +0x5e8
mongodb-exporter-1  | created by net/http.(*Server).Serve in goroutine 52
mongodb-exporter-1  | 	/opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:3290 +0x4b4

  exporter:
    image: percona/mongodb_exporter:0.41.0
    restart: always
    network_mode: "host"
    volumes:
      - /storage/mongodb/secrets:/secrets
    command: "--discovering-mode --collect-all --log.level=info --no-mongodb.direct-connect"
    environment:
      - MONGODB_URI=mongodb://root_user:xxxx@node1:27020,node2:27020,node3:27020/admin?ssl=true&tlsCertificateKeyFile=secrets/node.pem&tlsCAFile=secrets/ca.crt

[Deshke]

cc @BupycHuk

[BupycHuk]

Hi, could you share HTTP request as well, please?

[Deshke]

curl -vks localhost:9216/metrics
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9216 (#0)
> GET /metrics HTTP/1.1
> Host: localhost:9216
> User-Agent: curl/7.61.1
> Accept: */*
>
* Empty reply from server
* Connection #0 to host localhost left intact

[BupycHuk]

could you add ?target=node1 to your query? can be used with node2 or node3 as well

[Deshke]

root@...# curl -vks "localhost:9217/metrics?node=node1:27020"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9217 (#0)
> GET /metrics?node=node1:27020 HTTP/1.1
> Host: localhost:9217
> User-Agent: curl/7.61.1
> Accept: */*
>
* Empty reply from server
* Connection #0 to host localhost left intact
root@.....:/storage/mongodb# docker-compose logs exporter-cluster
mongodb-exporter-cluster-1  | level=info ts=2024-09-19T13:23:46.320Z caller=tls_config.go:313 msg="Listening on" address=[::]:9217
mongodb-exporter-cluster-1  | level=info ts=2024-09-19T13:23:46.320Z caller=tls_config.go:316 msg="TLS is disabled." http2=false address=[::]:9217
mongodb-exporter-cluster-1  | 2024/09/19 13:24:21 http: panic serving 127.0.0.1:35288: descriptor Desc{fqName: "collector_scrape_time_ms", help: "Time taken for scrape by collector", constLabels: {collector="profile"}, variableLabels: {exporter}} already exists with the same fully-qualified name and const label values
mongodb-exporter-cluster-1  | goroutine 259 [running]:
mongodb-exporter-cluster-1  | net/http.(*conn).serve.func1()
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:1903 +0xbe
mongodb-exporter-cluster-1  | panic({0xd1c980?, 0xc000aebba0?})
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/runtime/panic.go:770 +0x132
mongodb-exporter-cluster-1  | github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister(...)
mongodb-exporter-cluster-1  |   /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.19.1/prometheus/registry.go:405
mongodb-exporter-cluster-1  | github.com/percona/mongodb_exporter/exporter.(*Exporter).makeRegistry(0xc000536810, {0x13934f8, 0xc00063e620}, 0xc0000e0620, {0x13901b8, 0xc00064f020}, {{0xc00003e240, 0x1, 0x1}, 0x0, ...})
mongodb-exporter-cluster-1  |   /home/runner/work/mongodb_exporter/mongodb_exporter/exporter/exporter.go:247 +0xc19
mongodb-exporter-cluster-1  | github.com/percona/mongodb_exporter/exporter.RunWebServer.(*Exporter).Handler.func2({0x1392720, 0xc0000e00e0}, 0xc0000c2000)
mongodb-exporter-cluster-1  |   /home/runner/work/mongodb_exporter/mongodb_exporter/exporter/exporter.go:376 +0x6cf
mongodb-exporter-cluster-1  | net/http.HandlerFunc.ServeHTTP(0x19c6f40?, {0x1392720?, 0xc0000e00e0?}, 0x7fa63a?)
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2171 +0x29
mongodb-exporter-cluster-1  | net/http.(*ServeMux).ServeHTTP(0x471859?, {0x1392720, 0xc0000e00e0}, 0xc0000c2000)
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2688 +0x1ad
mongodb-exporter-cluster-1  | net/http.serverHandler.ServeHTTP({0xc0002126f0?}, {0x1392720?, 0xc0000e00e0?}, 0x6?)
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:3142 +0x8e
mongodb-exporter-cluster-1  | net/http.(*conn).serve(0xc0000c8000, {0x1393450, 0xc000536ba0})
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2044 +0x5e8
mongodb-exporter-cluster-1  | created by net/http.(*Server).Serve in goroutine 15
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:3290 +0x4b4

[BupycHuk]

curl -vks "localhost:9217/metrics?target=node1:27020"

[Deshke]

same diff

curl -vks "localhost:9217/metrics?target=node1:27020"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9217 (#0)
> GET /metrics?target=node1:27020 HTTP/1.1
> Host: localhost:9217
> User-Agent: curl/7.61.1
> Accept: */*
>
* Empty reply from server
* Connection #0 to host localhost left intact


mongodb-exporter-cluster-1  | 2024/09/19 13:44:59 http: panic serving 127.0.0.1:36706: descriptor Desc{fqName: "collector_scrape_time_ms", help: "Time taken for scrape by collector", constLabels: {collector="profile"}, variableLabels: {exporter}} already exists with the same fully-qualified name and const label values
mongodb-exporter-cluster-1  | goroutine 1631 [running]:
mongodb-exporter-cluster-1  | net/http.(*conn).serve.func1()
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:1903 +0xbe
mongodb-exporter-cluster-1  | panic({0xd1c980?, 0xc000e8aea0?})
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/runtime/panic.go:770 +0x132
mongodb-exporter-cluster-1  | github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister(...)
mongodb-exporter-cluster-1  |   /home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.19.1/prometheus/registry.go:405
mongodb-exporter-cluster-1  | github.com/percona/mongodb_exporter/exporter.(*Exporter).makeRegistry(0xc000536810, {0x13934f8, 0xc000476000}, 0xc0001b0540, {0x13901b8, 0xc000646810}, {{0xc00003e240, 0x1, 0x1}, 0x0, ...})
mongodb-exporter-cluster-1  |   /home/runner/work/mongodb_exporter/mongodb_exporter/exporter/exporter.go:247 +0xc19
mongodb-exporter-cluster-1  | github.com/percona/mongodb_exporter/exporter.RunWebServer.(*Exporter).Handler.func2({0x1392720, 0xc0001b00e0}, 0xc00037b7a0)
mongodb-exporter-cluster-1  |   /home/runner/work/mongodb_exporter/mongodb_exporter/exporter/exporter.go:376 +0x6cf
mongodb-exporter-cluster-1  | net/http.HandlerFunc.ServeHTTP(0x19c6f40?, {0x1392720?, 0xc0001b00e0?}, 0x7fa63a?)
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2171 +0x29
mongodb-exporter-cluster-1  | net/http.(*ServeMux).ServeHTTP(0x471859?, {0x1392720, 0xc0001b00e0}, 0xc00037b7a0)
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2688 +0x1ad
mongodb-exporter-cluster-1  | net/http.serverHandler.ServeHTTP({0xc00087c5d0?}, {0x1392720?, 0xc0001b00e0?}, 0x6?)
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:3142 +0x8e
mongodb-exporter-cluster-1  | net/http.(*conn).serve(0xc0006d2090, {0x1393450, 0xc000536ba0})
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:2044 +0x5e8
mongodb-exporter-cluster-1  | created by net/http.(*Server).Serve in goroutine 15
mongodb-exporter-cluster-1  |   /opt/hostedtoolcache/go/1.22.7/x64/src/net/http/server.go:3290 +0x4b4

[AsoTora]

+on this, got same error when changing the config

in systemd:

/usr/local/bin/mongodb_exporter --mongodb.uri=mongodb://user:pass@localhost:27017 --collect-all --compatible-mode

in logs:

24 09:02:08 db1.signbucks.com mongodb_exporter[2245363]: 2024/09/24 09:02:08 http: panic serving 127.0.0.1:55584: descriptor Desc{fqName: "collector_scrape_time_ms", help: "Time taken for scrape by collector", constLabels: {collector="profile"}, variableLabels: {exporter}} already exists with the same fully-qualified name and const label values

UPD: Rolling back to the 0.40.0 with the same config displayed metrics

[BupycHuk]

thank you, we know what caused the problem and will fix it as soon as possible