Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nacos uses 'http_sd_configs' which is not well adapted to prometheus #12196

Closed
kuman62 opened this issue Jun 9, 2024 · 4 comments · Fixed by #12239
Closed

nacos uses 'http_sd_configs' which is not well adapted to prometheus #12196

kuman62 opened this issue Jun 9, 2024 · 4 comments · Fixed by #12239
Labels
contribution welcome kind/discussion Category issues related to discussion

Comments

@kuman62
Copy link

kuman62 commented Jun 9, 2024

Describe the bug
Prometheus uses the http_sd_configs method, and the labels field in its protocol does not support the symbols . and -

#Nacos
#http://127.0.0.1:8848/nacos/prometheus http return
[{"targets":["192.168.31.20:9084"],"labels":{"preserved.register.source":"SPRING_CLOUD"}}]

#prometheus.yml
scrape_configs:

#prometheus run log
ts=2024-06-09T18:29:25.243Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=nacos-public msg="Unable to refresh target groups" err=""preserved.register.source" is not a valid label name"

@taomaree
Copy link
Contributor

taomaree commented Jun 11, 2024

nacos: 2.4.0-BETA
prometheus: 2.50 2.52

Prometheus label names allow to use _, not allow to use - or .
maybe need to auto replace lable name from - or . to _

https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels

Labels may contain ASCII letters, numbers, as well as underscores. They must match the regex [a-zA-Z_][a-zA-Z0-9_]*

#12144

@KomachiSion
Copy link
Collaborator

the metadata is registered by users, which means registered by yourself, please registered without this words when you want to use prometheus sd.

if use replace to _ , it might cause the message means dropped. See community suggest and callback.

BTW. I think we can add an switch properties to open or close this feature.

@KomachiSion KomachiSion added contribution welcome kind/discussion Category issues related to discussion labels Jun 12, 2024
@taomaree
Copy link
Contributor

taomaree commented Jun 12, 2024

BTW. I think we can add an switch properties to open or close this feature.

good idea.

label "preserved.register.source":"SPRING_CLOUD" is auto added by nacos sdk,
and another example is label "management.endpoints.web.base-path":"/xxxx" , which auto exported by nacos sdk, label name contains two invalid words . and - .
above 2 label name are not controlled or registered by enduser, and prometheus can't parse these label names, then this scrape job failed, this job results show nothing.

maybe convert label name from . and - words to_ in prometheus/src/main/java/com/alibaba/nacos/prometheus/utils/PrometheusUtils.java only affect prometheus exporter info.

@KomachiSion
Copy link
Collaborator

BTW. I think we can add an switch properties to open or close this feature.

good idea.

label "preserved.register.source":"SPRING_CLOUD" is auto added by nacos sdk, and another example is label "management.endpoints.web.base-path":"/xxxx" , which auto exported by nacos sdk, label name contains two invalid words . and - . above 2 label name are not controlled or registered by enduser, and prometheus can't parse these label names, then this scrape job failed, this job results show nothing.

maybe convert label name from . and - words to_ in prometheus/src/main/java/com/alibaba/nacos/prometheus/utils/PrometheusUtils.java only affect prometheus exporter info.

the label is not added by nacos sdk, it added by applications. same with management.endpoints.web.base-path.

KomachiSion pushed a commit that referenced this issue Jul 1, 2024
* prometheus http sd: fix invalid label names

* Update PrometheusUtils.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution welcome kind/discussion Category issues related to discussion
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants