Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receivercreator] Validate endpoint's configuration before starting receivers #35416

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Sep 25, 2024

Description:

The receivercreator should validate the runtime configurations before starting a receiver. This PR adds this validation.

Link to tracking Issue: Fixes #33145

Testing:

Using the following helm chart's values:

mode: daemonset

config:
  extensions:
    k8s_observer:
      auth_type: serviceAccount
      node: ${env:K8S_NODE_NAME}
      observe_nodes: true
  exporters:
    debug:
      verbosity: basic
  receivers:
    receiver_creator/3:
      watch_observers: [ k8s_observer ]
      receivers:
        apache:
          rule: type == "k8s.node"
          config:
            endpoint: "http://localhost:8080/server-status?manual"
  service:
    extensions: [health_check, k8s_observer]
    pipelines:
      metrics:
        receivers: [receiver_creator/3]
        processors: [batch]
        exporters: [debug]

Expect to see an error after the config's validation:

2024-09-25T10:46:56.778Z	error	receivercreator@v0.110.0/observerhandler.go:159	failed to start receiver	{"kind": "receiver", "name": "receiver_creator/3", "data_type": "metrics", "receiver": "apache", "error": "invalid runtime receiver config: receivers::apache: query must be 'auto': 'http://localhost:8080/server-status?manual'"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/receivercreator.(*observerHandler).OnAdd
	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/receivercreator@v0.110.0/observerhandler.go:159
github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer.(*EndpointsWatcher).updateAndNotifyOfEndpoints
	github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer@v0.110.0/endpointswatcher.go:111

This stops the execution of the receiver which would lead to fetch errors (as described in the respective issue #33145):

2024-05-21T09:17:46.749Z	error	apachereceiver@v0.100.0/scraper.go:67	failed to fetch Apache Httpd stats	{"kind": "receiver", "name": "receiver_creator/3", "data_type": "metrics", "name": "apache//receiver_creator/3{endpoint=\"http://localhost:8080/server-status?manual\"}/k8s_observer/kind-control-plane-b75cdcf9-ee9d-4cb8-aa53-1ceb030d898d", "error": "Get \"http://localhost:8080/server-status?manual\": dial tcp [::1]:8080: connect: connection refused"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/apachereceiver.(*apacheScraper).scrape
	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/apachereceiver@v0.100.0/scraper.go:67
go.opentelemetry.io/collector/receiver/scraperhelper.ScrapeFunc.Scrape
	go.opentelemetry.io/collector/receiver@v0.100.1-0.20240509190532-c555005fcc80/scraperhelper/scraper.go:20
go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport
	go.opentelemetry.io/collector/receiver@v0.100.1-0.20240509190532-c555005fcc80/scraperhelper/scrapercontroller.go:194
go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1
	go.opentelemetry.io/collector/receiver@v0.100.1-0.20240509190532-c555005fcc80/scraperhelper/scrapercontroller.go:169
2024-05-21T09:17:46.750Z	error	scraperhelper/scrapercontroller.go:197	Error scraping metrics	{"kind": "receiver", "name": "receiver_creator/3", "data_type": "metrics", "name": "apache//receiver_creator/3{endpoint=\"http://localhost:8080/server-status?manual\"}/k8s_observer/kind-control-plane-b75cdcf9-ee9d-4cb8-aa53-1ceb030d898d", "error": "Get \"http://localhost:8080/server-status?manual\": dial tcp [::1]:8080: connect: connection refused", "scraper": "apache"}

Documentation:

…eceivers

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

receivercreator does not honor target's config validation
2 participants