receiver/splunkhec does not start when both logs and metrics are enabled : failed to bind to address :8088: listen tcp :8088: bind: address already in use

[dsobolev-nr]

Component(s)

receiver/splunkhec

What happened?

Description

The collector with splunkhec receiver does not start if both logs and metrics flows enabled

Steps to Reproduce

Run the collector with the below config

Expected Result

The collector should start

Actual Result

Got the following error:
Error: cannot start pipelines: failed to bind to address :8088: listen tcp :8088: bind: address already in use

Collector version

v0.71.0

Environment information

Environment

OS: Amazon Linux 2
Compiler: go version go1.19.1 darwin/arm64 (cross-compiled)

OpenTelemetry Collector configuration

receivers:
  splunk_hec:

processors:

exporters:
  file:
    path: "/path/to/file"

service:
  telemetry:
    logs:
      level: "debug"
  pipelines:
    metrics:
      receivers: [splunk_hec]
      processors: []
      exporters: [file]
    logs:
      receivers: [splunk_hec]
      processors: []
      exporters: [file]

Log output

2023-02-16T23:24:58.708Z	info	service/telemetry.go:90	Setting up own telemetry...
2023-02-16T23:24:58.708Z	info	service/telemetry.go:116	Serving Prometheus metrics	{"address": ":8084", "level": "Basic"}
2023-02-16T23:24:58.708Z	debug	exporter/exporter.go:284	Alpha component. May change in the future.	{"kind": "exporter", "data_type": "metrics", "name": "file"}
2023-02-16T23:24:58.709Z	debug	exporter/exporter.go:284	Alpha component. May change in the future.	{"kind": "exporter", "data_type": "logs", "name": "file"}
2023-02-16T23:24:58.709Z	debug	receiver/receiver.go:305	Beta component. May change in the future.	{"kind": "receiver", "name": "splunk_hec", "data_type": "logs"}
2023-02-16T23:24:58.709Z	debug	receiver/receiver.go:305	Beta component. May change in the future.	{"kind": "receiver", "name": "splunk_hec", "data_type": "metrics"}
2023-02-16T23:24:58.709Z	info	service/service.go:140	Starting otelcontribcol...	{"Version": "0.71.0-dev", "NumCPU": 4}
2023-02-16T23:24:58.709Z	info	extensions/extensions.go:41	Starting extensions...
2023-02-16T23:24:58.709Z	warn	internal/warning.go:51	Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks	{"kind": "receiver", "name": "splunk_hec", "data_type": "logs", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2023-02-16T23:24:58.710Z	info	service/service.go:166	Starting shutdown...
2023-02-16T23:24:58.710Z	info	extensions/extensions.go:55	Stopping extensions...
2023-02-16T23:24:58.710Z	info	service/service.go:180	Shutdown complete.
Error: cannot start pipelines: failed to bind to address :8088: listen tcp :8088: bind: address already in use
2023/02/16 23:24:58 collector server run finished with error: cannot start pipelines: failed to bind to address :8088: listen tcp :8088: bind: address already in use

Additional context

I noticed, that when the Start() method is called for the second time, the following condition should become true

opentelemetry-collector-contrib/receiver/splunkhecreceiver/receiver.go

Line 186 in fdd01de

if r.server != nil && r.server.Handler != nil {

but it does not (r.server.Handler is not nil) and it's trying to bind again, and, obviously fails

[github-actions]

Pinging code owners:

• receiver/splunkhec: @atoulme

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

Yes, you will need to use separate ports for each as we don't currently support reusing the http server between receivers.

Make sure to define an additional splunk_hec receiver for metrics for example.

This is still a valid issue - we can look into this.

[Aneurysm9]

https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/internal/sharedcomponent can provide a way to use the same port for multiple signals.

[dsobolev-nr]

Thanks @atoulme. I guess, I just misunderstood the README (or it's might be not quite certain about that?).
@Aneurysm9 that's what I started with. I'm working on adding metrics to datadog receiver, and it did not work for me so far, so I just started looking at other receivers, and met this behavior. It's off-topic here though, so I'll keep trying, and if did not get any luck, will create a topic in Discussions. Thanks all!

[atoulme]

The Splunk HEC receiver is old, we can definitely use this shared component.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/splunkhec: @atoulme

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

Fixed by #19189