Support for default configuration

[pavolloffay]

The document https://docs.google.com/document/d/1NeheFG7DmcUYo_h2vLtNRlia9x5wOJMlV4QKEK05FhQ/edit# explains that OTEL collector will by default use a default configuration and if a configuration is provided it will be merged. Is it still on the roadmap?

[bogdandrutu]

The behavior is already supported, the only problem is that the default config for every component is very minimal right now and a lot of times incomplete

[bogdandrutu]

@pavolloffay Can you clarify what we miss here. For the moment we do offer every component to provide a default configuration, probably one missing part is to have default components registered by default.

When it comes to pipeline we don't have a default concept, that's something that we probably miss and need to add.

[tigrannajaryan]

@bogdandrutu we have default configs for components, but they are only effective if you manually enable the component.

In theory a service-wide default config can also enable certain components, so that if you run Collector without any config file for example it will always have several receivers enabled (e.g. OTLP, maybe other open-source ones), memory_limiter, batch and queue processors enabled, traces and metrics pipelines configured and ready. The only thing the end user will need to do to begin running a Collector is specify exporter endpoint.

For now we can achieve similar end-user experience simply by providing a default config.yaml file in this repo or in Docker images we publish.

[pavolloffay]

For the moment we do offer every component to provide a default configuration, probably one missing part is to have default components registered by default.

Correct, each component at the moment provides a default configuration. However these components have to be explicitly enabled and registered to a pipeline in the config file. The current behavior is that the collector fails to start without configuration fine.

Jaeger OTEL collector by default enables a set of hardcoded components - Jaeger receiver, some processors and exporter (storage implementation).

So I was wondering if the upstream OTEL collector will do the same and how that would affect Jaeger. I assume the default config could be changed for every distribution.

[tigrannajaryan]

I think by default we need to enable the following components:

receivers:
  otlp:

processors:
  memory_limiter:
    check_interval: 5s
    limit_mib: 128
    spike_limit_mib: 64
    ballast_size_mib: 0
  batch:
  queued_retry:

extensions:
  health_check:
  zpages:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter,batch,queued_retry]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter,batch]

  extensions: [health_check,zpages]

Any of these components can be disabled by overriding the list of receivers, processors or extensions under the service setting.

[tigrannajaryan]

Do we need to enable any other components by default? The config above is not sufficient to start the Collector. At the minimum you need one exporter. I don't know if we want to have a default exporter. Where would it point to?

[tigrannajaryan]

I think it should be safe to hard-code this default config in the Collector, and do the merging with user-defined config somewhere in

opentelemetry-collector/service/service.go

Line 127 in e3df9f3

func FileLoaderConfigFactory(v *viper.Viper, factories config.Factories) (*configmodels.Config, error) {

viper.MergeInConfig() appears to be an easy wat to implement the merging.

We will need to make sure user-defined config overrides the default one on a key-by-key basis. I.e. this should be a sufficient user-defined config to enable an exporter to be used with default receiver and processor:

exporters:
  otlp:
    endpoint: server:1234

service:
  pipelines:
    traces:
      exporters: [otlp]
    metrics:
      exporters: [otlp]

[nilebox]

I don't know if we want to have a default exporter. Where would it point to?

You may enable logging exporter by default if you want to have a working pipeline out of the box, but it's only useful for quick start.