[RFC] OTel collector modules

[jsoriano]

This change adds a new RFC that attempts to summarize the previous discussions around templates and the options that at the moment look more active.

The main output I would expect from the acceptance of this RFC is to reach an agreement on the overall implementation approach. There are still some open questions mentioned that we might let open by now.

So far there have been several attempts to introduce some kind of configuration templating in the OTel collector, and there seems to be some controversy about the steps forward. With this RFC we are proposing to limit the scope to the use case of sharing reusable configurations to collect signals from well-known services and applications. And with this scope in mind, consider use cases where this feature could be useful, as with autodiscovery. We also propose to call this feature "modules".

I skip in-depth technical details, but reference POCs where some of the discussed approaches are implemented.

I would like to thank @djaglowski for all his efforts driving previous discussions and experimentation. And @evan-bradley, @mx-psi, @rogercoll and others for their comments. It would be great if you could take a look to this RFC.

Related issues and PRs

• Simplify Collector Configuration for Users opentelemetry-collector-contrib#36116
• Template provider #8372
• Add template converter #8507
• Recursive unmarshaling of components #8940
• POC of integration receiver and processor elastic/opentelemetry-collector-components#96

cc @rogercoll @ChrsMark @andrzej-stencel @mlunadia

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.58%. Comparing base (d39dd7a) to head (0db8c96).

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #11631   +/-   ##
=======================================
  Coverage   91.58%   91.58%           
=======================================
  Files         440      440           
  Lines       23763    23763           
=======================================
  Hits        21764    21764           
  Misses       1627     1627           
  Partials      372      372           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.