[Proposal] Remove distros references from READMEs of individual components

[yurishkuro]

Component(s)

No response

Is your feature request related to a problem? Please describe.

As example, #29795 performs unnecessary maintenance work to list which components are included in a particular vendor distribution.

• is there really a class of users who say "I need this specific component let me see who includes it"?
• the lists are inherently incomplete and therefore elevate certain vendor distros over others
• frankly, overall feels like a free marketing mechanism with a maintenance burden on contrib repo

Describe the solution you'd like

Remove references to vendor distros from the repository.

I am ok with keeping one page that simply lists all distros that people bothered to add to that page. It will be low maintenance.

But the content of the distros should be documented in their respective websites (can be easily auto-generated if using ocb).

Describe alternatives you've considered

No response

Additional context

No response

[atoulme]

I think we started discussing this with #30164

I'll try and answer from my personal perspective, and I'm happy to talk about this at a SIG meeting.

• is there really a class of users who say "I need this specific component let me see who includes it"?

Yes, from personal experience our users typically will look at contrib components and check if our distribution is listed.

• the lists are inherently incomplete and therefore elevate certain vendor distros over others

I don't mind if we remove vendor distributions, that said the requirements for the distributions are not aligned with vendors. The only requirements are: the distribution must be open source and the distribution link point to a publicly accessible repository.
Would you like to offer additional requirements such as the distribution must be hosted under the opentelemetry project?

That said, I'm not sure I understand this:

The lists are inherently incomplete

Do you mean here that the list miss some vendors? Or that vendors miss some of the components included in their distribution?

and therefore elevate certain vendor distros over others

So vendors with more complete lists and better documentation will be elevated above others? Or maybe a vendor with a more comprehensive distribution will get more traction with end users?

I sure hope that's not true! Our distribution has markedly less components than others, but we certainly try to make up for it with our easy to use installers and great documentation :) (sorry, free marketing doesn't just make itself)

• frankly, overall feels like a free marketing mechanism with a maintenance burden on contrib repo

We've certainly carried that burden and I've shouldered along the way, but it was a sweet journey.
First, we have made all of this part of mdatagen, so generating distribution data is easy.
Then, I've taken the task to surface codeowners per distribution, so we can have folks dedicated to distribution maintenance away from the current component codeowners. I created https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/distributions.yaml, and you can generate the distribution report and CODEOWNERS changes with make gendistributions and make gengithub respectively.
The work of making this all automated and maintainable is more or less done.

can be easily auto-generated if using ocb

Sounds like a nice enhancement idea, would you be open to filing an issue to add this feature?

[codeboten]

The original intent behind listing distros in the readme was to answer the questions of whether a component was in the core or the contrib distro. I agree that including vendor distributions has added unnecessary overhead and would be ok with keeping only the list of distros maintained by the project in the readme.

That being said, maybe the easier thing is to generate a page on opentelemetry.io that lists the components of each distro we support.

I don't have a strong preference here, but anecdotally, as a user, going to the component's readme tends to be my first choice for information, since it also includes the configuration I would need for that component.

[yurishkuro]

but anecdotally, as a user, going to the component's readme tends to be my first choice for information, since it also includes the configuration I would need for that component.

@codeboten And it's completely reasonable workflow (for config etc), but I have a hard time believing that you go there to decide which vendor distro to pick.

Yes, from personal experience our users typically will look at contrib components and check if our distribution is listed.

@atoulme This is a very different workflow. If your user already has your distro in mind, they can just as easily go to the distro docs and see which components are included.

Would you like to offer additional requirements such as the distribution must be hosted under the opentelemetry project?

I don't follow why this is even a question - are there planes to host multiple distros from contrib? Even if we did, the source of truth for what's included in the distro would be the distro's builder config, and if it's used to auto-gen the matrix in the readme I wouldn't have an issue with that. But the current mechanism, as I understand (automation or not) is the mapping must be maintained manually in each component, which makes no sense. It's like if Linux were tracking in its source code where it's being used.

[codeboten]

which vendor distro to pick.

@yurishkuro right, this is more "i just found out about this component, and i want to use it in an otel distro, which one includes it" kinda question i want to answer

[atoulme]

I don't follow why this is even a question - are there planes to host multiple distros from contrib?

I am sorry if I'm asking dumb questions. Thanks for bearing with me through it.
As a matter of fact, I have an open issue to offer that we create a PaaS distribution here: open-telemetry/opentelemetry-collector-releases#459

[atoulme]

I plan on following up on this today and open a PR to any other distribution that is not core and contrib. You raise additional concerns in this issue. I will follow up on them as well with additional issues.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[atoulme]

sorry, I didn't follow up on this. I'll add it to the SIG meeting after KubeCon EU to discuss.

[adrielp]

Personally, I like the idea of removing references to vendor distributions from the repo, and I'm the one who opened up the chore pull request linked in this issue. 👍🏼

Context: The reason I opened that pull request was because there was a Collector SIG a while back that ran distribution metrics and noticed our listed distro only had one component in it, which was unlikely (and incorrect). At the time, I was asked to make the updates listed in the pull request.

We're adding in (and removing) components regularly in our distro. For example, we recently added some of the k8s receivers to our distro out of a need, and immediately removed the logging component as it became deprecated in favor of the debug component. Updating each of the components in contrib every time we do this certainly adds overhead for us, and for you. I'm a fan of removing this overhead.

However, on the flip side, the specific component we have been contributing (which is still in development & not built into the contrib distro yet) made sense to have our distro listed on it because that's the only place it's currently built into and feature complete. It has taken time to contribute this component upstream because of the processes in place (not a knock on the processes, just a fact). Thus, if someone wants to use the component with all it's feature in a distro they can just deploy, it made sense (and I believe was the ask at the time) to have it listed in our distro through the metadata.

That may be more of an edge case, but I do think it's an important example.

In full transparency (as a distro maintainer) it absolutely is free marketing. But IMO, it's not without its rewards for both vendors and the project. It's been easy to search and filter who supports OTEL, and what they support. For example, I've looked at the components through the registry to make comparisons on ADOT vs OTEL Contrib distorts in the past, helping better inform antiquated enterprise architecture conversations. And I'm sure there's benefits (if albeit small and bespoke) from searchabililty on the internet and within git.

I say that to say, there are some benefits to having it listed, and there was context why I made that original pull request, but I'm totally in favor of removing that overhead and finding a better, more minimal, solution. I also generally recommend most folks just build their own distro with the base set of components they need anyway as this follows the security principle of least privilege, using renovate to auto bump the latest from contrib.

[yurishkuro]

@adrielp I personally don't mind acknowledging the source of the contribution, e.g. "Developed by , included in the circa 2023", as a static notice that doesn't need to change. For up-to-date reference of distros that host the components I would invert the responsibilities and have the distro maintainers publish a standard file (possibly generated by ocb) and the OTEL repo would only store a static reference to those files. The OTEL website could periodically pull those files and rebuild a search index.

[adrielp]

@yurishkuro - I like & can support both of those solutions!