Proposal: Registry Support

[sincejune]

Is your feature request related to a problem? Please describe.
Opentelemetry-Collector can either be pull-based or push-based at this time.
For receivers that are pull-based (e.g. kafka and filelog receivers), we actually need a registry file for them so that Opentelemetry-Collector can continue consuming data from the last stop.

[pmm-sumo]

This capability is in progress for filelog receiver. Perhaps it could be generalised @djaglowski ? Related issue: open-telemetry/opentelemetry-collector-contrib#2947

[djaglowski]

Yes, filelogreceiver will make use of a file_storage extension, if configured as part of the service.

Any other component of the collector can do the same. You will need to enhance the component's Start method to check for a storage extension, and then make use of it. Here is where filelogreceiver checks for a storage extension.

[sincejune]

@djaglowski It looks great!
I can try to make a contribution to Kafka Receiver. But it looks like we need to move the storage definition into this repo as the first step. Thoughts?

[djaglowski]

@sincejune That is a good point. I believe there is a good chance the storage extension code may end up in this repo, but this will be up to the maintainers to decide. //cc @tigrannajaryan

[tigrannajaryan]

@sincejune That is a good point. I believe there is a good chance the storage extension code may end up in this repo, but this will be up to the maintainers to decide. //cc @tigrannajaryan

Let's give a bit time for storage extension to be polished and stabilized in the contrib and then we will move it to this repo. It should like happen after the GA to avoid delaying the GA further.

[sincejune]

I just realized that Kafka had already stored offset information itself so we don't have to add supports particularly for that.
Closing this as we already have filestorage.