OTLP exporter should be packaged separately per protocol

[codeboten]

Currently the opentelemetry-exporter-otlp only supports protobufs over gRPC. Once there's support for protobuf over http and json over http, we should have a separate package for each to ensure the dependencies can be reduced to only the bare minimum for each of those packages.

My proposal is to name the package based on the encoding and protocol:
opentelemetry-exporter-otlp-proto-grpc
opentelemetry-exporter-otlp-proto-http
opentelemetry-exporter-otlp-json-http

[codeboten]

for backwards compatibility and convenience, the opentelemetry-exporter-otlp package could include all available protocols

[srikanthccv]

Do we want to have support for Protobuf/HTTP and JSON/HTTP in the opentelemetry-exporter-otlp and then split it up into individual packages while keep maintaining original package opentelemetry-exporter-otlp ?

[lzchen]

@lonewolf3739
I'm not sure I understand your suggestion. What do you mean by supporting Protobuf/HTTP and JSON/HTTP in one package and then splitting into individual packages?

[codeboten]

My vote would be to split the packages as mentioned in the description, and then have opentelemetry-exporter-otlp just be a meta package that could pull the other 3 packages

[srikanthccv]

@lonewolf3739
I'm not sure I understand your suggestion. What do you mean by supporting Protobuf/HTTP and JSON/HTTP in one package and then splitting into individual packages?

By one package I mean as in other exporters such as Jaerger(Thrift/UDP, Proto/gRPC) and Zipkin(...) and making separate package for each encoding/protocol. I am having difficulty putting my thoughts into words. I mean something similar to what is described here https://packaging.python.org/guides/packaging-namespace-packages/

[lzchen]

@lonewolf3739
Do you just mean using namespace packages so that each protocol could still live under the opentelemetry-exporter-otlp namespace but can be individually installed as separate packges?

opentelemetry-exporter-otlp
   opentelemetry-exporter-otlp-proto-grpc
       ...
   opentelemetry-exporter-otlp-proto-http
       ...
   opentelemetry-exporter-otlp-json-http
      ...

[srikanthccv]

Yes, I was wondering if it is possible to create a opentelemetry-exporter-otlp with all of them included at the top level directory?

[lzchen]

@lonewolf3739
I think that is fine. The goal is just to have different installable packages for each protocol. However we choose to organize it is a separate issue.

[lzchen]

@codeboten
On a related note, do we want to have the same architecture for zipkin and jaeger for their respective protocols as well?

[codeboten]

@lzchen yes, we should follow the same path there

[owais]

My vote would be to split the packages as mentioned in the description, and then have opentelemetry-exporter-otlp just be a meta package that could pull the other 3 packages

Can we have this but with namespacing have the same import paths as today? If not, can we change the import paths today so when we break up the package into multiple ones, import paths don't change? If we can do this then this is not required for GA as packages can be broken up later without any interface or import path changes. When we break down the packages, users should be able to just upgrade opentelemetry-exporter-otlp to latest version and everything should just work.

[owais]

... Most users wouldn't even notice the change and the amount of effort required to get this done is a lot less than actually splitting the packages so it should help with getting to GA/RC2 a bit faster.

[lzchen]

@owais
Maybe I'm not understanding properly, but how can this be not a breaking change? The one package will be split from opentelemetry-exporter-otlp to opentelemetry-exporter-otlp-<protocol>, so they must change what library they install.

I agree with the namespacing, it provides a good user experience, but we would have to do everything before 1.0.0 due to the issue mentioned above.

[owais]

@codeboten suggested above that after the split opentelemetry-exporter-otlp can just be a meta-package that doesn't ship any code but only pulls in all protocol specific packages for OTLP exporter. So it'll have opentelemetry-exporter-otlp-protobuf and opentelemetry-exporter-http as dependencies. If we ship the package today as is and once we decide to split it post 1.0, we can turn this into a meta-package so when users upgrade, both http and protobuf/grpc packages get installed automatically. With imports paths not changing because of namespacing, such an upgrade will not break anything. New packages will be automatically installed on upgrade and imports paths/code won't have any changes.

[lzchen]

@owais
Doesn't that defeat the purpose of why we are packaging them separately in the first place if we are pulling in all packages as dependencies? We don't want users to have to take dependencies on specific protocols that they don't need right?

[owais]

Right. I was just thinking how/if we can delay work and get to GA faster by shipping this change post 1.0 while not breaking interfaces or requiring users to install/remove packages to keep things working but if having "lean" packages is a requirement for 1.0 then sure it won't work. How critical is it for the packages to be lean in 1.0? Can we live with shipping a big package in 1.0 and "fixing" it later?

This is not a big deal or concern for me. I was just going through to-do items and thinking if we can delay any of the work and get to GA faster.