OpenTelemetry Protocol with Apache Arrow Exporter initial skeleton #30619
Conversation
…tor-contrib into jmacd/otelarrow_skel_exporter
…tor-contrib into jmacd/otelarrow_skel_exporter
…tor-contrib into jmacd/otelarrow_skel_exporter
…tor-contrib into jmacd/otelarrow_skel_exporter
|
|
||
| require ( | ||
| github.com/open-telemetry/otel-arrow v0.16.0 | ||
| github.com/open-telemetry/otel-arrow/collector v0.16.0 |
There was a problem hiding this comment.
these dependencies will eventually go away once the implementation is complete in this repo correct?
There was a problem hiding this comment.
The otel-arrow module contains a large piece of code for translating between Arrow and pdata OTLP. This library probably has uses outside of the Collector and I think it should remain separate, at least until the Arrow proto definitions are accepted formally as OTLP protocol. There are OTel-Go SDK reasons we might use that library outside of collector components, probably.
The otel-arrow/collector module contains a number of components that were useful for us during development of the primary exporter/receiver, and we will want to keep them. They could each move into the -contrib repository, but that will take some additional effort, so I expect the repository itself to remain.
In addition to those statements, there are three packages for which an answer is needed, these being shared by the exporter and receiver and things I would recommend for the core collector itself.
The otel-arrow/collector/sharedcomponent package is a copy of the one from the core repo's internal/ package. I'm not sure why the core repo won't export that functionality, so I replicated it.
The otel-arrow/collector/netstats package is a set of very useful metrics that the Arrow components use, but which are applicable to all components and would also useful for the automatic benchmark harness.
The otel-arrow/collector/compression package is an implementation that supports multiple configurable Zstd compression levels on a per-exporter component basis. See mostynb/go-grpc-compression#24.
I suspect you will agree that all three of these packages belong in either -collector or -collector-contrib.
…pen-telemetry#30619) First PR to introduce the OpenTelemetry Protocol with Apache Arrow exporter. From the upstream repository: https://github.com/open-telemetry/otel-arrow/blob/main/collector/exporter/otelarrowexporter **Link to tracking Issue:** open-telemetry#26491 This is a skeleton PR, therefore only the skeleton contains tests. Compared with the upstream repository, the `factory_test.go` and `config_test.go` files have been kept, the implementation tests in `otelarrow_test.go` were excluded in this PR. New README, [copied from the upstream repository](https://github.com/open-telemetry/otel-arrow/blob/main/collector/exporter/otelarrowexporter/README.md).
…30766) First PR to introduce the OpenTelemetry Protocol with Apache Arrow receiver. From the upstream repository: https://github.com/open-telemetry/otel-arrow/blob/main/collector/receiver/otelarrowreceiver. Similar to #30619 for the corresponding receiver. **Link to tracking Issue:** #26491 **Testing:** This is a skeleton PR, therefore only the skeleton contains tests. Compared with the upstream repository, the factory_test.go and config_test.go files have been kept, the implementation tests in otelarrow_test.go were excluded in this PR. **Documentation:** New README, [copied from the upstream repository](https://github.com/open-telemetry/otel-arrow/blob/main/collector/receiver/otelarrowreceiver/README.md). --------- Signed-off-by: Joshua MacDonald <josh.macdonald@gmail.com>
Description: First PR to introduce the OpenTelemetry Protocol with Apache Arrow exporter. From the upstream repository: https://github.com/open-telemetry/otel-arrow/blob/main/collector/exporter/otelarrowexporter
Link to tracking Issue: #26491
Testing:
This is a skeleton PR, therefore only the skeleton contains tests. Compared with the upstream repository, the
factory_test.goandconfig_test.gofiles have been kept, the implementation tests inotelarrow_test.gowere excluded in this PR.Documentation: New README, copied from the upstream repository.