-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[exporter/awss3] Add compression option (27872) #31622
[exporter/awss3] Add compression option (27872) #31622
Conversation
Please take a look at the failing build. Do you expect the only compression will be .gz only? Could we have multiple types of compression? |
Seems like there is some port conflict for opencensusreceiver in CI. Not sure but maybe parallel builds?
Good question, the issue only mentioned gzip and it is most used. Only other I can think of is zip compression via |
sorry for the leading question, I meant we can use the configcompression.Type type: https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configcompression/compressiontype.go I will look at the build and file an issue. |
5f44a9e
to
7506b9d
Compare
Updated code to reuse the
Thanks! 😄 |
…contrib into awss3-exporter-compression
5e82ce6
to
ba202ad
Compare
ba202ad
to
5ca3d56
Compare
**Description:** Add `compression` option to compress files using `compress/gzip` library before uploading to S3. **Link to tracking Issue:** Fixes open-telemetry#27872 **Testing:** Sent n number of traces through the S3 exporter using k6 to compare sizes. Used Minio as the S3 backend. | Marshaler | Compression | k6 Requests | k6 Data Sent | S3 Objects | S3 Total Size | | --- | --- | --- | --- | --- | --- | | otlp_json | No | 101 | 118 KB | 101 | 36 KB | | otlp_proto | No | 101 | 118 KB | 101 | 11 KB | | otlp_json | Yes | 101 | 118 KB | 101 | 21 KB | | otlp_proto | Yes | 101 | 118 KB | 101 | 9.9 KB | Additionally, new unit test to check file name. **Documentation:** - Updated README.md file
**Description:** Add `compression` option to compress files using `compress/gzip` library before uploading to S3. **Link to tracking Issue:** Fixes open-telemetry#27872 **Testing:** Sent n number of traces through the S3 exporter using k6 to compare sizes. Used Minio as the S3 backend. | Marshaler | Compression | k6 Requests | k6 Data Sent | S3 Objects | S3 Total Size | | --- | --- | --- | --- | --- | --- | | otlp_json | No | 101 | 118 KB | 101 | 36 KB | | otlp_proto | No | 101 | 118 KB | 101 | 11 KB | | otlp_json | Yes | 101 | 118 KB | 101 | 21 KB | | otlp_proto | Yes | 101 | 118 KB | 101 | 9.9 KB | Additionally, new unit test to check file name. **Documentation:** - Updated README.md file
Description:
Add
compression
option to compress files usingcompress/gzip
library before uploading to S3.Link to tracking Issue:
Fixes #27872
Testing:
Sent n number of traces through the S3 exporter using k6 to compare sizes. Used Minio as the S3 backend.
Additionally, new unit test to check file name.
Documentation: