[BUG] Snapshot repositories don't work with S3 compatible storage in 3.0.0 #18240
Description
Describe the bug
Repository S3 plugin 3.0.0 upgrade AWS SDK version to 2.30.31
Unfortunately, there is incompatibilities with some S3-like storages, like Outscale (see https://docs.outscale.com/en/userguide/AWS-SDK-and-CLI-Compatibility-Warning.html).
Error when testing S3 repository :
"Caused by: software.amazon.awssdk.services.s3.model.S3Exception: trailing checksum is not supported (Service: S3, Status Code: 400, Request ID: 8d0e880c5c4786c2ed08, Extended Request ID: 8d0e880c5c4786c2ed08) (SDK Attempt Count: 1)"
The workaround provided (disable checksum calculation with env vars or java properties) doesn't work, because in that case no checksum is sent, and Outscale expect MD5 checksum.
Error with checksum disabled :
"Caused by: software.amazon.awssdk.services.s3.model.S3Exception: The Content-MD5 you specified did not match what we received. (Service: S3, Status Code: 400, Request ID: 6fc31e21444eb5fc1a6b, Extended Requ
est ID: 6fc31e21444eb5fc1a6b) (SDK Attempt Count: 1)"
To keep Opensearch S3 repository plugin compatible with all S3-like storages, you can :
- revert back AWS SDK dependency version to <= 2.29.52
- implement a workaround like S3 default integrity protection change - version `2.30.0` aws/aws-sdk-java-v2#5802 (comment)
It may be the same with others S3-like object storages, wich aren't compatible yet with new aws S3 integrity check
Related component
Plugins
To Reproduce
Try to configure a S3 repository, with Outscale Object Storage as backend
See the error in logs
Expected behavior
Opensearch should work with any S3-like storage
Additional Details
Plugins
repository-s3
Host/Environment (please complete the following information):
- OS: Debian
- Version 11