S3 repository plugin: since Opensearch 2.9 unable to use region defined in repository

[ICHx]

In my observation, when endpoint is also defined, I can use the repository. When endpoint is not defined, opensearch cannot access the repo. Versions below opensearch 2.8 don't have this issue.

Also, the fields I mentioned is absent albeit effective. Should the doc be updated accordingly?

https://docs.opensearch.org/latest/api-reference/snapshots/create-repository/

Hi @hm2thr33,

I tried that I can specify endpoint and region in es repo settings and then it worked like it used to. No need editing opensearch.yml in that way. And in this way no need turning on cross-region access although it is desirable.

This was undocumented in Opensearch's doc but mentioned in Elasticsearch's.

              es_settings['endpoint'] = f'https://s3.{ es_settings.get("region", "us-west-2") }.amazonaws.com'
              es_repo_params = {'type': 's3', 'settings': es_settings}

              logger.info('S3: creating ES repo %s with %s',repo, es_repo_params)

              status:dict ={}
              try:
                api_resp = self.ES_API.snapshot.create_repository(repository=repo,
                                                                  body=es_repo_params)

Here is my implementation involved. Without the endpoint, error is observed.

---

(Unrelated to the fix)
I also patched the plugin to allow cross region access (available since 2.20.111)The error message becomes

Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.

While when using awscli --debug we can see the same message, opensearch s3 client is unable to handle redirection.

vagrant@07c09107d183 tests → curl 0:19200/_snapshot/test-repo -H 'Content-Type: application/json' -d '{"type": "s3", "settings": {"bucket": "vos-elk-local-build-h55jk7fq96osa0jk"}}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   843  100   765  100    78   2602    265 --:--:-- --:--:-- --:--:--  2867
{
  "error": {
    "root_cause": [
      {
        "type": "repository_verification_exception",
        "reason": "[test-repo] path  is not accessible on cluster-manager node"
      }
    ],
    "type": "repository_verification_exception",
    "reason": "[test-repo] path  is not accessible on cluster-manager node",
    "caused_by": {
      "type": "i_o_exception",
      "reason": "Unable to upload object [tests-N0zLqlXVS8msemkaoG8eEQ/master.dat] using a single upload",
      "caused_by": {
        "type": "s3_exception",
        "reason": "Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests. (Service: S3, Status Code: 307, Request ID: 4Q7KKXR1M6F9M5AB, Extended Request ID: Y6dL/VC6wLPTOY4aEt6oG5/vEQotWxgS6fwvMmLk9WQZmcBPNt1Hiu9SiztIqE8pzrx2+MrcEuYprPdTLTPQGLCAwN0DGNDh)"
      }
    }
  },
  "status": 500
}

I searched and retried in awscli,

❯ aws s3 ls --region us-east-1 [s3://bucket-sndn51jcaxn3ot6j/](s3://bucket-sndn51jcaxn3ot6j/) --debug

2025-07-22 17:28:40,208 - MainThread - botocore.parsers - DEBUG - Response body:
b'<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><Endpoint>bucket-sndn51jcaxn3ot6j.s3-us-west-1.amazonaws.com</Endpoint><Bucket>bucket-sndn51jcaxn3ot6j</Bucket><RequestId>BQN93AHFQJHZRTZY</RequestId><HostId>iNduI12CIQOjWg4JFBVvk6GlKFfe7PDJa83qT/4zeGy/Hfr90AbgeEXWLDJregT6MRd2Sjrxsrvgkze+RQlyBhEOL45kcLKcbAOX11KUnek=</HostId></Error>'

In awscli it's just a debug message and was able to redirect itself, but not opensearch

The bucket is in us-west-1

aws s3 ls --region us-west-1 [s3://bucket-sndn51jcaxn3ot6j/](s3://bucket-sndn51jcaxn3ot6j/) --debug

Using the same region in cli do not have that message

Originally posted by @ICHx in #9265

[andrross]

Catch All Triage - 1 2 3

@ICHx Looks like the documentation needs to be updated. We would welcome a PR if you're interested!

[ICHx]

Thx for the update Andrross, sure I can work on a document change. However, this looks like a regression too