S3 bucket name automatically appended to URL path

[niccolomineo]

An S3 file URL is expected to be as follows:

https://<bucket_name>.s3.amazonaws.com/public/media/<file_name>

The problem is django-storages appears to automatically append the bucket name to the endpoint URL, which therefore leads to having:

https://<bucket_name>.s3.amazonaws.com/<bucket_name>/public/media/<file_name>

These are the init'ed Django settings:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_STORAGE_BUCKET_NAME=<bucket_name>
AWS_S3_ENDPOINT_URL="https://<bucket_name>.s3.amazonaws.com"
AWS_LOCATION="public/media"
AWS_QUERYSTRING_AUTH=True
AWS_S3_FILE_OVERWRITE=True

Any ideas on how to prevent this behavior cleanly?

[shukurillo0516]

Try to set AWS_S3_CUSTOM_DOMAIN in your settings

[niccolomineo]

AWS_S3_CUSTOM_DOMAIN

But then I would lose the query strings.

[shukurillo0516]

If you mean AWS_QUERYSTRING_AUTH you shouldn't lose it.

[niccolomineo]

If you mean AWS_QUERYSTRING_AUTH you shouldn't lose it.

I do lose it unfortunately, there is an open thread on this. If there is not other way around the bucket name thing but use AWS_S3_CUSTOM_DOMAIN, I'll resort to one of the hacks suggested in that Github issue to work around the query strings bug.

[niccolomineo]

I ended up omitting the AWS_S3_ENDPOINT_URL altogether in production, only using it for the local MinIO instance.

[amoralesc]

Hey, pretty sure this has to do with the AWS_S3_ADDRESSING_STYLE variable. Setting this variable to virtual, and setting your AWS_S3_ENDPOINT_URL to https://s3.amazonaws.com will automatically prepend your bucket name to the endpoint and not add it to the URL path.

[niccolomineo]

Hey, pretty sure this has to do with the AWS_S3_ADDRESSING_STYLE variable. Setting this variable to virtual, and setting your AWS_S3_ENDPOINT_URL to https://s3.amazonaws.com will automatically prepend your bucket name to the endpoint and not add it to the URL path.

I did see that but thanks for highlighting it, maybe someone who really need to set AWS_S3_ENDPOINT_URL explicitly will benefit from your suggestion. I managed to skip that step entirely, as it seems to be how django-storages works out of the box.