Double url-encoding of special characters in key names

[Noratrieb]

Describe the bug

I have tried to use the object_store crate to upload files to my S3-compatible garage store.

As a key name, I tried using /mäggi.png.
But afterwards, I was not able to access this file.

In the logs for garage, it logged that it received this request: PUT /$bucketname/m%25C3%25A4ggi.txt.
This is a double encoding, which it decoded to /m%C3%A4ggi.txt and used that as the stored filename.

I have not tried reproducing the problem with another object store like Amazon S3 or MinIO, but I have tried uploading /mäggi.png using minio-client (mc cp), which worked fine, which makes me believe that garage behaves as expected. It is not impossible that this is a problem in garage though.

To Reproduce

Use object_store AWS client with put_object with a key name of /mäggi.png with the garage object store.

Expected behavior

The object store receives the key name as mäggi.png and stores that file name.

Additional context

It's encoded when constructing the path string.

• arrow-rs-object-store/src/aws/client.rs

  Line 465 in 8a7bc6e

  let url = self.config.path_url(path);

  • arrow-rs-object-store/src/aws/client.rs

    Line 213 in 8a7bc6e

    format!("{}/{}", self.bucket_endpoint, encode_path(path))

While it's then later encoded again since the path string is passed to the request.

[tustvold]

I trust you have read https://docs.rs/object_store/latest/object_store/path/struct.Path.html.

It sounds like you have used Path::from which has encoded the path, this is then used to create a file with path percent encoded characters. You should use Path::parse if you wish to use a path with non-ASCII characters

[Noratrieb]

I have not read that, it somehow did not occur to me that I should, probably since i never really used Path explicitly but only via a not very obvious into(), but that answered my question. This seems like quite surprising behavior to change the semantics of the string via From, but using Path::parse has fixed my problem and fixed the double-encoding. Thanks!

[tustvold]

Yeah, it's a bit of a historical quirk that'd now be quite difficult to change