Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-49501][SQL] Fix double-escaping of table location
### What changes were proposed in this pull request? Change the implementation of `createTable` to avoid escaping of special chars in `UnresolvedTableSpec.location`. This field should contain the original user-provided `path` option and not the URI that is constructed by the `buildStorageFormatFromOptions()` call. In addition this commit extends `SparkFunSuite` and `SQLTestUtils` to allow creating temporary directories with a custom prefix. This can be used to create temporary directories with special chars. ### Why are the changes needed? Bug fix. The following code would result in the creation of a table that is stored in `/tmp/test%table` instead of `/tmp/test table`: ``` spark.catalog.createTable("testTable", source = "parquet", schema = new StructType().add("id", "int"), description = "", options = Map("path" -> "/tmp/test table")) ``` Note that this was not consistent with the SQL API, e.g. `create table testTable(id int) using parquet location '/tmp/test table'` ### Does this PR introduce _any_ user-facing change? Yes. The previous behaviour would result in table path be escaped. After this change the path will not be escaped. ### How was this patch tested? Updated existing test in `CatalogSuite`. ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#47976 from cstavr/location-double-escaping. Authored-by: Christos Stavrakakis <christos.stavrakakis@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
- Loading branch information