You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Attempting to create a literal value which is an array of structs causes a panic.
To Reproduce
DataFusion CLI v34.0.0
❯ set datafusion.execution.parquet.allow_single_file_parallelism=false;
0 rows in set. Query took 0.001 seconds.
❯ copy (values (make_array(struct('foo',1)))) to '/tmp/test.parquet';
thread 'main' panicked at /home/dev/.cargo/registry/src/index.crates.io-6f17d22bba15001f/datafusion-common-34.0.0/src/scalar.rs:524:74:
called `Result::unwrap()` on an `Err` value: Internal("Unsupported data type in hasher: Struct([Field { name: \"c0\", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: \"c1\", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }])")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Expected behavior
Should not result in panic. If there is some issue with how I have attempted to construct the array of struct, we could add a more descriptive error message.
Additional context
I came across this while writing tests related to #8551, but this issue does not appear related to parallel parquet writing specifically.
The line of code generating the error implies an internal constraint is violated if we reach it.
I run on main branch and does not reproduce the panic.
statement ok
set datafusion.execution.parquet.allow_single_file_parallelism=false;
query ?
copy (values (make_array(struct('foo',1)))) to '/tmp/test.parquet';
----
1
Describe the bug
Attempting to create a literal value which is an array of structs causes a panic.
To Reproduce
Expected behavior
Should not result in panic. If there is some issue with how I have attempted to construct the array of struct, we could add a more descriptive error message.
Additional context
I came across this while writing tests related to #8551, but this issue does not appear related to parallel parquet writing specifically.
The line of code generating the error implies an internal constraint is violated if we reach it.
https://github.com/apache/arrow-datafusion/blob/1dcdcd431187178d736cdd3a6c004204aa2faa14/datafusion/common/src/hash_utils.rs#L369-L374
The text was updated successfully, but these errors were encountered: