Describe the bug
parquet page level skipping (page index pruning) panic's with evolved schemas
To Reproduce
Enable setting like
pub enable_page_index: bool, default = true
And then run test:
cargo test -p datafusion --lib -- evolved_schema_disjoint_schema_filter
Which fails
test physical_plan::file_format::parquet::tests::evolved_schema_disjoint_schema_filter ... FAILED
failures:
---- physical_plan::file_format::parquet::tests::evolved_schema_disjoint_schema_filter stdout ----
thread 'physical_plan::file_format::parquet::tests::evolved_schema_disjoint_schema_filter' panicked at 'index out of bounds: the len is 1 but the index is 1', /Users/alamb/.cargo/registry/src/github.com-1ecc6299db9ec823/parquet-31.0.0/src/file/metadata.rs:251:10
Expected behavior
Test should pass
Additional context
I found this while working to enable this feature by default
#5099