-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Closed
Labels
enhancementAny new improvement worthy of a entry in the changelogAny new improvement worthy of a entry in the changelogparquetChanges to the parquet crateChanges to the parquet crate
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
See
arrow-rs/parquet/src/file/metadata.rs
Lines 888 to 889 in 7ba36b0
| // TODO: calc the order for all pages in this column | |
| boundary_order: BoundaryOrder, |
From parquet thrift:
/**
* Stores whether both min_values and max_values are ordered and if so, in
* which direction. This allows readers to perform binary searches in both
* lists. Readers cannot assume that max_values[i] <= min_values[i+1], even
* if the lists are ordered.
*/
4: required BoundaryOrder boundary_order
Describe the solution you'd like
Be able to set this boundary order when writing parquet files
Describe alternatives you've considered
Additional context
Some additional reading/discussion;
https://github.com/apache/parquet-format/blob/master/PageIndex.md
- Need to ensure for special types like Float16 where sort order differs from its physical type representation that this is accounted for & tested
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementAny new improvement worthy of a entry in the changelogAny new improvement worthy of a entry in the changelogparquetChanges to the parquet crateChanges to the parquet crate