Support file stats on nested fields #5917
asubiotto
started this conversation in
Feature Requests
Replies: 1 comment
-
|
We can store the stats for all element on a post-order walk of the dtype there each field
Then we would need to extend the the stats set footer with an optional/default=false bool Open question: How do we store list stats? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Currently, when writing file statistics, only top-level field stats are stored in the vortex file footer:
vortex/vortex-layout/src/layouts/file_stats.rs
Line 45 in 191228e
Pruning on nested fields does work at a chunk level I believe, so this is specifically about file level stats to allow pruning whole files where possible.
Beta Was this translation helpful? Give feedback.
All reactions