Layout Reader V2

[gatesn]

We have lots of things to learn from V1 and lots of things to improve. I'm going to start writing some of these down...

• Computing splits up-front can be slow, particularly for highly selective scans.
• Creating a task per split with Stream::buffered forces us to choose a split concurrency, this can artificially limit I/O requests and progress of the scan.
• The filter result should be able to run ahead, since holding the result in memory is quite small. TBD how to handle columns that appear in both filter + projection.
• It would be nice to perform some sort of CSE between filter and projection arrays (i.e. do not decompress FSST strings twice)
• I think we should have ~ncores partitions of a file, rather than ~natural splits of the layout tree. These reduces task overhead, and allows for more efficient compute over larger arrays since they're not artificially split by other columns.
• It should be easier to support custom pruning expressions such as bloom filters.
• It should be possible to push-down aggregation functions, including pruning with sketches.
• It should be easier to implement inverted index layouts for text search.
• LayoutReaders should wrap up into the new Scan API
• Is there some probability based weighting for I/O requests? Based on how often selectivity of a filter becomes zero for example?
• The concurrency of the scan and I/O should be limited by some actually limited resource, for example how much resident memory do we want to dedicate to pre-fetched buffers. Rather than concurrency being hard-coded.