[PERF] Make IO accesses parallel

[sanketkedia]

Description of changes

Summarize the changes made by this PR.

• Improvements & Bug fixes
  • Getting version of a point from the versions map blockfile was sequential. Made it parallel I/O
  • Similarly the final hydration of results in projection was sequential that is converted to parallel
• New functionality
  • ...

Test plan

How are these changes tested?

• Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

None

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

[sanketkedia]

• [PERF] Make IO accesses parallel #4988 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[propel-code-bot]

Parallelize IO in Projection and Spann Index for Performance Improvements

This PR refactors key IO operations in the projection operator and Spann index reader to use parallel async patterns instead of sequential execution. The main changes include making version lookups from the versions map blockfile and hydration of projection results fully parallel, leading to potential performance improvements in high-IO workloads.

Key Changes:
• Refactored hydration of records in rust/worker/src/execution/operators/projection.rs to use try_join_all for parallel async processing.
• Modified version fetch in rust/index/src/spann/types.rs, changing sequential fetching of versions for doc_offset_ids to a parallel future::try_join_all aggregation.
• Introduced or refactored helper methods to enable parallelism while maintaining correct record and version matching.
• Improved handling of empty input scenarios for projection execution.
• Added use of futures::future in both modules and related imports.

Affected Areas:
• Projection operator logic (hydration of records)
• Spann index reader fetch_posting_list and version map access
• Async IO patterns throughout relevant code

This summary was automatically generated by @propel-code-bot