-
Couldn't load subscription status.
- Fork 1.7k
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As discussed in #1010, it would add a lot of flexibility to the planning to make it possible to use async methods. Thanks to #962, the statistics resolution can be delayed to the creation of the physical plan. Adding async to TableProvider.scan(), the method that converts the logical plan to the physical plan, will make it possible to have async statistics resolution.
Describe the solution you'd like
Make the trait method TableProvider.scan() async and propagate that change to the outer APIs (mostly ExecutionContext.create_physical_plan(&LogicalPlan))
Describe alternatives you've considered
- also add statistics to optimizers: that could be a next step but it is not required for now
- leave the planning sync and force
TableProviderimplementers to make their async methods sync: syncifying in Rust is pretty hard (cloud-reader-rs is an example of this) and it is a huge constraint for implementers
Additional context
This will also help the useage of the ObjectStore abstraction added in #811.