You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As discussed in #1010, it would add a lot of flexibility to the planning to make it possible to use async methods. Thanks to #962, the statistics resolution can be delayed to the creation of the physical plan. Adding async to TableProvider.scan(), the method that converts the logical plan to the physical plan, will make it possible to have async statistics resolution.
Describe the solution you'd like
Make the trait method TableProvider.scan() async and propagate that change to the outer APIs (mostly ExecutionContext.create_physical_plan(&LogicalPlan))
Describe alternatives you've considered
also add statistics to optimizers: that could be a next step but it is not required for now
leave the planning sync and force TableProvider implementers to make their async methods sync: syncifying in Rust is pretty hard (cloud-reader-rs is an example of this) and it is a huge constraint for implementers
Additional context
This will also help the useage of the ObjectStore abstraction added in #811.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As discussed in #1010, it would add a lot of flexibility to the planning to make it possible to use async methods. Thanks to #962, the statistics resolution can be delayed to the creation of the physical plan. Adding async to
TableProvider.scan()
, the method that converts the logical plan to the physical plan, will make it possible to have async statistics resolution.Describe the solution you'd like
Make the trait method
TableProvider.scan()
async and propagate that change to the outer APIs (mostlyExecutionContext.create_physical_plan(&LogicalPlan)
)Describe alternatives you've considered
TableProvider
implementers to make their async methods sync: syncifying in Rust is pretty hard (cloud-reader-rs is an example of this) and it is a huge constraint for implementersAdditional context
This will also help the useage of the
ObjectStore
abstraction added in #811.The text was updated successfully, but these errors were encountered: