Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ballista ShuffleReaderExec should be able to read from multiple locations per partition #540

Closed
andygrove opened this issue Jun 11, 2021 · 0 comments · Fixed by #541
Closed

Ballista ShuffleReaderExec should be able to read from multiple locations per partition #540

andygrove opened this issue Jun 11, 2021 · 0 comments · Fixed by #541
Assignees
@andygrove
Labels
enhancement New feature or request

Comments

@andygrove
Copy link
Member

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As a step towards implementing true shuffle, we need ShuffleReaderExec to be able to read from multiple locations per partition instead of from a single location.

Describe the solution you'd like
ShuffleReaderExec should accept Vec<Vec<PartitionLocation>> instead of Vec<PartitionLocation>.

Describe alternatives you've considered
None

Additional context
None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andygrove andygrove

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ShuffleReaderExec now supports multiple locations per partition andygrove/datafusion
1 participant
@andygrove