Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ExecutionPlan::execute Sync #2434

Merged
merged 1 commit into from
May 4, 2022

Conversation

tustvold
Copy link
Contributor

@tustvold tustvold commented May 3, 2022

Which issue does this PR close?

Closes #2307

Rationale for this change

See ticket but the major benefits are:

What changes are included in this PR?

Makes ExecutionPlan::execute and by extension ExecutionPlan itself, sync

Are there any user-facing changes?

Yes, this is a breaking change to a fundamental trait.

@tustvold tustvold added the api change Changes the API exposed to users of the crate label May 3, 2022
@github-actions github-actions bot added ballista datafusion Changes in the datafusion crate labels May 3, 2022
@andygrove
Copy link
Member

I plan to review this PR carefully today

@tustvold tustvold marked this pull request as draft May 4, 2022 13:25
@tustvold
Copy link
Contributor Author

tustvold commented May 4, 2022

Oops meant to mark this as draft, #2428 needs to go in first

@andygrove
Copy link
Member

@tustvold This PR didn't take as long to review as I thought. It is really just the Ballista changes that were large and I will rely on the integration tests to confirm that everything is working as expected, so LGTM so far but needs rebasing/upmerging now before I can complete my review.

@tustvold tustvold marked this pull request as ready for review May 4, 2022 14:55
@tustvold
Copy link
Contributor Author

tustvold commented May 4, 2022

Rebased, this PR is pretty mechanical, as prior PRs laid the necessary groundwork for it

@@ -125,23 +122,8 @@ impl ExecutionPipeline {
// Construct the output streams
let output_count = proxied.output_partitioning().partition_count();
let outputs = (0..output_count)
.map(|x| {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👋

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ballista integration tests still pass with this change so LGTM. Thanks @tustvold!

@andygrove andygrove merged commit 807b7a5 into apache:master May 4, 2022
@alamb alamb changed the title Make ExecutionPlan Sync Make ExecutionPlan::execute Sync May 5, 2022
@alamb
Copy link
Contributor

alamb commented May 5, 2022

I agree this is a good change -- thanks @tustvold and @andygrove 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api change Changes the API exposed to users of the crate datafusion Changes in the datafusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make ExecutionPlan Sync
3 participants