Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract logical plans in LogicalPlan as independent struct: TableScan #1290

Merged
merged 1 commit into from
Nov 16, 2021

Conversation

xudong963
Copy link
Member

@xudong963 xudong963 commented Nov 12, 2021

Which issue does this PR close?

Related to #1228

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added ballista datafusion Changes in the datafusion crate labels Nov 12, 2021
@xudong963
Copy link
Member Author

xudong963 commented Nov 12, 2021

I extracted TableScan as an example, PTAL @alamb @houqp

I have two questions:

  1. How about extracting every plan to a single file.
  2. Extract All plans at once as one ticket or divide them into multiple tickets? I think dividing into multiple tickets will be good to review. (maybe every ticket is less than 200 lines.)

@liukun4515
Copy link
Contributor

liukun4515 commented Nov 13, 2021

@xudong963, If you split them into multiple tasks, you can list each subtask, may be I can help you to do someone.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution @xudong963 - this PR looks exactly like what is discussed on #1228 👍

I think this looks like an improvement to me. It will be a fairly large, but mechanical change to the codebase but I do think it will help over time

What do you think about this change @jimexist @houqp @rdettai and @Dandandan

@xudong963
Copy link
Member Author

Thank you for the contribution @xudong963 - this PR looks exactly like what is discussed on #1228 👍

I think this looks like an improvement to me. It will be a fairly large, but mechanical change to the codebase but I do think it will help over time

My pleasure! I believe after finishing this, it'll bring help for subqueries implementation.

@xudong963
Copy link
Member Author

The ticket can be merged? Then we can push related tickets🎯

@alamb
Copy link
Contributor

alamb commented Nov 15, 2021

The ticket can be merged? Then we can push related tickets🎯

Any concerns @Dandandan / @andygrove / @jimexist / @rdettai ?

Copy link
Contributor

@rdettai rdettai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your work @xudong963. I am not sure I fully understand the big picture of the change though. It seems from #1228 that this is part of adding subqueries, but I would find it clearer if there was an umbrella issue with bullet points that list to steps/issues.

I have also raised some questions here: #1228 (comment)

@alamb
Copy link
Contributor

alamb commented Nov 16, 2021

There is some good discussion on #1228 (comment) -- to be clear, in my mind, the purpose of this refactoring is to make LogicalPlan nodes easier to work with longer term.

Thanks @xudong963 for leading the charge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants