Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support spill+retry for GpuExpandExec #8461

Closed
revans2 opened this issue May 31, 2023 · 1 comment
Closed

[FEA] Support spill+retry for GpuExpandExec #8461

revans2 opened this issue May 31, 2023 · 1 comment
Assignees
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@revans2
Copy link
Collaborator

revans2 commented May 31, 2023

Is your feature request related to a problem? Please describe.
GpuExpandExec sadly keeps a ColumnarBatch around between calls. Not a SpillableGpuColumnarBatch or anything like that.

We should update it to use a SpillableColumnarBatch. Support tiered project so that we can get the retry easily.

We also have a lot of code in there for handling columnaEval returning a null, but it should never return a null it would return a GpuScalar, so we should rip out that code. It was a premature optimization that also could cache data in memory that is not spillable. So two wrongs are making a right here.

This really only runs right before a hash aggregate, but with the retry logic in hash aggregate it is kind of scary and I think we should fix this sooner than later.

@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels May 31, 2023
@revans2 revans2 self-assigned this May 31, 2023
@revans2
Copy link
Collaborator Author

revans2 commented Jun 5, 2023

This was fixed by #8467

@revans2 revans2 closed this as completed Jun 5, 2023
@revans2 revans2 removed the ? - Needs Triage Need team to review and classify label Jun 5, 2023
@sameerz sameerz removed the feature request New feature or request label Aug 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Development

No branches or pull requests

2 participants