Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support Retry for GpuBaseLimitExec #8315

Closed
revans2 opened this issue May 17, 2023 · 0 comments · Fixed by #9193
Closed

[FEA] Support Retry for GpuBaseLimitExec #8315

revans2 opened this issue May 17, 2023 · 0 comments · Fixed by #9193
Assignees
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@revans2
Copy link
Collaborator

revans2 commented May 17, 2023

Is your feature request related to a problem? Please describe.
GpuBaseLimitExec.sliceRDD is the basis for GpuGlobalLimitExec and GpuLocalLimitExec. The implementation is mostly deciding if a batch of data should be passed through unchanged, not at all, or sliced.

If the data is unchanged then there is no need for a retry because it is essentially a noop. If it is not passed through, then we never see the data, we just say in the iterator that there are no more batches. If we have to slice it, then we can put a retry block around the slice. But we cannot really support SplitAndRetry here. We are slicing the data into a subset of the input. If that does not work how do we expect to be able to split the input into multiple chunks that add up to the total size of the input before we slice it and throw away parts of it.

I guess if we had fragmentation..., but practically it is just not worth even trying here.

@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels May 17, 2023
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label May 24, 2023
@firestarman firestarman self-assigned this Sep 4, 2023
@sameerz sameerz removed the feature request New feature or request label Oct 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants