Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Handle SplitAndRetryOOM for BatchedByKey goal in GpuCoalesceBatches #7778

Closed
Tracked by #8029
abellina opened this issue Feb 15, 2023 · 0 comments · Fixed by #8058
Closed
Tracked by #8029

[FEA] Handle SplitAndRetryOOM for BatchedByKey goal in GpuCoalesceBatches #7778

abellina opened this issue Feb 15, 2023 · 0 comments · Fixed by #8058
Assignees
@revans2
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@abellina
Copy link
Collaborator

For the BatchedByKey goal we are required to preserve that sets of keys stay within a batch (that no key is split between two batches). This fits with the split/merge approach for SplitAndRetryOOM and we should be able to produce smaller batches satisfying this requirement rather than OOMing.
@abellina abellina added feature request New feature or request ? - Needs Triage Need team to review and classify reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels Feb 15, 2023
@abellina abellina mentioned this issue Feb 15, 2023
Closed
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Feb 21, 2023
@revans2 revans2 mentioned this issue Feb 21, 2023
Closed
7 tasks
@abellina abellina mentioned this issue Mar 7, 2023
Merged
This was referenced Apr 4, 2023
Closed
Open
Merged
@sameerz sameerz removed the feature request New feature or request label Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@revans2 revans2

Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add retry to BatchByKeyIterator revans2/spark-rapids
4 participants
@abellina @revans2 @mattahrens @sameerz