Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Add retry for host memory allocations for output file formats #8889

Open
1 task
revans2 opened this issue Jul 31, 2023 · 0 comments
Open
1 task

[FEA] Add retry for host memory allocations for output file formats #8889

revans2 opened this issue Jul 31, 2023 · 0 comments
Assignees
@gerashegalov
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin task Work required that improves the product but is not user facing

Comments

@revans2
Copy link
Collaborator

revans2 commented Jul 31, 2023 •
edited by gerashegalov
Loading

Is your feature request related to a problem? Please describe.
The output file formats tend to use host memory buffers in a fairly predictable way. We cache the data on the CPU until the batch is done being written and then we copy it out to the output stream or drop it.

#9862 should let limit the total memory used when allocating the data. But we need to make each of the buffers spillable as they are handed to us and add in retry as needed. After that it should just work how we want it to.

Tasks

Tasks
Preview Give feedback
@revans2 revans2 added ? - Needs Triage Need team to review and classify task Work required that improves the product but is not user facing reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels Jul 31, 2023
@revans2 revans2 mentioned this issue Jul 31, 2023
Open
30 tasks
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Aug 8, 2023
@gerashegalov gerashegalov self-assigned this Aug 28, 2023
@gerashegalov gerashegalov mentioned this issue Aug 28, 2023
Merged
3 tasks
gerashegalov added a commit to gerashegalov/cudf that referenced this issue Aug 29, 2023
@gerashegalov
Use HostMemoryAllocator in jni::allocate_host_buffer
a761260 
Fixes rapidsai#13940
Contributes to NVIDIA/spark-rapids#8889

- Pass an explicit host memory allocator to  `jni::allocate_host_buffer`
- Consistently check for errors from NewGlobalRef
- Consistently guard against DelteteGlobalRef on a null

Signed-off-by: Gera Shegalov <[email protected]>
gerashegalov added a commit to gerashegalov/cudf that referenced this issue Aug 30, 2023
@gerashegalov
Use HostMemoryAllocator in jni::allocate_host_buffer
8e97f15 
Fixes rapidsai#13940

Contributes to NVIDIA/spark-rapids#8889

- Pass an explicit host memory allocator to  `jni::allocate_host_buffer`
- Consistently check for errors from NewGlobalRef
- Consistently guard against DelteteGlobalRef on a null

Signed-off-by: Gera Shegalov <[email protected]>
rapids-bot bot pushed a commit to rapidsai/cudf that referenced this issue Aug 30, 2023
@gerashegalov
Use HostMemoryAllocator in jni::allocate_host_buffer (#13975)
7b9f4a1 
Fixes #13940 
Contributes to NVIDIA/spark-rapids#8889

- Pass an explicit host memory allocator to  `jni::allocate_host_buffer`
- Consistently check for errors from NewGlobalRef
- Consistently guard against DelteteGlobalRef on a null

Authors:
  - Gera Shegalov (https://github.com/gerashegalov)

Approvers:
  - https://github.com/nvdbaranec
  - Jason Lowe (https://github.com/jlowe)

URL: #13975
@revans2 revans2 changed the title [FEA] Limit host memory usage for output file formats [FEA] Add retry for host memory allocations for output file formats Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerashegalov gerashegalov

Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin task Work required that improves the product but is not user facing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gerashegalov @revans2 @mattahrens