You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Memory utilization continuously grows while running some of the C++ benchmarks, leading to OOM
Steps/Code to reproduce bug
make .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -GNinja
ninja -j 40 install
bench/prims_benchmark
Output
...
DistanceCosineF/10/manual_time 25.6 ms 25.7 ms 27
DistanceCosineF/11/manual_time 50.0 ms 50.1 ms 14
DistanceCosineF/12/manual_time 3.63 ms 3.65 ms 193
terminate called after throwing an instance of 'rmm::out_of_memory'
what(): std::bad_alloc: out_of_memory: CUDA error at: /opt/conda/envs/rapids/include/rmm/mr/device/cuda_memory_resource.hpp:70: cudaErrorMemoryAllocation out of memory
Aborted (core dumped)
Might be related to particular benchmarks. The following lead to OOM:
Describe the bug
Memory utilization continuously grows while running some of the C++ benchmarks, leading to OOM
Steps/Code to reproduce bug
Output
Might be related to particular benchmarks. The following lead to OOM:
bench/prims_benchmark --benchmark_filter=Gram*
bench/prims_benchmark --benchmark_filter=Distance*
Expected behavior
Memory consumption within bounds.
Environment details (please complete the following information):
rapidsai/rapidsai-core-dev-nightly:22.02-cuda11.5-devel-ubuntu20.04-py3.8
The text was updated successfully, but these errors were encountered: