Skip to content

Commit

Permalink
Fix CAGRA OOM handling (rapidsai#167)
Browse files Browse the repository at this point in the history
Fixes handling OOM error during CAGRA index creation, that was introduced in rapidsai#131.

Authors:
  - Tamas Bela Feher (https://github.com/tfeher)

Approvers:
  - Corey J. Nolet (https://github.com/cjnolet)

URL: rapidsai/cuvs#167
  • Loading branch information
tfeher authored May 30, 2024
1 parent 13e0732 commit 66959b1
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 3 deletions.
9 changes: 6 additions & 3 deletions cpp/src/neighbors/detail/cagra/cagra_build.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -480,9 +480,12 @@ index<T, IdxT> build(
RAFT_LOG_DEBUG("Insufficient GPU memory to construct CAGRA index with dataset on GPU");
// We just add the graph. User is expected to update dataset separately (e.g allocating in
// managed memory).
index<T, IdxT> idx(res, params.metric);
idx.update_graph(res, raft::make_const_mdspan(cagra_graph.view()));
return idx;
} catch (raft::logic_error& e) {
// The memory error can also manifest as logic_error.
RAFT_LOG_DEBUG("Insufficient GPU memory to construct CAGRA index with dataset on GPU");
}
index<T, IdxT> idx(res, params.metric);
idx.update_graph(res, raft::make_const_mdspan(cagra_graph.view()));
return idx;
}
} // namespace cuvs::neighbors::cagra::detail
4 changes: 4 additions & 0 deletions cpp/src/neighbors/nn_descent.cu
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
#include <cuvs/neighbors/nn_descent.hpp>
#include <raft/core/logger.hpp>

using namespace raft;
namespace cuvs::neighbors::nn_descent {

/**
Expand Down Expand Up @@ -53,6 +54,9 @@ bool has_enough_device_memory(raft::resources const& res,
} catch (std::bad_alloc& e) {
RAFT_LOG_DEBUG("Insufficient memory for NN descent");
return false;
} catch (raft::logic_error& e) {
RAFT_LOG_DEBUG("Insufficient memory for NN descent (logic error)");
return false;
}
}

Expand Down

0 comments on commit 66959b1

Please sign in to comment.