[ENH] Do not create temporary device memory pools in IVF-Flat, IVF-PQ and balanced k-means #1167
Labels
0 - Backlog
In queue waiting for assignment
cpp
FAISS
improvement
Improvement / enhancement to an existing function
These memory pools can currently exist concurrently with each other or with a user-defined memory pool, creating problems of both performance and memory limits/control. It should instead be the responsibility of the user to create an appropriate memory pool, and the algorithms should use the resources attached to the raft handle.
Places where memory pools are currently created:
cluster::detail::build_hierarchical
ivf_flat::detail::search
ivf_pq::detail::extend
ivf_pq::detail::build
ivf_pq::detail::search
Typical code used to create these pools:
raft/cpp/include/raft/spatial/knn/detail/ivf_pq_build.cuh
Lines 1173 to 1174 in 0076101
@cjnolet's message that explains why we want to avoid doing that: #1113 (comment)
Note: once we do this change, we should also update our benchmarks to reflect the typical/proper use of these pools (note that the use of pools in benchmarks can slow down benchmarks significantly though if setup for each instance of the benchmark class, bonus points if we can reuse the pools properly).
cc @achirkin @tfeher
The text was updated successfully, but these errors were encountered: