Invalid GPU device for faiss.GpuIndexFlatL2()

[preethiseshadri518]

I am trying to use FAISS to perform a similarity search using GPUs. However, I seem to have trouble specifying the GPU device correctly.

Here is what I do to specify the GPU device when I run code in tensorflow. This works properly if I want to run code in my jupyter notebook using device 2.

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID" 
os.environ['CUDA_VISIBLE_DEVICES'] = '2'

However, I seem to have an issue when I try to setup faiss.GpuIndexFlatL2. I get the following error: Error in faiss::gpu::GpuIndex::GpuIndex(std::shared_ptr<faiss::gpu::GpuResources>, int, faiss::MetricType, float, faiss::gpu::GpuIndexConfig) at /__w/faiss-wheels/faiss-wheels/faiss/faiss/gpu/GpuIndex.cu:53: Error: 'config_.device < getNumDevices()' failed: Invalid GPU device 2

features = np.random.random((1000, 768)) # just a placeholder for now
res = faiss.StandardGpuResources()
config = faiss.GpuIndexFlatConfig()
config.useFloat16 = False
config.device = 2
index = faiss.GpuIndexFlatL2(res, features, config)

I have also tried changing config.device to other device numbers 0, 1, 2, etc. since I have access to 8 GPU devices and the same thing happens for all of them except for device 0, where I get a OOM message (cudaMalloc error out of memory). Note: I am using faiss-gpu 1.7.1.

Any ideas on what is happening and how to fix this issue? Thanks!

[mdouze]

CUDA_VISIBLE_DEVICES is restricted to only GPU #2 so it is normal that only GPU 0 is visible (CUDA_VISIBLE_DEVICES maps GPU 2 to the only visible GPU at index 0).

[phanxuanduc1996]

I used Anaconda, I installed bellow, it worked. If you use Docker, please install Miniconda.
conda install faiss-gpu cudatoolkit=11.1 -c pytorch-gpu
conda install -c anaconda pytorch-gpu