Understanding which underlying lib faiss uses

[mariokostelac]

Summary

Hello, I'm installing faiss-cpu package through pip and I'm wondering whether I'm missing out on some performance (given that I want to run on CPU ofc).

Is it possible to know whether faiss is linked to Intel MKL or not? In general, is it possible to know what libraries faiss links to?

Platform

OS: Linux $hostname 4.14.243-185.433.amzn2.x86_64 #1 SMP Mon Aug 9 05:55:52 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Faiss version: faiss-cpu 1.7.1.post2

Installed from: pip install faiss-cpu

Faiss compilation options: Not applicable

Running on:

• CPU
• GPU

Interface:

• C++
• Python

[mdouze]

If you installed via conda, then you are guaranteed to use MKL.
What platform are you running on? Please use the issue template.

[mariokostelac]

Hey @mdouze, sorry for not using the template, I've changed the initial comment.

[mariokostelac]

Here's what I did to find out what it's linked to, but thought that there might be a function within faiss package to get something similar

[ec2-user@hostname faiss]$ pwd
/app/.heroku/python/lib/python3.6/site-packages/faiss
[ec2-user@hostname faiss]$ ldd _swigfaiss.cpython-36m-x86_64-linux-gnu.so 
        linux-vdso.so.1 (0x00007ffe67a9f000)
        libgfortran-7e18e706.so.4.0.0 => /app/.heroku/python/lib/python3.6/site-packages/faiss/./../faiss_gpu.libs/libgfortran-7e18e706.so.4.0.0 (0x00007ff48bbb6000)
        librt.so.1 => /lib64/librt.so.1 (0x00007ff48b9ae000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007ff48b62c000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ff48b2ec000)
        libgomp-a34b3233.so.1.0.0 => /app/.heroku/python/lib/python3.6/site-packages/faiss/./../faiss_gpu.libs/libgomp-a34b3233.so.1.0.0 (0x00007ff48b0c2000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff48aeac000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff48ac8e000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff48a8e3000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff49ba41000)
        libquadmath-96973f99.so.0.0.0 => /app/.heroku/python/lib/python3.6/site-packages/faiss/./../faiss_gpu.libs/libquadmath-96973f99.so.0.0.0 (0x00007ff48a6a6000)

It's certainly not linking mkl.

[mdouze]

Sorry, we do not support pip for precisely that reason: if you use conda normally there is some guarantee that the libraries we link with are the correct ones.

[mariokostelac]

No need to apologize, it's my bad. I thought that faiss-cpu was published by somebody from your team, but it doesn't look like that.