swigfaiss.so incompatible with mac arm64

[CaucherWang]

Summary

swigfaiss.so cannot be read by arm infrastructure, like Apple M1

Platform

OS: macOS 12.5

Installed from: compiled from source

Faiss compilation options: --without-cuda

Running on:

• CPU

Interface:

• Python

Reproduction instructions

at python/Makefile, I only have the rule to generate swigfaiss.so, despite in arm64 infra.

when I try to import faiss:
ImportError: dlopen(/faiss-learned-termination/python/faiss/_swigfaiss.so, 0x0002): tried: '/faiss-learned-termination/python/faiss/_swigfaiss.so' (mach-o file, but is an incompatible architecture (have (arm64), need (x86_64)))

[mdouze]

you mean you are cross-compiling by accident?

[CaucherWang]

No, I want to build Faiss on my MBP(arm infra.)，but meet problems on the python interface, where the makefile can only generate swigfaiss.so rather than "swigfaiss.dylib".
.so in the python package cannot be used in mac.

[wx257osn2]

the makefile can only generate swigfaiss.so rather than "swigfaiss.dylib".
.so in the python package cannot be used in mac.

So this is not a problem with arm64 , but a problem on macOS with arm64 . I install and develop faiss on Linux with arm64.

Then, how did you install swig and other dependencies of faiss? And how did you build faiss? I think that current information is not enough to help you.

[CaucherWang]

the makefile can only generate swigfaiss.so rather than "swigfaiss.dylib".
.so in the python package cannot be used in mac.

So this is not a problem with arm64 , but a problem on macOS with arm64 . I install and develop faiss on Linux with arm64.

Then, how did you install swig and other dependencies of faiss? And how did you build faiss? I think that current information is not enough to help you.

You are right! I have changed the title by adding "mac".
I install swig (4.0.2) by homebrew, and build Faiss as the guidelines in INSTALL.md (lines 100-132)

[wx257osn2]

Hmm, it seems that current homebrew has prebuilt swig for apple silicon, so swig probably works correctly. Of course your homebrew is for arm64, not for x86_64, right?

[CaucherWang]

Hmm, it seems that current homebrew has prebuilt swig for apple silicon, so swig probably works correctly. Of course your homebrew is for arm64, not for x86_64, right?

Yes, my homebrew is for arm64.
% brew -v
Homebrew 3.5.9-138-gd81bd6a
Homebrew/homebrew-core (git revision bb6e3862b8f; last commit 2022-08-22)
Homebrew/homebrew-cask (git revision 7eb97cddbc; last commit 2022-08-22)

% swig -version

SWIG Version 4.0.2

Compiled with clang++ [arm-apple-darwin21.1.0]

Configured options: +pcre

[wx257osn2]

@CaucherWang Hmm... some people had succeeded to build faiss on M1 Mac like #2111 and #2386, so it should be able, but actually I have no mac environment, so I can't any check at my own... Would you show me your CMake command and log?

[mdouze]

This worked for me:
https://gist.github.com/mdouze/2b96e804ee2825e72323c0a296061c7e#file-install_faiss_host_conda-bash