Numba and pytorch: LLVM crash #7299
Comments
There are two LLVM versions installed because numba is depending on llvm10 while pytorch probably uses llvm11. I am not even sure if loading two llvm shared library into the same process is suppose to work. |
|
Seems to be related to pytorch/pytorch#62754 |
|
*We had a simillar issue few months ago I think, this is caused by llvmlite.so (statically linked with llvm) exporting some llvm api as public symbols and it will conflict with any other library which also exports llvm symbols. On Numba side this can be mitigated by not exporting any llvm api from llvmlite.so and only exporting llvmlite api actually used by Numba. |
|
I think conda-forge builds of libraries depending on LLVM are typically dynamically linked to the LLVM DSOs. The issue described by @Hardcode84 is hopefully fixed by numba/llvmlite#730. |
|
This issue is marked as stale as it has had no activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with any updates and confirm that this issue still needs to be addressed. |
|
numba/llvmlite#730 has been merged and released with llvmlite 0.37.0 -- I assume this issue is now resolved? @fzimmermann89 please feel free to re-open in case the issue remains. |
I don't think this will resolve it? The above seems to be due to two libraries dynamically linking to different versions of LLVM. |
OK, makes sense, lets try to get a confirmation the bug still exists? |
|
This issue is marked as stale as it has had no activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with any updates and confirm that this issue still needs to be addressed. |
|
@fzimmermann89 is there any chance you could look into this again to see if your issue is still present? |
|
With current versions from conda-forge/pytorch nightly, e.g. the issue seems to be gone. Not sure if the root cause is actually solved or if it might break again if any other library moves to a different libllvm version, but I'll close this for now. Thank you for all your work! numba is great :) |
stuartarchibald
added
no action required
Thank you for the kind words, happy you enjoy it and thank you for following up! 👍 |
See pytorch/pytorch#51511 :
Importing numba and pytorch causes an crash of python with the following llvm related message:
python: /tmp/llvm/include/llvm/Support/CommandLine.h:813: void llvm::cl::parser<DataType>::addLiteralOption(llvm::StringRef, const DT&, llvm::StringRef) [with DT = llvm::FunctionPass* (*)(); DataType = llvm::FunctionPass* (*)()]: Assertion findOption(Name) == Values.size() && "Option already exists!"' failed.Not sure who is at fault and what is going on there...
The conda environment used:
The text was updated successfully, but these errors were encountered: