pip doesn't work from statically linked executables #6543
Comments
|
I filed a CPython issue at https://bugs.python.org/issue37060. |
Non-dynamic executables can raise OSError when importing ctypes because dlopen(NULL) is called on module import and dlopen() won't work on non-dynamic executables. This commit teaches the glibc version sniffing module to handle a missing or not working ctypes module.
|
setuptools is also affected. I figure I'll wait for someone from the PyPA to comment on matters before I file an issue / submit a PR to setuptools. |
|
I would go ahead and file an issue on the setuptools tracker as you'll get a different set of readers there. |
|
Note - your actual issue here is in But assuming you do want to treat the broader issue as worth addressing: I strongly believe that this is a ctypes bug - there's no way that it's reasonable to expect every import of a stdlib module to be checked for errors in every project. And it's not as if ctypes is documented as an optional stdlib module (at least not that I could see). Having said that, I see the reasoning that we have to live with the reality, and therefore we might have to deal with this. But many of our ctypes usages are in vendored libraries (colorama, urllib3) so you'll need to raise issues against those projects as well. And I'd still argue for fixing issues on a case by case basis, prompted by real world problems, rather than just making a blanket rule that we always have to protect ctypes usage (it's not like we make provision for systems where threading isn't available, for example). I would want to watch the Python bug report closely, though - if the CPython core devs don't accept that it's a bug for "import ctypes" to fail, then at a minimum the CPython docs should make that clear - there's nothing in the docs at the moment to suggest that it's any more acceptable for "import ctypes" to fail than (say) "import subprocess". |
This is related to https://bugs.python.org/issue37060 and pypa/pip#6543. With patched versions of setuptools and pip installed, we can now `pip install` without issue on non-dynamic Python executables!
|
I agree that this is a ctypes bug. I think it is best for ctypes import to gracefully handle the OSError and continue with import. Then the question becomes what should consumers of ctypes do - if anything - to work around the import time failure in existing Python releases. I highly doubt many people out there have built a non-dynamic Python binary. I think it is a reasonable stance to tell people they need a minimum version of Python - perhaps 3.7.4 - to run a fully static binary. But there's still the run-time issue of In any case, I've worked around this issue in python-build-standalone by patching cpython, setuptools, and pip. |
triage-new-issues
bot
removed
the
S: needs triage
lock
bot
added
the
auto-locked
Environment
Description
In the bowels of pip, the
ctypesmodule is imported. Importing this module on statically linked executables (such as Python builds compiled with MUSL) fails withOSErrorbecause executingctypes/__init__.pyattempts to calldlopen(), which will always fail on binaries that don't support dynamic loading.This is arguably a bug in CPython's
ctypesmodule, as it could possibly gracefully fail on failure callingdlopen(). But that bug has shipped for years and it is everyone else's responsibility to work around it.Expected behavior
I think pip should handle failure to import the
ctypesmodule gracefully and not abort.How to Reproduce
With a statically linked Python executable that doesn't have a
.dynamicor other related ELF sections, attempt to runpython -m ensurepip install <package>.The zstd compressed tarball at https://github.com/indygreg/python-build-standalone/releases/download/20190505/cpython-3.7.3-linux64-musl-20190526T0219.tar.zst contains such an executable under
python/install/bin/python3.7.Output
The problem with CPython specifically:
The text was updated successfully, but these errors were encountered: