setuptools 60: "No module named 'setuptools'" when installing Poetry from source #2980
Comments
alkasm
changed the title
alkasm
changed the title
alkasm
changed the title
alkasm
changed the title
|
Thanks for the report. In this case, I’d like someone from the poetry project to help triage and ascertain what’s going wrong in Setuptools that’s triggering issues in poetry. I can likely help debug after I deal with another emergent issue on Ubuntu. |
|
I also got this error but slightly different context, hope this helps:
I wouldn't expect poetry to be involved in step two as its in a fresh pipeline agent and just using pip to install the prebuilt package. That said, I guess there could be something I'm missing with the backend from poetry that's causing this issue. |
|
I can confirm that either using |
|
Also running into this, but even with setuptools 57.4.0. When trying to install a random package, I'm getting the following: Traceback (most recent call last):
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
status = run_func(*args)
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
return func(self, options, args)
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 338, in run
requirement_set = resolver.resolve(
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in collect_root_requirements
req = self._make_requirement_from_install_req(
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 430, in _make_requirement_from_install_req
cand = self._make_candidate_from_link(
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 201, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 281, in __init__
super().__init__(
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 546, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 47, in prepare_distribution_metadata
self._install_build_reqs(finder)
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 106, in _install_build_reqs
build_reqs = self._get_build_requires_wheel()
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 83, in _get_build_requires_wheel
return backend.get_requires_for_build_wheel()
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 172, in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
File "/home/rr-/.local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend
obj = import_module(mod_path)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/tmp/pip-build-env-sbcna1z1/overlay/lib/python3.10/site-packages/poetry/core/masonry/__init__.py", line 10, in
from .builder import Builder
File "/tmp/pip-build-env-sbcna1z1/overlay/lib/python3.10/site-packages/poetry/core/masonry/builder.py", line 7, in
from .builders.sdist import SdistBuilder
File "/tmp/pip-build-env-sbcna1z1/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in
from .wheel import WheelBuilder
File "/tmp/pip-build-env-sbcna1z1/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 23, in
from packaging.tags import sys_tags
File "/tmp/pip-build-env-sbcna1z1/overlay/lib/python3.10/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in
import distutils.util
File "/usr/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 88, in create_module
return importlib.import_module('setuptools._distutils')
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'
I don't know much about PEP-517 but hearing "build isolation" and seeing Poetry requiring setuptools, makes me think it's Poetry that should add this dependency to be downloaded whenever the user chooses to install a project? |
|
FWIW I checked the following setuptools versions: ❌ 60.0.0 |
|
I notice in this stack trace: That |
|
After reinstalling the system package Anyway, what's interesting is that now after sorting out my mess, I cannot reproduce OP's issue even with a user installation of setuptools 60.0.0: $ pip install --upgrade pip setuptools==60.0.0 --user Here's how Python sees setuptools: $ python3 Python 3.10.1 (main, Dec 11 2021, 17:22:55) [GCC 11.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import setuptools >>> setuptools <module 'setuptools' from '/home/rr-/.local/lib/python3.10/site-packages/setuptools/__init__.py'> >>> setuptools.__version__ '60.0.0' Here's my OS info: $ uname -a Linux lightning 5.15.5-arch1-1 pypa/setuptools#1 SMP PREEMPT Thu, 25 Nov 2021 22:09:33 +0000 x86_64 GNU/Linux $ cat /etc/os-release NAME="Arch Linux" PRETTY_NAME="Arch Linux" ID=arch BUILD_ID=rolling ANSI_COLOR="38;2;23;147;209" HOME_URL="https://archlinux.org/" DOCUMENTATION_URL="https://wiki.archlinux.org/" SUPPORT_URL="https://bbs.archlinux.org/" BUG_REPORT_URL="https://bugs.archlinux.org/" LOGO=archlinux-logo |
|
This is a bug (?) in pip's isolated env builder. I don't know what the purpose of these two lines is but the effect is that if you have setuptools 60+ installed in your environment when you invoke Under line 123 of print(sys.meta_path)Then try to build any Poetry package with |
|
I've also no idea why the finder is added to the meta path three times... |
Should this be filed on pypa/pip then? |
|
I think so, yeah. |
Seems they are aware of something related: pypa/pip#10742 |
jaraco
changed the title
|
Before I transfer this issue to pip, I want to be careful to understand that there's not something that Setuptools could/should be doing differently.
Where is I'm confused. Is the issue when installing Poetry from source or when installing a package which uses Poetry from source? I see conflicting reports above? Moreover, I'm unable to replicate the issue. What are the conditions needed to replicate the error?
I can transfer the issue if appropriate.
I'm not sure that's related. The error message is different and that issue pertains to pip's internal tests and not a user-facing issue. |
jaraco
changed the title
Any Poetry package from source including Poetry itself.
|
I observe that using Thanks for the Moreover, I find if I add At this point, I do agree that the issue exists primarily with pip's isolation technique... although there may need to be some setuptools support needed to honor pip's intentions. |
|
The reason the hook appears twice from site.py is because Python adds site-packages twice, causing .pth files to be processed twice: In particular, in a venv, addsitepackages is called twice apparently to give site-packages precedence over user packages. |
|
With setuptools 60.0.4: ❯ pip install git+ssh://[email protected]/python-poetry/poetry
Collecting git+ssh://****@github.com/python-poetry/poetry
Cloning ssh://****@github.com/python-poetry/poetry to /private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-req-build-jo2iq4ih
Running command git clone --filter=blob:none -q 'ssh://****@github.com/python-poetry/poetry' /private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-req-build-jo2iq4ih
Resolved ssh://****@github.com/python-poetry/poetry to commit 6a35cc41e950d4ff0f1efad649c1d15d689a9ce2
Installing build dependencies ... done
Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
status = run_func(*args)
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
return func(self, options, args)
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 338, in run
requirement_set = resolver.resolve(
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in collect_root_requirements
req = self._make_requirement_from_install_req(
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 430, in _make_requirement_from_install_req
cand = self._make_candidate_from_link(
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 201, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 281, in __init__
super().__init__(
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 546, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 47, in prepare_distribution_metadata
self._install_build_reqs(finder)
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 106, in _install_build_reqs
build_reqs = self._get_build_requires_wheel()
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 83, in _get_build_requires_wheel
return backend.get_requires_for_build_wheel()
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 172, in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend
obj = import_module(mod_path)
File "/opt/homebrew/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-build-env-3yd07mfg/overlay/lib/python3.9/site-packages/poetry/core/masonry/__init__.py", line 10, in <module>
from .builder import Builder
File "/private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-build-env-3yd07mfg/overlay/lib/python3.9/site-packages/poetry/core/masonry/builder.py", line 7, in <module>
from .builders.sdist import SdistBuilder
File "/private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-build-env-3yd07mfg/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in <module>
from .wheel import WheelBuilder
File "/private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-build-env-3yd07mfg/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 23, in <module>
from packaging.tags import sys_tags
File "/private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-build-env-3yd07mfg/overlay/lib/python3.9/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
import distutils.util
File "/Users/pradyunsg/Developer/sphinx-basic-ng/.venv/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 92, in create_module
return importlib.import_module('setuptools._distutils')
File "/opt/homebrew/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'Hmm... It seems that |
…/issues/10743#issuecomment-998321115
…/issues/10743#issuecomment-998321115
…/issues/10743#issuecomment-998321115
…/issues/10743#issuecomment-998321115
…/issues/10743#issuecomment-998321115
This is important for user (e.g., AISE) development alongside of framework development.
|
Alright, so the problem is that setuptools is using import hooks which is not something that pip's build isolation isolates against. I think the immediate fix belongs in setuptools (at best, pip can roll out a fix in Jan/Apr 2022 and this likely needs to be fixed by setuptools in a way that is compatible with older pip versions), so I'm gonna throw this back over to that repository. I'll file a separate follow up issue on pip's end to track isolating from import hooks/pth files (likely through something like pypa/pip#10720). |
As I explained 7 days ago. I understand how asottile might have missed it since he's blocked me (apparently that's a thing we do in the packaging community - just block people we disagree with), not sure how everybody else has. |
I meant to respond to this sooner, then got distracted and it fell off my radar. The hack is pretty simple. A top-level module _distutils_hack is installed and loaded by a distutils-precedence.pth file. All that An open question to the pip team (@pradyunsg) is what are these lines intended to accomplish? Looking at the whole block, it looks like the sitecustomize explicitly adds It's not obvious to me that there's anything a I want to explore whether the MetaPathFinder should be more lenient when its accompanying module cannot be found. #2962 captures a related problem where the hook exists but picks up an unrelated setuptools module. |
|
Released as 60.1.1. Please test and let me know if there are outstanding issues. |
- `setuptools` issues. - pypa/setuptools#2980.
I ran into this issue in one of my CI flows with poetry 1.1.12 and setuptools 60.0.0 I tried to fix it by simply applying 60.1.1, but that did not fix it. I had to additionally downgrade poetry to 1.1.11. That and this setuptools 60.1.1 fix worked for me. |
|
I was worried that there was some race between fixes between poetry 1.1.12 and setuptools 60.1.1, but I'm unable to replicate the issue even with poetry 1.1.12: Can you divine some details on how to reproduce? |
setuptools version
60.0.0
Python version
3.9.6
OS
macOS 11.5.2
Additional environment information
No response
Description
Installing a package which uses a Poetry backend from source fails on 60.0.0. Poetry packages that I've tried (my own, as well as Poetry itself) fail to install. Package installs are successful after reverting to previous setuptools versions.
The changelog states that v60.0.0 introduces a breaking change:
When using the environment variable as suggested, the package installs are again successful.
I tried a few flit and setuptools projects and observed no issues with those.
Expected behavior
The package should install without error.
How to Reproduce
Poetry itself uses poetry as a build tool, so installing poetry from source fails.
Alternatively, clone poetry and run
pip install .locally inside the repo.Output
pip install git+ssh://[email protected]/python-poetry/poetry
Code of Conduct
The text was updated successfully, but these errors were encountered: