Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python: import errors namespace packages since setuptools egg/wheel change #71826

Open
FRidh opened this issue Oct 23, 2019 · 5 comments
Open

python: import errors namespace packages since setuptools egg/wheel change #71826

FRidh opened this issue Oct 23, 2019 · 5 comments
Labels
0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: python

Comments

@FRidh
Copy link
Member

FRidh commented Oct 23, 2019 •
edited
Loading

Describe the bug
The way setuptools is installed was changed from egg to wheel in 90be4c2. This has caused import errors when importing namespace packages in case of Python 2.

History

  1. bootstrapped-pip has setuptools installed from wheel.
  2. setuptools was installed as egg. buildPythonPackage would always add this setuptools. It likely was at the front of PYTHONPATH so they would receive the egg version.
  3. buildPythonPackage was reimplemented as setup hooks. setuptools was no longer included during runtime, so users had to add it. When they would add setuptools it would be in the front of PYTHONPATH, thus they would still receive the egg version.
  4. setuptools installation method was changed from egg to wheel in 90be4c2, causing regressions.

Relevant issues:
@FRidh FRidh added 0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 6.topic: python labels Oct 23, 2019
@FRidh FRidh changed the title python: import errors namespace packages since setuptools change python: import errors namespace packages since setuptools egg/wheel change Oct 23, 2019
@FRidh FRidh added this to the 20.03 milestone Oct 23, 2019
@singron
Copy link
Contributor

singron commented Oct 23, 2019

Hmmm. I think 73d077e seems to fix it.

I'm not sure how this ever worked honestly. The trick is that python only loads *.pth files (which implement namespace packages in python2) from "site" directories and PYTHONPATH doesn't set "site" directories. You can set NIX_PYTHONPATH and the nixpkgs sitecustomize.py will add them to the "site" directories.

@FRidh
Copy link
Member Author

FRidh commented Oct 24, 2019

The analysis is correct, PYTHONPATH does indeed not load .pth files which is how namespace packages are implemented.

For twisted the reason it now works is that, by executing it in the postFixup, it is already correctly "wrapped" (actually, injected), and thus able to load the .pth files (site.addsitedir). NIX_PYTHONPATH was inplemented as an alternative to exactly this, so it would deal with .pth files.

@anderslundstedt
Copy link
Contributor

Could this be what is causing #71209?

@FRidh FRidh modified the milestones: 20.03, 20.09 Mar 30, 2020
@FRidh
Copy link
Member Author

FRidh commented Mar 30, 2020

Removing milestone. I don't consider this a 20.03 blocker. Also not adding a new milestone because Python 2 is EOL.

@marcinkuzminski marcinkuzminski mentioned this issue May 18, 2020
Open
@stale

This comment has been minimized.

Sign in to view
@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Sep 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0.kind: bug Something is broken 0.kind: regression Something that worked before working no longer 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: python
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@singron @FRidh @anderslundstedt