Allow equivalent distutils and sysconfig locations #9839
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
uranusjr
force-pushed
the
sysconfig-allow-samefile
branch
from
d995c6f to
c4649ae
Compare
uranusjr
force-pushed
the
sysconfig-allow-samefile
branch
from
c4649ae to
7ac41ad
Compare
For compatibility with distutils. This is only done when pip is not inside a virtual environment due to a quirk in pip's previous implementation to the header path.
uranusjr
force-pushed
the
sysconfig-allow-samefile
branch
from
7ac41ad to
d9be2a0
Compare
|
I'm on MacOS Big Sur using python as installed by Homebrew. After the changes in #9839, the system-level issues look like sysconfig doing the right thing and distutils doing the wrong thing. On my system:
There is a similar discrepency for |
|
Could you post the content of this file at the following location? (If I’m understanding Homebrew’s build script correctly; try to dig around for a I think Homebrew is using it to configure distutils.cfg, but sysconfig does not support this mechanism. This mechanism is thought to be obscure and no longer in use, so we might need to tell Homebrew to stop doing it… |
|
~~Here's the Homebrew install script, which might give some insight?
|
|
I think it’s this part that’s breaking things: From the build script, Homebrew builds Python with prefix The distutils configuration is something the packaging community has been trying to deprecate and remove for a while (the documentation I’m linking to is ancient!) because the mechanism is extremely easy to break by the user. Instead, Python distributors are adviced to patch For reference, this is Debian’s install layout patch (Python 3.7 used as an example, a similar patch is available for every version): https://sources.debian.org/patches/python3.7/3.7.3-2+deb10u3/distutils-install-layout.diff/ I’m not sure how best to report this to Homebrew, so any assistance would be awesome. Regarding what pip can do, I think I will add an additional check to detect whether the different path value comes from a distutils config, and emit a different message instead (along the lines of hey you have a distutils config, that’s deperecated and we’ll drop it soon; if you didn’t set it yourself, tell whoever provides your Python installation to stop doing this). |
|
I don't quite understand the whole situation, or what's considered "correct", and I'm glad it makes sense to you! Here's the public forum for Homebrew, which I think is the best place to bring this up. https://github.com/Homebrew/discussions/discussions There also seem to be some good discussions with concrete user confusions after making changes to Python. The first 3 I found: |
|
Homebrew is not the whole story. I get the below using the system Python3 on MacOS.
|
pradyunsg
approved these changes
Apr 25, 2021
File an issue on https://github.com/Homebrew/homebrew-core/issues, with the title starting with "Python3: ". |
|
Re: System Python 3 on Mac This issue is known; sysconfig is consistently giving different paths on all platforms except Windows when |
|
@rotu Can you run the following against your system from distutils.dist import Distribution
Distribution().find_config_files() # This should return a list.Also $ python3 -m sysconfig |
|
On MacOS Big Sur: $ /usr/bin/python3 -c "from distutils.dist import Distribution; print(Distribution().find_config_files())"
[]
|
Ah, so macOS is doing the same thing as Debian, inventing its own scheme. This explains things. We’ll need yet another special case in the prefix scheme selection logic. Could you verify It’d be awesome to |
|
I get basically the same as #9839 (comment) I noticed that the developer paths are mentioned by the system python. So removing them and trying to launch python3 shows this window: So the system python is actually just a stub.
{'data': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8',
'include': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8',
'platinclude': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8',
'platlib': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages',
'platstdlib': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'purelib': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages',
'scripts': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin',
'stdlib': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8'}
('nt',
'nt_user',
'osx_framework_library',
'osx_framework_user',
'posix_home',
'posix_prefix',
'posix_user')
{'data': '/Library/Python/3.8',
'include': '/Library/Python/3.8/include',
'platinclude': '/Library/Python/3.8/include',
'platlib': '/Library/Python/3.8/site-packages',
'platstdlib': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8',
'purelib': '/Library/Python/3.8/site-packages',
'scripts': '/usr/local/bin',
'stdlib': '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8'}
{'data': '/Users/dan/Library/Python/3.8',
'include': '/Users/dan/Library/Python/3.8/include',
'platlib': '/Users/dan/Library/Python/3.8/lib/python/site-packages',
'platstdlib': '/Users/dan/Library/Python/3.8/lib/python',
'purelib': '/Users/dan/Library/Python/3.8/lib/python/site-packages',
'scripts': '/Users/dan/Library/Python/3.8/bin',
'stdlib': '/Users/dan/Library/Python/3.8/lib/python'}On my system, |
|
On further inspection, I think this patch does not affect anything meaningfully and am going to close without merging. Introducing |
|
I’ve posted to Homebrew/discussions#1405 to inform Homebrew about the issue. (Not on the issue tracker since they seem to be very strict about limiting its usage to bugs.) |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends on #9838 (filed separately so I can get a number for the news fragment).