New test_set_include_dirs leaves files in working directory #169
Comments
|
So apparently, the C compiler class intentionally replaces absolute paths with relative ones. |
|
Removing that line (and the one above it), the tests still pass, so there's no good way to tell how important that behavior is. Do all known use-cases rely on relative paths? Or do some pass absolute paths and rely on this rewriting? In my opinion, the behavior should be consistent, to place the compiled file alongside its source regardless of whether it was specified absolute or relative. It's going to take some work to avoid potential breakage. |
|
Those lines were added in 5daaf7c, apparently for bpo-668662 → python/cpython#37775. |
…version 65.0.0
Eva Maxfield Brown (2):
Update keywords note on editable installs
Include changelog fragment
Jason R. Coombs (14):
Remove bdist_msi.
Remove old msvc modules.
Update changelog
Remove problems, in part because they cause shopkeep/pytest-black#62, but also because they're probably of limited relevance.
Extract method for mangling the base path. Ref pypa/distutils#169.
Suppress path mangling when running tests. Ref pypa/distutils#169.
Deprecate behavior in _mangle_base. Ref pypa/distutils#169.
Remove bdist_wininst.
👹 Feed the hobgoblins (delint).
Remove more references to bdist_wininst.
Remove more references to bdist_msi
Remove another reference to wininst
Update changelog
Bump version: 64.0.3 → 65.0.0
Pradyun Gedam (1):
Change line endings for `.cmd` file to CRLF
wim glenn (1):
typo fix
|
On my Ubuntu 22.04.1 LTS system, this deprecation warning behaves in a very odd way when using Cython's pyximport on Python 3.10.5. If I create a small test python file, and a small then running the script produces: AND the .pyx file is not built in the relative path, but in the absolute one: The I'm not sure how I tested on a few different versions of setuptools, with the following results:
|
|
Small clarification: The behaviour described as the future behaviour in the deprecation warning (i.e. that the absolute path will be used in future) is already the actual behaviour that occurs (at least in how pyximport uses this). This is the part that is very confusing. Maybe the "deprecated" behaviour only ever occurred on some specific platform? |
|
I suspect what's happening is pyximport is triggering the behavior because it's specifying an absolute path, and distutils is passing that absolute path to the |
|
In an attempt to replicate the issue, I created this Dockerfile: # syntax = docker/dockerfile:1.4.0
from jaraco/multipy-tox
RUN <<EOF cat >> pyxtest.py
import pyximport
pyximport.install()
import foo # noqa
foo.hello()
EOF
RUN <<EOF cat >> foo.py
# cython: language_level=3
def hello():
print("hello")
EOF
CMD pip-run cython setuptools -- -W default::DeprecationWarning pyxtest.pyWhen I run that image, it completes but doesn't emit the reported warning: As you can see, the latest version of Setuptools is getting installed. I also confirmed that "import distutils" does resolve to the Setuptools-local version of distutils. Any idea why I'm not replicating the failure? |
|
@jaraco Would it be easy to try with Python 3.10? I don't see it on Python 3.9 and earlier, so maybe 3.11 is fine too? |
|
I did try Python 3.10 also. Same result: I suspect there's another factor - perhaps something to do with using the native Ubuntu Python. aside: I've updated jaraco/multipy-tox so it no longer emits the pip as root warning. |
|
I figured out my mistake. I left the |
|
This Dockerfile replicates the issue nicely: # syntax = docker/dockerfile:1.4.0
from jaraco/multipy-tox
RUN <<EOF cat >> pyxtest.py
import pyximport
pyximport.install()
import foo # noqa
foo.hello()
EOF
RUN mkdir test
RUN <<EOF cat >> foo.pyx
# cython: language_level=3
def hello():
print("hello")
EOF
CMD py -3.10 -m pip-run -q cython setuptools -- -W default::DeprecationWarning -W 'ignore:the imp module' pyxtest.py |
|
I'm able to more directly trigger the error with this: # syntax = docker/dockerfile:1.4.0
from jaraco/multipy-tox
RUN <<EOF cat >> pyxtest.py
import pyximport
pyximport.install()
pyximport.build_module('foo', 'foo.pyx')
EOF
RUN mkdir test
RUN <<EOF cat >> foo.pyx
# cython: language_level=3
def hello():
print("hello")
EOF
ENV PYTHONWARNINGS="default::DeprecationWarning,ignore:the imp module"
CMD py -3.10 -m pip-run -q cython setuptools -- pyxtest.pyMoving the warnings to an environment variable makes it easier not to have to supply it. And invoking |
|
Even better, this installs the dependencies and pins to Python 3.10 by default: # syntax = docker/dockerfile:1.4.0
from jaraco/multipy-tox
RUN <<EOF cat >> pyxtest.py
import pyximport
pyximport.install()
pyximport.build_module('foo', 'foo.pyx')
EOF
RUN mkdir test
RUN <<EOF cat >> foo.pyx
# cython: language_level=3
def hello():
print("hello")
EOF
ENV PYTHONWARNINGS="default::DeprecationWarning,ignore:the imp module"
ENV PY_PYTHON=3.10
RUN py -m pip install cython setuptools
CMD py pyxtest.py |
|
Aha, so calling Passing |
|
The absolute path gets generated by this logic when |
|
When I do some debugging, I can see the effect of the path getting made relative because the outputs show the path twice: Now that I look at the object_filenames code again, I observe that the mangling is probably important. If the mangling isn't done, when os.path.join is invoked, the outdir will be disregarded. So maybe the mangling should be brought back, but documented more clearly... or maybe this code should stop trying so hard to provide every possible option such that it does unexpected things when no options are supplied. |
|
After thinking about this overnight, I'm pretty sure this deprecation will need to get removed. I think what this function is really trying to achieve is to ensure that object filenames are stored somewhere in the "output_dir", even if the source files are indicated by an absolute path. According to the docs, the files should be stored relative to their originally specified path in the output dir, but that example only considers relative paths. And to make matters worse, the default output dir is I think three things need to happen:
|
|
Ugh. I see |
|
Arg. Thank you for digging into this morass over the weekend! |
|
Setuptools 65.2 has should have the release that no longer emits a warning. I'm going to open a new issue to track the possible follow-up actions. |
|
Thanks! I will give 65.2 a try locally and in the QuTiP CI (which covers Python 3.6-3.10 and Ubuntu, Mac OS and Windows). |
…version 65.2.0
Anderson Bravalheri (6):
Capture recursion problem with editable finder
Fix recursion problem in finder
Limit number of string replacements
Add news fragment
Improve news fragment
Bump version: 65.1.0 → 65.1.1
Helio Machado (1):
Fix typo in docs/userguide/extension.rst
Jason R. Coombs (15):
Suppress warnings in deprecated msvc compilers
Update to setup-python v4. Fixes jaraco/skeleton#65.
Rename _mangle_base to _make_relative and add documentation about its purpose. Ref pypa/distutils#169.
In _make_relative, remove deprecation warning. Ref pypa/distutils#169.
Extract method for _make_out_path.
In _msvccompiler, only override _make_out_path
In _msvccompiler, re-use _make_relative.
Extract property for mapping src extensions to out extensions.
Extract property for out_extensions in _msvccompiler
Deprecate bcppcompiler.
Remove _msvccompiler._make_out_path.
In cygwincompiler, re-use object_filenames from ccompiler.
Add unit tests capturing the expectation
Update changelog
Bump version: 65.1.1 → 65.2.0
rnhmjoj (1):
Fix, again, finding headers during cross compiling
In #153, I added test_set_include_dirs. At least on my mac, this operation leaves a file in the working directory:
private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pytest-of-jaraco/pytest-32/test_set_include_dirs0/foo.oIt seems something about invoking a compiler on a file causes the path for that file to get built relative to the current directory :/
The text was updated successfully, but these errors were encountered: