-
Notifications
You must be signed in to change notification settings - Fork 783
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove pyconfig.h
header parsing
#1521
Conversation
The config header parsing code was supposed to be only invoked when cross-compiling for Windows, but in reality it fails to correctly parse the config header files shipped with the upstream Python for Windows. Given that there are now better options for reliable cross-compiling for Windows such as `PYO3_CROSS_PYTHON_VERSION` or the `abi3-py3*` features, it should be OK to remove this config for v0.14. Update the cross-compilation instructions section of the user guide. Fixes PyO3#1337
Thanks, yes this is probably fine to go, however it does lead to a possible situation where a user trying to cross-compile for a very unusual windows build might not be able to configure pyo3 as they need. I've been conisidering a PR which enables users to provide exact configuration externally (maybe as JSON), but didn't finish it yet. We might want to consider adding that before merging this, just incase anyone needs the flexibility. |
I agree that a configuration tool which is independent of the target Python installation would be nice to have, Mainly because it is only capable of parsing autoconf/autoheader generated headers, but the Windows Python port does not use these tools. It instead supplies a hand-written config header containing multiple One such example is the In the case someone customizes their Python build on Windows, they are supposed to define some magic preprocessor macros like |
I think it's OK to remove this if we can prepare a more flexible solution until the next release.
Maybe |
This must be a per-target setting, and the target triple is supplied externally via Can we tolerate another 3-4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, if it were an JSON file it would be an environment variable specifying where to find the JSON.
Using JSON will likely introduce a build dependency on serde and serde_json, which take quite a long time to compile.
I was thinking I would make this "advanced" build configuration opt-in via a feature :)
Regarding this PR specifically, I'm now convinced that leaving the header parsing in causes more headaches than it solves. Users who really need advanced configuration can always patch their local PyO3 temporarily!
I merged main to fix CI. |
I've looked through the Windows
Defining Disabling Line 296 in 3bc5caa
As a follow-up, I propose adding IMO, as far as cross-compiling for Windows is concerned, the JSON configuration feature may be an overkill. |
Note that the last time I experimented with this, I concluded linking the Python interpreter statically doesn't work very well (probably not until we have some upstream support). When linking binaries and tests statically Rust seems to trim out the symbols that aren't used directly by the program, so when other Python extension modules are imported by the statically-linked binary, often there are missing symbol errors. E.g. #742 So it may not be worth your effort implementing that variable. Thanks for looking though! |
I was thinking about programs embedding a Python interpreter for application-level scripting, just like Lua and Tcl interpreters are commonly used. Loading Python extension modules is not a requirement in this case, because it would allow to escape the user script sandboxing... |
Mmm. This would include parts of the Python standard library which are implemented as C extension modules! |
As a side note, apparently it is possible to link in the standard library modules statically, as described here: |
PYO3_CROSS_VERSION was renamed to PYO3_CROSS_PYTHON_VERSION. PYO3_CROSS_INCLUDE_DIR was removed in PyO3#1521
The config header parsing code was supposed to be only invoked when
cross-compiling for Windows, but in reality it fails to correctly parse
the config header files shipped with the upstream Python for Windows.
Given that there are now better options for reliable cross-compiling
for Windows such as
PYO3_CROSS_PYTHON_VERSION
or theabi3-py3*
features,it should be OK to remove this config for v0.14.
Update the cross-compilation instructions section of the user guide.
Fixes #1337