-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
parse_requirements fails on requirements.txt files with multiple line continuations. #2422
Comments
Thanks for the offer and taking the time to get an early read... and your caution is warranted. Although historically, this project and much of the Python packaging tools would develop de-facto standards, often in isolation from the other tools, these days we're aiming to avoid the pitfalls that creates. What that means for a problem like the one you've described is that rather than accept the output of pip-compile as correct, we'd instead want to develop and drive to acceptance a standard for requirements files on which all tools can rely. In the case of If we can find some reference that line continuations are a supported format, then by all means Setuptools should support it. If such a reference doesn't exist, then we have a bigger barrier ahead, which is to create that standard. A change of this magnitude may not require a PEP (a documented reference is sufficient) or it may require a PEP or modification to one. @pradyunsg Do you know who owns the requirements.txt format spec and are you aware of any documentation that specifies the syntax? |
Nope, there isn't really a spec for it. It's purely implementation defined. There have been some efforts toward designing a "requirements 2.0" that'll make things nicer on that front -- but that's not happened yet. |
That's what I thought might be the case, since nothing related to packaging is ever simple. ;) I could argue that since a) pip itself accepts this format (and checks the multiple hashes), and b) a single line continuation is accepted, but not a double one, it feels more like an oversight, rather than a conscious decision not to allow continuations, but I definitely do understand it's best to be very cautious in making any changes to something so central. |
According to |
Encountered this same issue. If/until this issue is ever addressed, setuputils should handle Exception, from the error message from vendored pyproject_hook package, gracefully. Something like,
pyproject.toml can be successfully parse,
While build fails, often swallowing the Exception. Since it's running the backend in a subprocess, the Exception isn't propagated.
Save the ExceptionHacked a hack so as to log the Exception. Here is the original. Again this is a hack to diagnose the issue, otherwise the log file would be written to a sane location. pyproject_hooks/_in_process/_in_process.py
Turn off the hashes and regenerating requirements.txt and requirements-[group].txt then
|
pip files with hashes, as generated by for instance the
pip-compile
command in pip-tools, are currently not parseable byparse_requirements
inpkg_resources/__init__.py
. It looks like the issue is multiple line continuations, as for instance here:I think this:
https://github.com/pypa/setuptools/blob/master/pkg_resources/__init__.py#L3074
should be
rather than
if
.I may have missed some subtle reason why this is not already the case, but it solved the issue I was running into locally where
python setup.py [any command]
was failing because I had a requirements.txt with hashes generated by pip-tools. (I think it's a fairly unusual edge case because we also usepbr
, which I think is what's triggering the parsing of the requirements to happen when calling setup.py).I'd like to get a reading on how likely a patch is to be accepted before I write up the PR and tests, but I'd be happy to do so.
If it's not something that's likely to get in, I'll probably dump pbr as a dependency instead.
The text was updated successfully, but these errors were encountered: