pip 20.2.4 with 2020 resolver does not use pinned version in the constraints file

[jwhitlock]

What did you want to do?

Consider this requirements.txt:

-c constraints.txt

Django==3.1 \
    --hash=sha256:1a63f5bb6ff4d7c42f62a519edc2adbb37f9b78068a5a862beff858b68e3dc8b \
    --hash=sha256:2d390268a13c655c97e0e2ede9d117007996db692c1bb93eabebd4fb7ea7012b

Django 3.1 has three dependencies, listed with hashes in constraints.txt:

pytz==2020.1 \
    --hash=sha256:a494d53b6d39c3c6e44c3bec237336e14305e4f29bbf800b599253057fbb79ed \
    --hash=sha256:c35965d010ce31b23eeb663ed3cc8c906275d6be1a34393a1d73a41febf4a048
sqlparse==0.3.1 \
    --hash=sha256:022fb9c87b524d1f7862b3037e541f68597a730a8843245c349fc93e1643dc4e \
    --hash=sha256:e162203737712307dfe78860cc56c8da8a852ab2ee33750e33aeadf38d12c548
asgiref==3.2.10 \
    --hash=sha256:7e51911ee147dd685c3c8b805c0ad0cb58d360987b56953878f8c06d2d1c6f1a \
    --hash=sha256:9fc6fb5d39b8af147ba40765234fa822b39818b12cc80b35ad9b0cef3a476aed

With pip 20.2.4, this installs the four packages:

pip install -r requirements.txt

Using the 2020 resolver fails:

pip install -r requirements.txt --use-feature=2020-resolver

It also fails with the in-development version of pip.

Output

Working output:

$ pip install -r requirements.txt 
Collecting Django==3.1
  Using cached Django-3.1-py3-none-any.whl (7.8 MB)
Collecting asgiref==3.2.10
  Using cached asgiref-3.2.10-py3-none-any.whl (19 kB)
Collecting sqlparse==0.3.1
  Using cached sqlparse-0.3.1-py2.py3-none-any.whl (40 kB)
Collecting pytz==2020.1
  Using cached pytz-2020.1-py2.py3-none-any.whl (510 kB)
Installing collected packages: pytz, sqlparse, asgiref, Django
Successfully installed Django-3.1 asgiref-3.2.10 pytz-2020.1 sqlparse-0.3.1

Failed output:

$ pip install -r requirements.txt --use-feature=2020-resolver
Collecting Django==3.1
  Using cached Django-3.1-py3-none-any.whl (7.8 MB)
Collecting asgiref~=3.2.10
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    asgiref~=3.2.10 from https://files.pythonhosted.org/packages/d5/eb/64725b25f991010307fd18a9e0c1f0e6dff2f03622fc4bcbcdb2244f60d6/asgiref-3.2.10-py3-none-any.whl#sha256=9fc6fb5d39b8af147ba40765234fa822b39818b12cc80b35ad9b0cef3a476aed (from Django==3.1->-r requirements.txt (line 3))

The dependency that causes the error can vary from call to call.

Additional information

Here's the output of pipdeptree for the successful case:

$ pipdeptree
Django==3.1
  - asgiref [required: ~=3.2.10, installed: 3.2.10]
  - pytz [required: Any, installed: 2020.1]
  - sqlparse [required: >=0.2.2, installed: 0.3.1]
pipdeptree==1.0.0
  - pip [required: >=6.0.0, installed: 20.2.4]
setuptools==49.6.0
wheel==0.35.1

This was first reported in issue #8792, which is marked as fixed in pip 20.2.4 with PR #8839.

A working demo is at https://github.com/jwhitlock/pip-resolver-demo. I've simplified it to this example since it was first written.

[pfmoore]

asgiref~=3.2.10 isn't an == requirement. Under the new resolver, constraints don't override the existing requirements, they simply constrain what versions are visible as input to the resolver. The change in behaviour of constraints is probably what triggered this issue. The bad news is, I don't know how we'd fix this apart from suggesting you use a requirements file rather than a constraints file.

[jwhitlock]

I do not understand how constraints are used with the new resolver. Can you give an example?

[pfmoore]

They are basically a set of version specifications like

foo==1.0
bar>=2.0

The difference isn't so much in how they are specified (except that the new resolver doesn't accept more complicated requirement syntax like extras or markers in constraint files) as in how they are used. In the old resolver, they were essentially treated as another set of requirements, but they didn't trigger an install unless there was a "non-constraint" requirement for that package as well. In the new resolver, constraints are applied much earlier, as part of finding candidates to feed into the resolver, so that the resolver never even sees candidates that don't match the constraints (which IMO is much closer in behaviour to what I think "constraint" means). The new resolver also doesn't see the constraints, as they have been applied before the resolver gets involved.

The consequence here is that the resolver never sees a requirement asgiref==3.2.10 because that's in a constraint file. But that's OK, as it never sees a version of asgiref other than 3.2.10, because of the constraint, so there's no need to have it as a requirement as well.

The problem is that hash-checking mode requires that all requirements are specified as a == match on a version, and the only requirement you have is asgiref~=3.2.10, presumably from a dependency somewhere. So hash-checking mode complains that it's not an == match. If you add asgiref==3.2.10 to your command line (or to a requirements file), that would act as a version pin and satisfy hash-checking mode.

[pfmoore]

The reason I say "I don't know how we'd fix this" is because I think the issue is in hash-checking mode, which shouldn't be checking for explicit == requirements, but should actually be ensuring that only one version can satisfy the combined set of constraints/requirements on the resolution. That's a more general way of achieving the same result, but it's harder to implement. And I don't really know the code for hash-checking mode, so I've no idea how you'd go about implementing that more general check.

[jwhitlock]

the only requirement you have is asgiref~=3.2.10, presumably from a dependency somewhere

This question is an easy one to answer, it is from Django:

https://github.com/django/django/blob/197b55c53469cf8344d1ba35175236780cb83bd1/setup.cfg#L43-L46

The other one is harder, I'd have to get into the new resolver code. I'm guessing that the hash will have to get attached to the version earlier in the process. A requirement of asgiref==3.2.10 would be interpreted as
"asgiref, version 3.2.10, with any hash", and asgiref==3.2.10 --hash=sha256:7e51911ee... would be "asgiref, version 3.2.10, with hash sha256:7e5189...", and get carried through resolving to installation.

[jwhitlock]

I captured the verbose output of pip install -vvv --use-feature=2020-resolver. This line stood out:

https://gist.github.com/jwhitlock/c09714f7df4557333a38591ac1b642bf#file-output-txt-L1196

Checked 2 links for project 'pytz' against 2 hashes (2 matches, 0 no digest): discarding no candidates

My reading of this is that filter_unallowed_hashes knows that it needs to check hashes, and successfully filters by one with the given hashes. That gives me more hope that the hashes can be available on the Candidate.

[uranusjr]

I modified the title to reflect my observation to the issue. The error message indicates that hashes are not the issue, but the resolver not “using” the asgiref==3.2.10 line in constraints. It uses asgiref~=3.2.10 from Django instead, causing the failure.

With the current implementation, the user is expected to == all requirements as concrete requirements, and constraints don’t count. Personally I feel both pip’s current behaviour and your expectation make sense, and is (very) slightly in favour of making this work as you expect.

[jwhitlock]

I'd consider it a regression if this case wasn't handled by the 2020 resolver. But, it could be an intentional regression. Those of us that need to use hashes would avoid constraints, and use tools like pip-compile from pip-tools.

It would probably require a note in the docs for both constraints and hashes to say that the two shouldn't be used together.

[brainwane]

I just spoke with @pradyunsg and we confirmed that the docs update in https://pip.pypa.io/en/latest/user_guide/#watch-out-for within https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020 serves to resolve this issue.

Thank you for the bug report and for working through it with us!

[aawilson]

Got here from #9243. It seems weird to suddenly expect that, when a constraints file specifies a version pin and a hash, the resolver would ignore the hash completely when requirements doesn't specify it. That seems to make constraints files essentially useless, because now for anyone using require-hashes, it makes no sense to specify the dependency in both requirements.txt and constraints.txt, and it also doesn't make sense to go through repeated installs and find every subdep specified with a range and special-casing those for requirements.txt. Is there some reasoning someone can point to for breaking this flow?

[brianhwitte]

I am using pip-tools and pip-compile to generate a requirements.txt file. In the

requirements.in:
...
django-storages[boto3]==1.10.1
collectfast==2.2.0


requirements.txt:
collectfast==2.2.0 \
    --hash=sha256:2f6abc8cab7ec5... \
    --hash=sha256:e716b2234ab50... \
    # via -r requirements.in
django-storages[boto3]==1.10.1 \
    --hash=sha256:12de8fb26... \
    --hash=sha256:652275ab... \
    # via -r requirements.in, collectfast

As of pip 21, I get this error on CI:

$ pip install --no-cache-dir -r requirements.txt
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    django-storages>=1.6 from https://files.pythonhosted.org/packages/f8/3d/9cf5b269de2930.../django_storages-1.11.1-py3-none-any.whl#sha256=f28765826d507a... (from collectfast==2.2.0->-r requirements.txt (line 86))

in the collectfast setup.cfg:

install_requires =
    django-storages>=1.6

I'm guessing that the resolver is not recognizing django-storages[boto3]==1.10.1 as superseding the django-storages>=1.6.

Edit: removing the [boto3] specification in my requirements.in file does allow the dependencies to resolve (so my requirements.in file now reads

django-storages==1.10.1
boto3==1.16.61
collectfast==2.2.0

Not a catastrophic work-around, but I'd prefer to avoid the extra install if I could.

[aberres]

I am using pip-tools and pip-compile to generate a requirements.txt file. In the
...
Not a catastrophic work-around, but I'd prefer to avoid the extra install if I could.

We ran into this as well and came up with the same solution.

Is there any open issue to tackle this specific situation?

[pradyunsg]

If you're using the resolver on a fully pinned package set, you can use --no-deps to avoid it doing pedantic dependency resolution.

[aberres]

If you're using the resolver on a fully pinned package set, you can use --no-deps to avoid it doing pedantic dependency resolution.

Great, I can confirm this fixes our problem.

Thanks a million!

[pradyunsg]

The implementation-related fix is being tracked in #9243.