pip-compile choses yanked version which leads to pip install failure

[alexanderilyin]

Environment Versions

OS Type

$ devcontainer-info 

Development container image information

- Image version: 1.2.1
- Definition ID: python
- Variant: 3.12-bookworm
- Source code repository: https://github.com/devcontainers/images
- Source code release/branch: v0.4.10
- Timestamp: Thu, 12 Dec 2024 21:37:55 GMT

More info: https://github.com/devcontainers/images/tree/main/src/python/history/1.2.1.md

Python version: $ python -V

Python 3.12.8

pip version: $ pip --version

pip 24.3.1 from /usr/local/lib/python3.12/site-packages/pip (python 3.12)

pip-tools version: $ pip-compile --version

pip-compile, version 7.4.1

Steps to replicate

1. Create ./requirements.in:

# This file is used to generate requirements.txt.
# NOTE:
# Use Python 3.8 or greater which ever is the minimum version of the python
# you plan on supporting when creating the environment or using pip-tools.
# Only run the commands below to manully upgrade packages in requirements.txt:
# 1) python -m pip install pip-tools
# 2) pip-compile --generate-hashes --resolver=backtracking --upgrade ./requirements.in
# If you are using nox commands to setup or build package you don't need to
# run the above commands manually.

# Required packages
pygls
packaging

# https://test.pypi.org/project/partcad/
--extra-index-url https://test.pypi.org/simple/
partcad>=0.7.53

2. Run:

pip-compile --generate-hashes --resolver=backtracking --upgrade ./requirements.in

3. Check requirements.txt:

cadquery-ocp==7.7.2.1 \
    --hash=sha256:032e65afc9e2423eb1bd6feb7085a5d583f70c23c559156baf69383186489e7a \
    --hash=sha256:0b14c86b4abd8d91a6121770e6945c951b7b7a323812ab8463e5ef64ee399b94 \
    --hash=sha256:238a3917142353d8ee72b8240e27619eaece1f88882984be00336700a7f1d573 \
    --hash=sha256:275d1528d603fad2ba0dfdad0d4bfe5c6aab578609db3dcec989e65432a0ed6d \
    --hash=sha256:3126a01735d2826c28fb9f3b288f4adc98490caa08abb84adffb0cc7f47cc73e \
    --hash=sha256:51f17f1816adba9494d7adac9c4be385ed6934aeaa13cf64b943c5f09d2a0aaa \
    --hash=sha256:5d5fa1420eb7cf4243e3ce9253ec229bef5f7247d3a7939f32341cd0f5c613f6 \
    --hash=sha256:5dd52b25458753e60c30e83828e7ef270d6a6019b84cea075a48228b06cef94b \
    --hash=sha256:5f776a006e9a4b59ee6aa0fa64ad5efc22c1a9e7ee1d4e210de4113832f47096 \
    --hash=sha256:6e9f0dd7550f6881ed019154e3b7035dad5a50d3f3b43f81ad6e04546089723b \
    --hash=sha256:73bc366df6e2152529ec293f46b93983647446da96c18b9e93db03098db813e6 \
    --hash=sha256:78a122f58b886a0b8d7a98b674c5cb690865ba2a453894049105da879cbd2af8 \
    --hash=sha256:b14aa7e0b1ba6f8799501879a65b803a0ca56ad6e71d9d6cc3c31290ffaa683d \
    --hash=sha256:c59bafb6fe20722dad03ca7e4e79f40d680ec0b2054c2cd71cc427cd9d57ad05 \
    --hash=sha256:c89084a62e2d90f36065c0b738503e9e6746b7aeacad0a664aa0cba13628e534 \
    --hash=sha256:c990748c7bcf8307356ed5ccd6db1f086acb964918e8a22d9c5ff7026bf0cd1c \
    --hash=sha256:cddfd6272b5ecae60e5e6c2eabe46f5be8d31293c7cba562537062659fbfb81e \
    --hash=sha256:ce1e3c40b666ffa12a7f115333ddbf4785d1300fb6eabec678dcb8ca4240c949 \
    --hash=sha256:e68bd18f41fe497e4d721aec52dca228b4586d3fc0d35a814408468228e61fdf \
    --hash=sha256:e88eff3daf62886244a84304528818ca6a0de22e82071818b53f67e7965e31b9
    # via
    #   build123d
    #   ocpsvg

4. Run

python -m pip install -t ./bundled/libs --no-cache-dir --implementation py --no-deps --upgrade -r ./requirements.txt

Expected result

Actual result

...
Could not find a version that satisfies the requirement cadquery-ocp==7.7.2.1 (from versions: none)
No matching distribution found for cadquery-ocp==7.7.2.1

Notes

pip-compile run chose cadquery-ocp==7.7.2.1 which is yanked, which caused pip install to fail:

[webknjaz]

which caused pip install to fail:

Yanking is unlikely the reason. The installers are specifically supposed to allow installing yanked thinks if that specific version is requested via ==. There must be another incompatibility with your system, like the supported Python version.

[webknjaz]

There must be another incompatibility with your system, like the supported Python version.

Seeing that this version only has wheels published and no sdist, if you're on a platform that does not match any of those wheels, and pip cannot use the sdist to build the project for your environment, it'd end up showing that error, which is accurate and expected.

[webknjaz]

The reason the yanked version was chosen in the first place is unclear (both dependents don't pin it: https://inspector.pypi.io/project/build123d/0.8.0/packages/43/96/f29a26ac97080b2327ea59f6cabc136c89958e79923b147aab70c58e5395/build123d-0.8.0-py3-none-any.whl/build123d-0.8.0.dist-info/METADATA / https://inspector.pypi.io/project/ocpsvg/0.4.0/packages/24/e5/d9e5c49886eef625e1f674f23917746ff2aad021b6e80df67bc5edd180b2/ocpsvg-0.4.0-py3-none-any.whl/ocpsvg-0.4.0.dist-info/METADATA). #2089 implies that it's possible, while another maintainer said that it couldn't be reproduced.

It might've been that you're adding TestPyPI into the resolution, which is quite dangerous and is highly discouraged. It is possible that it adds things into the tree with weird metadata, steering the depresolver into making bad choices.

Somebody needs to dig deeper and come up with a good reproducer that could be made into a regression test.