21.3 has broken pip-tools (specifically pip-sync): ImportError: cannot import name 'get_installed_distributions' from 'pip._internal.utils.misc'

[adamcunnington-gdt]

Description

21.3, released last night, has broken pip-tools, specifically pip-sync:

. ".venv/bin/activate" && pip-sync extras/dev-requirements.txt requirements.txt

Traceback (most recent call last):
  File "/home/ac/Projects/adamcunnington.info/MLG/test/.venv/bin/pip-sync", line 5, in <module>
    from piptools.scripts.sync import cli
  File "/home/ac/Projects/adamcunnington.info/MLG/test/.venv/lib/python3.9/site-packages/piptools/scripts/sync.py", line 12, in <module>
    from pip._internal.utils.misc import get_installed_distributions
ImportError: cannot import name 'get_installed_distributions' from 'pip._internal.utils.misc' (/home/ac/Projects/adamcunnington.info/MLG/test/.venv/lib/python3.9/site-packages/pip/_internal/utils/misc.py)

Expected behavior

No response

pip version

21.3

Python version

3.9.5

OS

Debian

How to Reproduce

python -m venv .venv
pip install pip=21.3 pip-tools
echo "requests" > requirements.in
pip-sync

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

This is already discussed in the pip-tools repo.

[adamcunnington-gdt]

@uranusjr maybe you could link to that discussion next time.
For others; jazzband/pip-tools#1503
But discussing it there is a different thing. A minor version of pip has caused breaking change. Surely there is action to take here in the immediate term ahead of any accommodation in pip-tools.
EDIT: pip-tools is referencing an internals here so I guess it's a grey area... but pip surely would know which parts of the codebase are depended on by a large and popular tool

[uranusjr]

Pip uses calver, 21.3 means the fourth release in 2021. And pip does not expose any internals as stable API. pip-tools devs knows all these and build the package on top of pip regardless. pip maintainers don't object to the effort, but also don't go out of our way to support it.

Please consider reading the documentation and check issue trackers before you start making assumptions and pointing fingers next time. pip's documentation describe both the versioning and compatibility policies, and the pip-tools issue was created within moments of the 21.3 release and you should have noticed it had you check.

[pradyunsg]

pip-tools is referencing an internals here so I guess it's a grey area... but pip surely would know which parts of the codebase are depended on by a large and popular tool

We do, and the contract is the same still -- stuff inside pip._internal may change at any time, in any release, for any reason. It's the responsibility of tools that use _prefixed names to ensure that they're compatible with changes that are made in pip's API.

See https://pip.pypa.io/en/stable/user_guide/#using-pip-from-your-program for the details, and our position on the use of pip inside Python programs.