pip attempt to uninstall a system package when installing the same package in --prefix

[perillo]

Description

When installing a package that also exists in the system site packages to --prefix, pip tries to delete the system package.
This does not happen when installing using a virtual environment.

See also #4337.

An additional problem is that pip reports on stderr:

ERROR: Could not install packages due to an OSError.
Consider using the `--user` option or check the permissions.

This may confuse some users, since --user and --prefix are incompatible.

See also the Invalid cross-device link error.

Thanks.

Expected behavior

Installing collected packages: flit
  Attempting uninstall: flit
    Found existing installation: flit 3.9.0
    Not uninstalling flit at /usr/lib/python3.11/site-packages, outside environment /tmp/000
    Can't uninstall 'flit'. No files were found to uninstall.
  changing mode of /tmp/000/bin/flit to 755
Successfully installed flit-3.9.0

This is the message when installing to a virtual environment:
$ pip --python /tmp/000 install .

pip version

23.3.1

Python version

3.11.6

OS

Linux

How to Reproduce

Assuming flit is installed in the system site packages:

$ git clone https://github.com/pypa/flit.git
$ cd flit
$ pip install --prefix /tmp/prefix .

pip.conf:

[global]
verbose = 1

[install]
# Arch Linux always have the latest packages
no-build-isolation = false

Output

Using pip 23.3.1 from /usr/lib/python3.11/site-packages/pip (python 3.11)
Processing /code/git/github.com/pypa/flit
  Running command Preparing metadata (pyproject.toml)
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: flit_core>=3.9.0 in /usr/lib/python3.11/site-packages (from flit==3.9.0) (3.9.0)
Requirement already satisfied: requests in /usr/lib/python3.11/site-packages (from flit==3.9.0) (2.31.0)
Requirement already satisfied: docutils in /usr/lib/python3.11/site-packages (from flit==3.9.0) (0.20.1)
Requirement already satisfied: tomli-w in /usr/lib/python3.11/site-packages (from flit==3.9.0) (1.0.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/python3.11/site-packages (from requests->flit==3.9.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.11/site-packages (from requests->flit==3.9.0) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3.11/site-packages (from requests->flit==3.9.0) (1.26.18)
Building wheels for collected packages: flit
  Running command Building wheel for flit (pyproject.toml)
  Building wheel for flit (pyproject.toml) ... done
  Created wheel for flit: filename=flit-3.9.0-py3-none-any.whl size=50538 sha256=ff026101c73cb9950dd03794fd7625ea305b917b986fbab43806879634b4e058
  Stored in directory: /tmp/pip-ephem-wheel-cache-j826qwqg/wheels/35/77/ff/b353c2e24da9f2c4f836bc47a66e6077ecbeae68c7280ce1f1
Successfully built flit
Installing collected packages: flit
  Attempting uninstall: flit
    Found existing installation: flit 3.9.0
    Uninstalling flit-3.9.0:
ERROR: Could not install packages due to an OSError.
Consider using the `--user` option or check the permissions.
Traceback (most recent call last):
  File "/usr/lib/python3.11/shutil.py", line 825, in move
    os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/usr/bin/flit' -> '/tmp/pip-uninstall-kuhdsrsc/flit'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 452, in run
    installed = install_given_reqs(
                ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pip/_internal/req/__init__.py", line 67, in install_given_reqs
    uninstalled_pathset = requirement.uninstall(auto_confirm=True)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 725, in uninstall
    uninstalled_pathset.remove(auto_confirm, verbose)
  File "/usr/lib/python3.11/site-packages/pip/_internal/req/req_uninstall.py", line 381, in remove
    moved.stash(path)
  File "/usr/lib/python3.11/site-packages/pip/_internal/req/req_uninstall.py", line 272, in stash
    renames(path, new_path)
  File "/usr/lib/python3.11/site-packages/pip/_internal/utils/misc.py", line 362, in renames
    shutil.move(old, new)
  File "/usr/lib/python3.11/shutil.py", line 846, in move
    os.unlink(src)
PermissionError: [Errno 13] Permission denied: '/usr/bin/flit'

Code of Conduct

• I agree to follow the PSF Code of Conduct.