[2020-resolver & fast-deps] BadZipFile when force-reinstalling packages #8701
Comments
triage-new-issues
bot
added
the
S: needs triage
|
Thank you very much for reporting this! However, I can reproduce this neither the latest release 20.2.1 nor the current master ee4371c. Furthermore, there seems to be a typo in version you listed, 5a61475 is part of GH-6236 which was merged on 2019-02-05 and was released as pip 19.0.2. The traceback you provided from forcing reinstalling setuptools and pip suggests that the implementation of the wheel over HTTP range requests thing got some segment merging done wrong, but it would be difficult to debug without failure occurring. It would be really nice if you can additionally see if the crash is reproducible on pip 20.2.0 and 20.2.1. |
pradyunsg
added
kind: crash
triage-new-issues
bot
removed
the
S: needs triage
|
Would you be able to locate |
$ pip cache info
Location: /home/akaihola/.cache/pip/wheels
Size: 0 bytes
Number of wheels: 0It seems wheels aren't stored, but HTTP responses are. I believe I found the correct one in Here's a link to a Dropbox folder with two files:
However, it seems that the differences between these files don't matter, because this fails: and this succeeds: So based on this, the cached response is always written correctly, but |
|
I also reproduced this on the
#!/bin/bash
python -m pip install -U "pip @ https://github.com/pypa/pip/archive/master.zip"
pip install black
pip config set --site global.use-feature "2020-resolver fast-deps"
pip install --force-reinstall blackI then ran that in the image using Podman: |
|
Thank you @akaihola, this is really helpful, I'll investigate this ASAP. Until then, I want to note that the |
|
The problem seem to be with incorrect (?) caching of range responses (please have my apologies, @akaihola, for putting you through creating a container for this—I have diff --git a/src/pip/_internal/network/lazy_wheel.py b/src/pip/_internal/network/lazy_wheel.py
index 16be0d29..c03eaeb8 100644
--- a/src/pip/_internal/network/lazy_wheel.py
+++ b/src/pip/_internal/network/lazy_wheel.py
@@ -194,7 +194,8 @@ class LazyZipOverHTTP(object):
def _stream_response(self, start, end, base_headers=HEADERS):
# type: (int, int, Dict[str, str]) -> Response
"""Return HTTP response to a range request from start to end."""
- headers = {'Range': 'bytes={}-{}'.format(start, end)}
+ headers = {'Range': 'bytes={}-{}'.format(start, end),
+ 'Cache-Control': 'no-cache'}
headers.update(base_headers)
return self._session.get(self._url, headers=headers, stream=True) |
162: Update pip to 20.2.2 r=duckinator a=pyup-bot This PR updates [pip](https://pypi.org/project/pip) from **20.2.1** to **20.2.2**. <details> <summary>Changelog</summary> ### 20.2.2 ``` =================== Bug Fixes --------- - Only attempt to use the keyring once and if it fails, don't try again. This prevents spamming users with several keyring unlock prompts when they cannot unlock or don't want to do so. (`8090 <https://github.com/pypa/pip/issues/8090>`_) - Fix regression that distributions in system site-packages are not correctly found when a virtual environment is configured with ``system-site-packages`` on. (`8695 <https://github.com/pypa/pip/issues/8695>`_) - Disable caching for range requests, which causes corrupted wheels when pip tries to obtain metadata using the feature ``fast-deps``. (`8701 <https://github.com/pypa/pip/issues/8701>`_, `8716 <https://github.com/pypa/pip/issues/8716>`_) - Always use UTF-8 to read ``pyvenv.cfg`` to match the built-in ``venv``. (`8717 <https://github.com/pypa/pip/issues/8717>`_) - 2020 Resolver: Correctly handle marker evaluation in constraints and exclude them if their markers do not match the current environment. (`8724 <https://github.com/pypa/pip/issues/8724>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pip - Changelog: https://pyup.io/changelogs/pip/ - Homepage: https://pip.pypa.io/ </details> Co-authored-by: pyup-bot <[email protected]>
Environment
Description
Force-reinstalling a cached package causes pip to fail if
use-feature = 2020-resolver fast-depsboth are enabled.I saw this behavior with Black, pip itself, wheel and setuptools. The example below is for Black.
If only either
2020-resolverorfast-depsis enabled, everything works correctly.Expected behavior
The package should be reinstalled successfully.
How to Reproduce
pip config set --site global.use-feature "2020-resolver fast-deps"--force-reinstalloptionOutput
The final error for the other packages I tried was:
The text was updated successfully, but these errors were encountered: