downloads 50 different versions of setuptools and eventually fails with a cryptic syntax error

[tetron]

What did you want to do?

On Ubuntu 16.04, and Python 3.5.2, I created a python3 virtualenv and ran this command:

$ pip install toil[cwl]

Output

The install downloads 50 different versions of setuptools and eventually fails with a cryptic syntax error:

piplog.txt

Additional information

It turns out the toil package doesn't actually support Python 3.5. With the legacy resolver, it figures that out:

$ pip --use-deprecated=legacy-resolver install toil[cwl] 
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
Collecting toil[cwl]
  Using cached toil-5.0.0-py3-none-any.whl (497 kB)
ERROR: Package 'toil' requires a different Python: 3.5.2 not in '>=3.6'

I would expect the new resolver to fast-fail the same way the legacy resolver did.

[notatallshaw]

So this specific issue will almost certainly be fixed by this: #9249

But I would like to point out to the pip maintainers that this highlights a key problem the resolver strategy, which has already been discussed a few times but I honestly I believe will increasingly flood this GitHub's issue page so I want to reiterate:

1. It keeps pulling packages to an old enough date where the assumptions that they had then no longer hold (or there was an error in a specific release). Looking at setuptools 0.7.8 it seems that it expect 2to3 to be run on all the code before Python 3 can run it. How it expects that to happen (automatically or manually) I am not sure, but apparently back in 2013 this was a reasonable assumption.

2. Pip errors out when failing to parse the metadata rather than generically catching all errors and discarding versions that fail for whatever reason. This makes sense when you're only pulling the latest but because of the above packages can go back so far in time that assumptions no longer hold and unexpected errors can happen.

[tetron]

I found the fact that it actually goes and starts downloading older packages instead of having that metadata available in PyPi kind of concerning, although perhaps it's making the best of a bad situation.

[pfmoore]

It keeps pulling packages to an old enough date where the assumptions that they had then no longer hold (or there was an error in a specific release).

Agreed. The problem is that it's not practical to know what counts as "old enough". Some stable packages haven't been updated in quite a time. (Maybe not as far back as 2013, but 🤷)

Pip errors out when failing to parse the metadata rather than generically catching all errors and discarding versions that fail for whatever reason.

That might be something we could do. Again, the problem is knowing when we're looking at something "old enough" to switch to that option.

I found the fact that it actually goes and starts downloading older packages instead of having that metadata available in PyPi kind of concerning, although perhaps it's making the best of a bad situation.

Precisely that. We're working on encouraging packages not to calculate metadata at build time, and once we have that we can look at exposing that metadata on PyPI, but it's been a multi-year project so far, so it's not going to happen soon 🙁

[notatallshaw]

Agreed. The problem is that it's not practical to know what counts as "old enough". Some stable packages haven't been updated in quite a time. (Maybe not as far back as 2013, but 🤷)

Totally understand this is basically an undefinable and as a fellow coder I'm very sympathetic to undefinable requirements.

But this is the strategy pip has decided to take so from my naive point of view with no appreciation of complexity of implementation it seems to me if the issues is common enough that pip will continue to receive more and more reports then some arbitrary decisions might have to taken (max depth parameter?) or a change of strategy (BFS instead of DFS?).

Pip errors out when failing to parse the metadata rather than generically catching all errors and discarding versions that fail for whatever reason.

That might be something we could do.

I find it truly miraculous that pip is able to run arbitrary code from so many of package versions (10'000s, 100'000s, millions?) and not have these errors far more frequently. So maybe I'm wrong here and each package-version should be yanked or fixed with constraints.

But taking this issue as example, let's say pip doesn't find a way to fail fast and no heuristic is implemented to give up before a certain depth or age of package, as is the case of the time of reporting. Then what is the right course of action?

• toil add a constraint on setuptools? (and do so retroactively for previous releases that users may still want to install?)
• setuptools yank all packages 0.7.8 and older?

To me it feels like either of these solutions aren't ideal.

To me it feels that it's something pip must attempt to solve to some extent (generically catching errors / finding new ways to fail fast / heuristics to fail early). But maybe I'm over-blowing what is a niche issue, I'm again surprised this isn't far more common.

[uranusjr]

Duplicate to #9246 and #9187 (comment)