Cryptic error when a dependency's PKG_INFO/METADATA is missing

[nvictus]

• Pip version: 9.0.1
• Python version: 3.6.4
• Operating system: Ubuntu 17.10

Description:

Pip crashes with a cryptic error when it doesn't find a dependency's package metadata.

What I've run:

A certain dependency was installed using conda and was lacking a PKG-INFO file in the egg-info. When installing the desired package, pip fails right after processing the dependency with this error traceback:

Exception:
Traceback (most recent call last):
  File "/home/nezar/miniconda3/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/nezar/miniconda3/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/home/nezar/miniconda3/lib/python3.6/site-packages/pip/wheel.py", line 756, in build
    self.requirement_set.prepare_files(self.finder)
  File "/home/nezar/miniconda3/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/home/nezar/miniconda3/lib/python3.6/site-packages/pip/req/req_set.py", line 666, in _prepare_file
    check_dist_requires_python(dist)
  File "/home/nezar/miniconda3/lib/python3.6/site-packages/pip/utils/packaging.py", line 48, in check_dist_requires_python
    feed_parser.feed(metadata)
  File "/home/nezar/miniconda3/lib/python3.6/email/feedparser.py", line 175, in feed
    self._input.push(data)
  File "/home/nezar/miniconda3/lib/python3.6/email/feedparser.py", line 103, in push
    self._partial.write(data)
TypeError: string argument expected, got 'NoneType'

On inspecting pip/utils/packaging.py:

def get_metadata(dist):
    if (isinstance(dist, pkg_resources.DistInfoDistribution) and
            dist.has_metadata('METADATA')):
        return dist.get_metadata('METADATA')
    elif dist.has_metadata('PKG-INFO'):
        return dist.get_metadata('PKG-INFO')


def check_dist_requires_python(dist):
    metadata = get_metadata(dist)
    feed_parser = FeedParser()
    feed_parser.feed(metadata)
    pkg_info_dict = feed_parser.close()
    requires_python = pkg_info_dict.get('Requires-Python')
    try:
        if not check_requires_python(requires_python):
            raise exceptions.UnsupportedPythonVersion(
                "%s requires Python '%s' but the running Python is %s" % (
                    dist.project_name,
                    requires_python,
                    '.'.join(map(str, sys.version_info[:3])),)
            )
    except specifiers.InvalidSpecifier as e:
        logger.warning(
            "Package %s has an invalid Requires-Python entry %s - %s" % (
                dist.project_name, requires_python, e))
        return

get_metadata is returning None when it doesn't find anything, which causes feed_parser to crash.

[pradyunsg]

@cytolentino Hi!

Could you take a look at this issue? There should probably be a conditional somewhere, that prevents us from passing None to feed_parser, aborting early. :)

[cytolentino]

Hi @pradyunsg! I've just had a look at this and it seems like someone else has already fixed this (see here). That said, I've added in a regression test for this case to confirm that it's fixed over here: PR #6003 :)

[cjerdonek]

Marking this as a bug because pip shouldn't necessarily be crashing here (at least sometimes I don't think).

[cjerdonek]

I posted a fix for this (PR #6542). However, I only addressed the "enhancement" aspect (better error message) because in working on the patch I found that it should probably continue to error out (at least for now).

If there are other get_metadata() callers in the code base that shouldn't be erroring out in this situation, that should probably be handled on a case-by-case basis in different issues (when they are filed). There are many callers of get_metadata(), so I don't think we'd want to do a blanket "fix."