./pants binary can fail with unhandled wheel.install.BadWheelFile: No expected hash for file ...

[kwlzn]

seen during release testing:

...
11:07:03 00:04   [pyprep]
11:07:03 00:04     [interpreter]
11:07:03 00:04     [requirements]
                   Invalidated 16 targets.
               Waiting for background workers to finish.
11:07:15 00:16   [complete]
               FAILURE
Exception caught: (<class 'wheel.install.BadWheelFile'>)
  File ".bootstrap/_pex/pex.py", line 360, in execute
    self._wrap_coverage(self._wrap_profiling, self._execute)
  File ".bootstrap/_pex/pex.py", line 288, in _wrap_coverage
    runner(*args)
  File ".bootstrap/_pex/pex.py", line 320, in _wrap_profiling
    runner(*args)
  File ".bootstrap/_pex/pex.py", line 403, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File ".bootstrap/_pex/pex.py", line 461, in execute_entry
    return runner(entry_point)
  File ".bootstrap/_pex/pex.py", line 479, in execute_pkg_resources
    return runner()
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/bin/pants_exe.py", line 26, in main
    PantsRunner(exiter).run()
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/bin/pants_runner.py", line 57, in run
    options_bootstrapper=options_bootstrapper)
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/bin/pants_runner.py", line 46, in _run
    return LocalPantsRunner(exiter, args, env, options_bootstrapper=options_bootstrapper).run()
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/bin/local_pants_runner.py", line 37, in run
    self._run()
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/bin/local_pants_runner.py", line 79, in _run
    goal_runner_result = goal_runner.run()
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/bin/goal_runner.py", line 263, in run
    result = self._execute_engine()
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/bin/goal_runner.py", line 252, in _execute_engine
    result = engine.execute(self._context, self._goals)
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/engine/legacy_engine.py", line 26, in execute
    self.attempt(context, goals)
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/engine/round_engine.py", line 224, in attempt
    goal_executor.attempt(explain)
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/engine/round_engine.py", line 47, in attempt
    task.execute()
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/backend/python/tasks2/resolve_requirements.py", line 22, in execute
    pex = self.resolve_requirements(req_libs)
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/backend/python/tasks2/resolve_requirements_task_base.py", line 50, in resolve_requirements
    self._build_requirements_pex(interpreter, safe_path, req_libs)
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/backend/python/tasks2/resolve_requirements_task_base.py", line 55, in _build_requirements_pex
    dump_requirements(builder, interpreter, req_libs, self.context.log)
  File "/Users/kwilson/dev/source/.pex/install/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl.b8215da13073fb89c49d2e20a9b08740bfe78827/pantsbuild.pants-1.4.0.dev8+1680375244-py2-none-any.whl/pants/backend/python/tasks2/pex_build_util.py", line 106, in dump_requirements
    builder.add_distribution(dist)
  File "/Users/kwilson/dev/source/.pex/install/pex-1.2.8-py2.py3-none-any.whl.c061a88b75ba16e401a0dc63f783fe83b04ce8ff/pex-1.2.8-py2.py3-none-any.whl/pex/pex_builder.py", line 321, in add_distribution
    dist_hash = self._add_dist_zip(dist.location, dist_name)
  File "/Users/kwilson/dev/source/.pex/install/pex-1.2.8-py2.py3-none-any.whl.c061a88b75ba16e401a0dc63f783fe83b04ce8ff/pex-1.2.8-py2.py3-none-any.whl/pex/pex_builder.py", line 284, in _add_dist_zip
    wf.install(overrides=self._get_installer_paths(whltmp), force=True)
  File "/Users/kwilson/dev/source/.pex/install/wheel-0.29.0-py2.py3-none-any.whl.c6b9e44d951cdabf4dc67205b0f30184a1b602bb/wheel-0.29.0-py2.py3-none-any.whl/wheel/install.py", line 338, in install
    source = self.zipfile.open(info)
  File "/Users/kwilson/dev/source/.pex/install/wheel-0.29.0-py2.py3-none-any.whl.c6b9e44d951cdabf4dc67205b0f30184a1b602bb/wheel-0.29.0-py2.py3-none-any.whl/wheel/install.py", line 466, in open
    raise BadWheelFile("No expected hash for file %r" % ef.name)

Exception message: No expected hash for file 'packages/future-0.15.2-py2-none-any.whl'

re-running the ./pants binary produces a different filename each time:

Exception message: No expected hash for file 'packages/certifi-2017.4.17-py2.py3-none-any.whl'
Exception message: No expected hash for file 'packages/pytz-2017.2-py2.py3-none-any.whl'
Exception message: No expected hash for file 'packages/flask_swagger-0.2.13-py2-none-any.whl'
Exception message: No expected hash for file 'packages/tabulate-0.7.7-py2.py3-none-any.whl'

related to pex-tool/pex#388 and #4735 which consumes it.

[alecbz]

Is there any recourse if we run up against this? I'm not super well-versed on the innards of the wheel format, but is the best option to just construct a wheel that doesn't have whatever problem causes this?

On a related note: is the issue here that a wheel is 'bad' in some way, or is pants/pex expecting more from a wheel than the wheel "standard" (to the extent that there even is one?) guarantees?

[jsirois]

@AlecBenzer there currently is no way around this. The issue is a bad wheel - they should all have a RECORD file in their metadata with file hashes: https://www.python.org/dev/peps/pep-0427/#the-dist-info-directory Pex here is actually using wheel's own code to install wheels. This issue comes before the changes in pex 1.6.x that vendor setuptools and wheel. It could be that, in the past, pex could get in a state with bad wheels if the un-vendored version of wheel used was "bad". This is a wild guess.

[Eric-Arellano]

I believe this should be fixed by now using Pex 2.0, which uses Pip instead of Pex to handle wheels.

If anyone still encounters this error on a more recent version of Pants, please feel free to reopen this ticket.