Suggest package names for undeclared imports (when possible)

[jherland]

(This comes from a discussion with @layus on Slack)

Currently, all undeclared dependencies are reported using their import name. More specifically: declared dependencies are mapped via Pyhon packages into a set of provided import names, and any actual 3rd-party imports that fall outside this set are reported as undeclared dependencies.

We currently don't "reverse-map" these import names into the Python package names that could/would provide these import names.

Nor can we do so - in the general case:

• The package that provides a given import name might not be available in the local Python environment.
• There might not even exist a package that provides the import name; e.g. the import name might be misspelled.
• There might be multiple Python packages that provides this import name, and we have no way of choosing between them.

However, in the common case, there would usually be a single candidate package available in the local Python environment that provides the undeclared import name, and in this case we should report this package name to the user, to indicate the package name that they probably want to add to their declared dependencies.

In the case where multiple local packages are found to provide the same import name, we should report all of them as candidates, as we have no way of selecting the most appropriate choice.

Potential example output:

Undeclared imports:
  skimage  # No candidate package found

or

Undeclared imports:
  skimage
    # hint: this import is provided by package scikit_image
    # hint: this import is provided by package other_skimage_stuff

[layus]

Keeping notes in this thread:

This endeavor is very close to what rules_python gazel plugin provides in bazel.
The core extraction is done here https://github.com/bazelbuild/rules_python/blob/main/gazelle/modules_mapping/generator.py where .whl files are analyzed to find all the import paths that they provide.
I think the same can be done in fawltydeps to get more precise dependency tracking.

A lot of different packages can provide the same toplevel import. I winder why fawltydeps stop at the toplevel.

Once we have that mapping, it becomes easier to suggest dependencies.

[layus]

@jherland I have a patch that does the bulk of the work. It is surprisingly easy. All that remains is to print the suggestions based on the extra data attached to the analysis.

https://gist.github.com/layus/bbb1a4c699f3d48ff18f08ec76486558

PS: The patch comes bundled with some other, unrelated changes

[jherland]

Thanks! I took the liberty of taking the relevant parts of your patch and making an initial commit in your name here: #470. Following up with some refactoring to keep things maintainable...