Use pkgutil.iter_modules to get un-imported module for test_pre_import

[charlesbluca]

Changed this because IIUC pkg_resources.working_set is listing the installed distributions and not necessarily the importable modules; this becomes an issue if the distribution and module names aren't the same (e.g. one would conda install pillow and then import PIL), which was causing some failures in CI that seem unrelated to the changes here.

Originally posted by @charlesbluca in #981 (comment)

[jakirkham]

Wonder if we can do this with importlib.metadata instead

Note that there is a backport package importlib-metadata for anything that is not yet included in the Python standard library of the Python versions we support

[pentschev]

I think you are right indeed @charlesbluca , good catch and thanks for the investigation/fixing work! 😄

[pentschev]

/merge

[charlesbluca]

Converted to draft to try out @jakirkham's suggestion of using importlib.metadata/importlib_metadata instead

[charlesbluca]

After doing some digging into importlib.metadata's API, I wasn't able to find a suitable method that could grab a list of all working import packages; the closest thing I could find was packages_distributions(), but at least locally this seems to list some non-importable package names:

>>> importlib_metadata.packages_distributions()
{'... 'docker': ['ucx-py'], ...}
>>> import docker
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'docker'

Since this issue is blocking CI at the moment, and my local testing of pkgutil.iter_modules() seems to show that it doesn't have this issue, I'm inclined to go ahead and merge this and revisit if it becomes a problem again. Though if you have a specific API in mind here would be happy to open up / review a follow up 🙂

[charlesbluca]

/merge