Improve 'module not found' message

[nicoddemus]

In #1567 it was men, user has spent a long time trying to figure out why their namespace package was not being found, reason being that pytest does not support namespace packages via --pyargs (#478).

One easy suggestion is to improve the error message, from:

ERROR: module or package not found: module.misc (missing __init__.py?)

To something like this:

ERROR: pytest cannot import module or package 'module.misc'
Possible reasons:
- Missing an __init__.py file
- Namespace package (not supported)
- Not reachable via PYTHONPATH
See <link to docs> for more information.

Bonus points if we link to a documentation where we have an explanation for each bullet point above.

Originally posted by @nicoddemus in #1567 (comment)

[dimpase]

Are there solid technical reasons for not supporting PEP 420, or it's just lack of labour?

[nicoddemus]

Are there solid technical reasons for not supporting PEP 420, or it's just lack of labour?

TBH the details are murky, I recall a very long discussion involving pytest maintainers and other members of the community (like @jaraco), and IIRC we could not arrive to a solution on how to support this in pytest.

Others who remember more details and/or have links to share please do so.

Some refs:
#478
#3396

[dimpase]

Our (for SageMath) main use of pytest would be to run doctests (in Python and in Cython). This can be done on per-file basis, in parallel (otherwise it takes many hours of wallclock time). But pytest makes it unnecessary hard to do things like pytest foo/*/* -- all these missing __init__.py errors. Currently we have a homemade tool which uses a bit of Python doctest module for this, but it's old and tired, and we really want an off the shelf solution instead.

[dimpase]

We'd like to ask for an option to provide a package discovery hook for namespace modules/packages (e.g. in our case (our homegrown doctest system also does Python module discovery, after all). That is to say, a pytest-approved way to monkey-patch https://github.com/pytest-dev/pytest/blob/main/src/_pytest/_py/path.py @mkoeppe

[RonnyPfannschmidt]

there will never be a way to monkeypatch this legacy path thing that we will remove as soon as we can as its only vendor-ed due to a need to drop pylib asap to begin with

as for what to do about the discovery of homegrown doctest, thats something else

pep440 awareness should help to fix most of the issues, for anything else we need to see the use-case to properly figure what api is necessary to provide

[RonnyPfannschmidt]

@nicoddemus i believe the key issue is that python collection currently works in terms of files, to properly collect in namespace packages, python collection needs to be module/package/import aware

as this would in general be a prerequisite for package shippable testsuites it may be a good idea to focus on "pythonic" python collection in then ext year

[dimpase]

or anything else we need to see the use-case to properly figure what api is necessary to provide

in SageMath directories containing namespace packages are recognised by presence of all.py or an all_*.py file (for some value of *). E.g. this contains a namespace package, as it has all.py (all.py may also contain import and lazy_import statements). Whereas that does not contain a Python package.
(there are also "classical" packages, with __init__.py, present).

[dimpase]

as for what to do about the discovery of homegrown doctest, thats something else

Our doctests are more or less standard, and in fact pytest is already mostly able to run them, on the level of individual files. That's not what we are worried about.
It's the framework to test them that's homegrown, and to be replaced by pytest.

[dimpase]

there will never be a way to monkeypatch this legacy path thing

OK, I see. Is this the place responsible for collecting tests pythonically?

pytest/src/_pytest/python.py

Line 747 in 9fbd67d

def collect(self) -> Iterable[Union[nodes.Item, nodes.Collector]]:

Is it stable - so that we can monkey-patch it without a fear that it'll break soon?

[RonnyPfannschmidt]

No promises