infra: Improve testing usability

[DropD]

Improve testing usability

mark slow tests and deselect them by default (but not on CI)

Mark the test running AiiDA as slow. Change the default test args to include -m "not slow", which can be overridden by passing -m "".

Now to run the fast tests during development we can use the default

hatch test

Whereas on CI or to manually run everything we use

hatch test -m ""

which overrides any selection based on marks (consequently selecting everything).

ignore warnings from libraries and default to treating warnings as errors

This will save us from building up third party warnings from code out of our control in the test output until the actual test results get pushed out of the console history (tends to happen in python projects over time).
Now we have to decide for every warning we get, whether we can / need to deal with it or should ignore it explicitly in the config.

cmdline option for regenerating serialized references

The skipped tests which could be manually un-skipped to update references had their logic moved into a config hook for pytest, which checks for a custom cmdline option. We can now:

hatch test --reserialize

to regenerate everything (can be differenciated in the future by type of data if regenerating everything ever becomes too slow).

parallelize tests on CI

--parallel option was added to hatch test on CI (github runners can typically not run many workers, so the speedup is probably small)

[DropD]

now hatch test does not run doctests, but hatch test -v does, while neither runs the "slow" tests.

[leclairm]

now hatch test does not run doctests, but hatch test -v does, while neither runs the "slow" tests.

Some observations:

• hatch test -v enables the doctests again
• setting extra-args = ["--doctest-modules", '-m not slow', '-v'] doesn't help, doctests are still skipped while the output is verbose.
• Now, still with -v in the extra-args, running hatch test -v, which should make no difference, does enable the doctests again.

So, apparently, appending an extra arg (probably random) re-enables doctests

[leclairm]

Also hatch test --doctest-modules activates the doctests as expected

[leclairm]

Finally found workarounds there and there. This seems to work:

default-args = []
extra-args = ["--doctest-modules", "-m not slow"]

[leclairm]

Fix the doctest issue, potentially with my proposition. Otherwise all good.

[agoscinski]

I am happy with it but I did not rigorously review it, since @leclairm already had a look

[agoscinski]

ah super nice that hatch already comes with pytest-xdist and coverage

[DropD]

Finally found workarounds there and there. This seems to work:

default-args = []
extra-args = ["--doctest-modules", "-m not slow"]

I had deleted that line without checking whether it was needed. Turns out it was, thanks for finding the source of the problem, @leclairm.