Using pytest-testmon plugin.

[prady0t]

pytest-testmon is a very useful pytest plugin. It runs only those tests where the related changes have been done. It can be helpful as an option for local development and CI as it would be significantly faster (since we are running a lot lesser tests).

[prady0t]

Originally suggested by @agriyakhetarpal

[kratman]

Something like that is fine for local dev, but we should test everything for CI. Dependencies can change, we have a complex codebase, and I am not sure anyone really looks at the periodic tests.

I don't think this is worth the effort to implement

[agriyakhetarpal]

I agree with Eric. We would like to always test everything in CI, but please feel free to test it out over a few months on your fork and see how reliable it is.

In the meantime, documentation on how to set it up and run it so that someone can test the changes they've made in a local feature branch (maybe through a nox session or a positional argument to the tests session) would be helpful, such that it picks up the common ancestor between the default branch and said branch automatically.

[kratman]

I think the other thing is that tests don't take that long locally. Running the unit tests takes about a minute. We don't need to go overboard with infrastructure or documentation

[agriyakhetarpal]

Yeah, fair point – the integration tests have usually passed for me in two minutes, and I expect this to be the same on modern computers running Windows and Linux as well. Unless @prady0t has something else to suggest, we could close this.

(P.S. I know I was the one to originally suggest it, but times have changed and pytest-testmon still remains experimental like it was back then).

[prady0t]

Yes I guess it's not much needed then. This would have anyways made sense in local development but have never personally felt the need that much. Users already have an option to run selected tests via nox -s tests -- "path". I guess we should close this.

[agriyakhetarpal]

Closing as "not planned" for tracking this better down the line