Do we need to test notebooks in every commit of a new PR ?

[ismael-mendoza]

It occurred to me that we might not necessarily want to test notebooks every time we make a new commit in a PR (otherwise we might as well have it as a unit test). Since notebooks are important to maintain but it is not absolutely critical to check that they are not broken, I suggest to only use nbval when there is a push to the (soon to be made) dev branch rather than in every commit of a new PR.

[thuiop]

I'm bumping this issue ; actually, do we need to run the tests at every commit too ? As long as it remains mandatory that they pass before we merge, it should not be necessary to run them every time.

[aboucaud]

I agree that running tests for every commit pushed into a PR is overkill, especially when the installation is heavy.

But I think for external collaborators, the CI is hold until a maintainer asks for it (when we want to merge basically), which already helps a lot.

Concerning the notebooks, I guess we could make it a separate CI action that runs on a the main branch with a cron every week or so. That would be sufficient IMO.

[ismael-mendoza]

Yes that is exatly what I was thinking :) I'll work on that

[ismael-mendoza]

Closed by #263