Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a test collection workflow pre-step #2098

Merged
merged 7 commits into from
Dec 21, 2022

Conversation

ThomasLaPiana
Copy link
Contributor

@ThomasLaPiana ThomasLaPiana commented Dec 21, 2022

Closes #2033

Code Changes

  • add a nox command to collect tests
  • add a workflow job that runs the test collection nox session
  • make all test-related workflows conditional on the Collect-Tests workflow step via the Build step

Steps to Confirm

  • run nox -s collect_tests locally (first run will take awhile as it caches the installations)
  • verify the Collect-Tests job runs as expected and the output looks correct
  • verify that if the Collect-Tests doesn't pass, the pytest jobs don't get run

Pre-Merge Checklist

Description Of Changes

This is meant to be a straightforward PR that will serve to both

  1. Transparently show code coverage for the entire test suite on every run (at least when analyzed statically)
  2. Fail fast if there are any test collection errors and block the expensive pytest jobs from running

@ThomasLaPiana ThomasLaPiana self-assigned this Dec 21, 2022
@ThomasLaPiana
Copy link
Contributor Author

the above CI check shows that when the collection passes, everything runs as usual.

I'll now purposefully break the collection to confirm that the tests wont run

@ThomasLaPiana
Copy link
Contributor Author

ThomasLaPiana commented Dec 21, 2022

the above CI run should break and prevent pytests from running

edit: note that they were skipped as expected

@ThomasLaPiana ThomasLaPiana marked this pull request as ready for review December 21, 2022 16:58
Copy link
Contributor

@sanders41 sanders41 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

This is good like it is, just a question. Would it be worth it to move the collection test to the docker build since that step is what takes longest to run and the pytest steps won't run without build?

@ThomasLaPiana
Copy link
Contributor Author

LGTM!

This is good like it is, just a question. Would it be worth it to move the collection test to the docker build since that step is what takes longest to run and the pytest steps won't run without build?

Oh, that makes way more sense! Ok, i'll make build rely on collect-tests

@sanders41
Copy link
Contributor

The skipped and failed tests here look to me to be github network related and not PR related. Restarting to see if that fixes it.

Copy link
Contributor

@sanders41 sanders41 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure again looks unrelated to the PR

@ThomasLaPiana ThomasLaPiana merged commit bf91022 into main Dec 21, 2022
@ThomasLaPiana ThomasLaPiana deleted the ThomasLaPiana-add-collect-workflow branch December 21, 2022 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a pytest collection CI workflow
2 participants