Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deny manual tests in new PRs #34672

Closed
andy31415 opened this issue Jul 31, 2024 · 1 comment
Closed

Deny manual tests in new PRs #34672

andy31415 opened this issue Jul 31, 2024 · 1 comment
Assignees

Comments

@andy31415
Copy link
Contributor

andy31415 commented Jul 31, 2024

We should not allow manual CI steps in PRs. Manual tests are anything that requires the user to look at logs, generally identified via a verification tag in yaml files. For an example see #34614 (Test_TC_TSTAT_4_2.yaml has several manual steps in that PR)

Note that existing tests may already be manual so we cannot blanket-check all files. What we want is:

  • check against files changed in the PR only
  • allow a bypass label for the check (we could use a github label or for failing files check for some "why it is bypassed" comment in a specific format)
  • reject any yaml test files that have the text ^\s*verification: in them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

2 participants