Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run all tests in release build in long suite on master merge #5360

Open
derekbruening opened this issue Feb 16, 2022 · 2 comments
Open

Run all tests in release build in long suite on master merge #5360

derekbruening opened this issue Feb 16, 2022 · 2 comments

Comments

@derekbruening
Copy link
Contributor

Historically we've been focused on debug-build tests, under the assmption that it is pretty rare for an issue to happen only in release build. Debug build has extra checks that catch more errors. Today we run only a subset of tests in release build, based on that assumption and to keep the PR CI time down. But: we have a separate "long" suite on master merge where the time doesn't matter as much. We should enable all the tests in release-build there.

This is becoming more important as we have lately been seeing release-build-only failures: #4928, #4954, #2417 drcachesim.coherence.

@derekbruening
Copy link
Contributor Author

Xref #4800, #1807

@derekbruening
Copy link
Contributor Author

For the long suite x86_64 today we're running just over half the tests in release vs debug:

debug-internal-64: 791 tests passed, **** 7 tests failed, of which 2 were flaky, but ignoring 5 for i#2941: ****
release-external-64: all 397 tests passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant