-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix FreeBSD (incl. CI) reporting swallowing errors on reruns #13156
Conversation
Turns out, when your test-suite fails with Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory that's a bad thing!! Fixes: 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Signed-off-by: Ahelenia Ziemiańska <[email protected]>
1439236
to
5477673
Compare
This passes, it's all green: http://build.zfsonlinux.org/builders/FreeBSD%20stable%2F13%20amd64%20%28TEST%29/builds/3961/steps/shell_4/logs/summary, it has 7 FAILs, among them receive-o-x_props_override, which is not ignored, and is caught correctly, e.g., here: http://build.zfsonlinux.org/builders/FreeBSD%20stable%2F13%20amd64%20%28TEST%29/builds/3968/steps/shell_4/logs/summary. I don't actually know what broke it, because, well: FreeBSD CI was unreliable at best since 2320e6e. @behlendorf it is for this reason I'd like to (modestly, it's just CI, but I cannot possibly tell you how stark raving mad this has driven me over the past however-long I've been trying to even remotely understand why #12996 is failing in stuff I haven't vaguely touched) fast-track this so we do actually catch new test failures on FreeBSD. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's unfortunate. It seems we didn't catch this initially since the GitHub Actions builders only test Linux, and then when we enabled it 3 weeks ago for the other CI builders in openzfs/zfs-buildbot#243 we didn't notice the errors in the FreeBSD log. Let's just wait for the CI to finish with it.
(Not worrying about largest_pool_001_pos or receive -o/-x on FBSD, because those appear to be broken on vanilla and not noticed because openzfs#13156 isn't landed yet.) Signed-off-by: Rich Ercolani <[email protected]>
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
(Not worrying about largest_pool_001_pos or receive -o/-x on FBSD, because those appear to be broken on vanilla and not noticed because openzfs#13156 isn't landed yet.) Signed-off-by: Rich Ercolani <[email protected]>
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Turns out, when your test-suite fails on FreeBSD the rerun logic would fail as follows: Results Summary PASS 1358 FAIL 7 SKIP 47 Running Time: 04:00:02 Percent passed: 96.2% Log directory: /var/tmp/test_results/20220225T092538 mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix mktemp: illegal option -- p usage: mktemp [-d] [-q] [-t prefix] [-u] template ... mktemp [-d] [-q] [-u] -t prefix /usr/local/share/zfs/zfs-tests.sh: cannot create : No such file or directory ... This change resolves a flaw from the original commit, 2320e6e ("Add zfs-test facility to automatically rerun failing tests") Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13156
Motivation and Context
Turns out, when your test-suite fails with
and succeeds, that's a bad thing!!
Description
Make it not do that. Hopefully. I'm delirious and there's a like a war or something next door.
How Has This Been Tested?
This has driven me crazy in #12996 (comment) as I tried to bisect it on CI, so if the FreeBSD CI fails then it's good.
Types of changes
Checklist:
Signed-off-by
.