ZTS: rsend tests #12876
Closed
ZTS: rsend tests #12876
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Signed-off-by: Brian Behlendorf <[email protected]>
behlendorf
added
Component: Test Suite
behlendorf
force-pushed
the
zfs-rsend
branch
from
64f1cb2 to
80956b5
Compare
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Signed-off-by: Brian Behlendorf <[email protected]>
behlendorf
force-pushed
the
zfs-rsend
branch
from
80956b5 to
c565cae
Compare
ghost
approved these changes
Dec 21, 2021
| @@ -121,6 +121,7 @@ function cleanup_pool | |||
| log_must rm -rf $BACKDIR/* | |||
|
|
|||
| if is_global_zone ; then | |||
| log_must df >/dev/null | |||
There was a problem hiding this comment.
It's terrible, terrible black magic. I'm not sure why it avoids the issue, but it does.
There was a problem hiding this comment.
See #12898 for the reason this works. I've used that PR to add a comment here which references the root issue, at least until it's properly addressed.
tonynguien
approved these changes
Dec 21, 2021
behlendorf
added
Status: Accepted
behlendorf
added a commit
that referenced
this pull request
Dec 21, 2021
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 10, 2022
The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#5665 Closes openzfs#6086 Closes openzfs#6087 Closes openzfs#6446 Closes openzfs#12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 10, 2022
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 14, 2022
The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#5665 Closes openzfs#6086 Closes openzfs#6087 Closes openzfs#6446 Closes openzfs#12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 14, 2022
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 16, 2022
The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#5665 Closes openzfs#6086 Closes openzfs#6087 Closes openzfs#6446 Closes openzfs#12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 16, 2022
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 17, 2022
The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#5665 Closes openzfs#6086 Closes openzfs#6087 Closes openzfs#6446 Closes openzfs#12876
tonyhutter
pushed a commit
to tonyhutter/zfs
that referenced
this pull request
Feb 17, 2022
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#12876
nicman23
pushed a commit
to nicman23/zfs
that referenced
this pull request
Aug 22, 2022
The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#5665 Closes openzfs#6086 Closes openzfs#6087 Closes openzfs#6446 Closes openzfs#12876
nicman23
pushed a commit
to nicman23/zfs
that referenced
this pull request
Aug 22, 2022
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#12876
nicman23
pushed a commit
to nicman23/zfs
that referenced
this pull request
Aug 22, 2022
The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#5665 Closes openzfs#6086 Closes openzfs#6087 Closes openzfs#6446 Closes openzfs#12876
nicman23
pushed a commit
to nicman23/zfs
that referenced
this pull request
Aug 22, 2022
With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#12876
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Assess the long standing
rsendtest failures which have been addedto the exceptions file to determine if they're still applicable to the latest
code. Additionally, do a little tweaking while here to speed up the test
group.
Description
See commits for additional details.
64f1cb2 ZTS: speed up rsend tests
ef6dda8 ZTS: rsend_007_pos failures
Note that for the
rsend_007_poschange I haven't carefully investigatedwhy this avoids the issue on Linux (it is not needed for FreeBSD). However, I
wanted to get this open for CI to have a crack at it and thought others might
have some ideas.
[Update] This minor addition does seem to reliably avoid the issue.
How Has This Been Tested?
Locally running the
rsendtest group on Ubuntu 20.04. The testsreliably pass in my testing, this PR has been opened to verify that
is the case on all other test platforms.
Types of changes
Checklist:
Signed-off-by.