forked from openzfs/zfs
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix race in parallel mount's thread dispatching algorithm
Strategy of parallel mount is as follows. 1) Initial thread dispatching selects sets of mount points that don't have dependencies on other sets, hence threads can/should go lock-less and shouldn't race with other threads for other sets. Each thread dispatched corresponds to top level directory which may or may not have datasets mounted on sub directories. 2) Subsequent recursive thread dispatching for each thread from 1) is to mount datasets for each set of mount points. The mount points within each set have dependencies (i.e. child directories), so child directories are processed only after parent directory completes. The problem is that initial thread dispatching can spawn >1 threads for datasets with the same top level directory, and this puts threads under race condition. This appeared as mount issue on ZoL for ZoL having different timing regarding mount(2) execution due to fork(2)/exec(2) of mount(8) on mount. Fix it by libzfs_path_contains() returning true for same paths. Closes openzfs#8450 Closes openzfs#8833 Signed-off-by: Tomohiro Kusumi <[email protected]>
- Loading branch information
Showing
4 changed files
with
120 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
114 changes: 114 additions & 0 deletions
114
tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_test_race.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
#!/bin/ksh | ||
|
||
# | ||
# This file and its contents are supplied under the terms of the | ||
# Common Development and Distribution License ("CDDL"), version 1.0. | ||
# You may only use this file in accordance with the terms of version | ||
# 1.0 of the CDDL. | ||
# | ||
# A full copy of the text of the CDDL should have accompanied this | ||
# source. A copy of the CDDL is also available via the Internet at | ||
# http://www.illumos.org/license/CDDL. | ||
# | ||
|
||
# | ||
# Copyright (c) 2019 by Tomohiro Kusumi. All rights reserved. | ||
# | ||
|
||
. $STF_SUITE/include/libtest.shlib | ||
. $STF_SUITE/tests/functional/cli_root/zfs_mount/zfs_mount.cfg | ||
|
||
# | ||
# DESCRIPTION: | ||
# Verify parallel mount ordering is consistent. | ||
# See github.com/zfsonlinux/zfs/issues/8833 for details. | ||
# | ||
# STRATEGY: | ||
# 1. Create pools and filesystems. | ||
# 2. Set same mount point for >1 datasets. | ||
# 3. Unmount all datasets. | ||
# 4. Mount all datasets. | ||
# 5. Unmount all datasets. | ||
# 6. Mount all datasets. | ||
# | ||
|
||
verify_runnable "both" | ||
|
||
TMPDIR=${TMPDIR:-$TEST_BASE_DIR} | ||
MNTPT=$TMPDIR/zfs_mount_test_race_mntpt | ||
|
||
DISK1="$TMPDIR/zfs_mount_test_race_disk1" | ||
TESTPOOL1=zfs_mount_test_race_tp1 | ||
TESTFS1=zfs_mount_test_race_tf1 | ||
|
||
DISK2="$TMPDIR/zfs_mount_test_race_disk2" | ||
TESTPOOL2=zfs_mount_test_race_tp2 | ||
TESTFS2=zfs_mount_test_race_tf2 | ||
|
||
export __ZFS_POOL_RESTRICT="$TESTPOOL1 $TESTPOOL2" | ||
log_must zfs $unmountall | ||
unset __ZFS_POOL_RESTRICT | ||
|
||
function cleanup | ||
{ | ||
zpool destroy $TESTPOOL1 | ||
zpool destroy $TESTPOOL2 | ||
rm -rf $MNTPT | ||
rm -rf /$TESTPOOL1 | ||
rm -rf /$TESTPOOL2 | ||
rm -f $DISK1 | ||
rm -f $DISK2 | ||
export __ZFS_POOL_RESTRICT="$TESTPOOL1 $TESTPOOL2" | ||
log_must zfs $mountall | ||
unset __ZFS_POOL_RESTRICT | ||
} | ||
log_onexit cleanup | ||
|
||
log_note "Verify parallel mount ordering is consistent" | ||
|
||
log_must truncate -s $MINVDEVSIZE $DISK1 | ||
log_must truncate -s $MINVDEVSIZE $DISK2 | ||
|
||
log_must zpool create -f $TESTPOOL1 $DISK1 | ||
log_must zpool create -f $TESTPOOL2 $DISK2 | ||
|
||
log_must zfs create $TESTPOOL1/$TESTFS1 | ||
log_must zfs create $TESTPOOL2/$TESTFS2 | ||
|
||
log_must zfs set mountpoint=none $TESTPOOL1 | ||
log_must zfs set mountpoint=$MNTPT $TESTPOOL1/$TESTFS1 | ||
|
||
# Note that unmount can fail (due to race condition on `zfs mount -a`) with or | ||
# without `canmount=off`. The race has nothing to do with canmount property, | ||
# but turn it off for convenience of mount layout used in this test case. | ||
log_must zfs set canmount=off $TESTPOOL2 | ||
log_must zfs set mountpoint=$MNTPT $TESTPOOL2 | ||
|
||
# At this point, layout of datasets in two pools will look like below. | ||
# Previously, on next `zfs mount -a`, pthreads assigned to TESTFS1 and TESTFS2 | ||
# could race, and TESTFS2 usually (actually always) won in ZoL. Note that the | ||
# problem is not this layout, but rather how two or more threads could initially | ||
# be assigned to the same top level directory. This layout is just an example | ||
# that can reproduce race, and is also the layout reported in #8833. | ||
# | ||
# NAME MOUNTED MOUNTPOINT | ||
# ---------------------------------------------- | ||
# /$TESTPOOL1 no none | ||
# /$TESTPOOL1/$TESTFS1 yes $MNTPT | ||
# /$TESTPOOL2 no $MNTPT | ||
# /$TESTPOOL2/$TESTFS2 yes $MNTPT/$TESTFS2 | ||
|
||
log_must ismounted $TESTPOOL1/$TESTFS1 | ||
log_must ismounted $TESTPOOL2/$TESTFS2 | ||
# This unmount always succeeds, because potential race hasn't happened yet. | ||
log_must zfs unmount -a | ||
# This mount always succeeds, whether threads are under race condition or not. | ||
log_must zfs mount -a | ||
|
||
# Verify datasets are mounted (TESTFS2 fails if the race broke mount order). | ||
log_must ismounted $TESTPOOL1/$TESTFS1 | ||
log_must ismounted $TESTPOOL2/$TESTFS2 | ||
# Verify unmount succeeds (fails if the race broke mount order). | ||
log_must zfs unmount -a | ||
|
||
log_pass "Verify parallel mount ordering is consistent passed" |