Unable to disable MMP at import time

[stuartthebruce]

System information

Type	Version/Name
Distribution Name	Scientific Linux
Distribution Version	7.8
Linux Kernel	3.10.0-1127.18.2.el7
Architecture	x86_64
ZFS Version	0.8.4
SPL Version	0.8.4

Describe the problem you're observing

zpool import blocks on "MMP suspending pool" even with "-o multihost=off".

There should be a way to disable MMP at import time.

Describe how to reproduce the problem

Start 5 concurrent "zfs receive" to a pool with multihost=on as described in #10828 before losing power. Then attempt to get past "MMP suspending pool" by disabling MMP at import time,

zpool import -o multihost=off

Include any warning/errors/backtraces from the system logs

[root@node809 ~]# cat /proc/spl/kstat/zfs/dbgmsg
timestamp    message
1598655869   spa.c:5638:spa_tryimport(): spa_tryimport: importing jbod2-backup
1598655869   spa_misc.c:408:spa_load_note(): spa_load($import, config trusted): LOADING
1598655869   vdev.c:125:vdev_dbgmsg(): disk vdev '/dev/mapper/35000cca2530a7408': best uberblock found for spa $import. txg 136430
1598655869   spa_misc.c:408:spa_load_note(): spa_load($import, config untrusted): using uberblock with txg=136430
1598655879   spa_misc.c:408:spa_load_note(): spa_load($import, config trusted): LOADED
1598655879   spa_misc.c:408:spa_load_note(): spa_load($import, config trusted): UNLOADING
1598655879   spa.c:5638:spa_tryimport(): spa_tryimport: importing jbod2-backup
1598655879   spa_misc.c:408:spa_load_note(): spa_load($import, config trusted): LOADING
1598655880   vdev.c:125:vdev_dbgmsg(): disk vdev '/dev/mapper/35000cca2530a7408': best uberblock found for spa $import. txg 136430
1598655880   spa_misc.c:408:spa_load_note(): spa_load($import, config untrusted): using uberblock with txg=136430
1598655891   spa_misc.c:408:spa_load_note(): spa_load($import, config trusted): LOADED
1598655891   spa_misc.c:408:spa_load_note(): spa_load($import, config trusted): UNLOADING
1598655891   spa.c:5490:spa_import(): spa_import: importing jbod2-backup
1598655891   spa_misc.c:408:spa_load_note(): spa_load(jbod2-backup, config trusted): LOADING
1598655892   vdev.c:125:vdev_dbgmsg(): disk vdev '/dev/mapper/35000cca2530a7408': best uberblock found for spa jbod2-backup. txg 136430
1598655892   spa_misc.c:408:spa_load_note(): spa_load(jbod2-backup, config untrusted): using uberblock with txg=136430
1598655907   mmp.c:249:mmp_thread_start(): MMP thread started pool 'jbod2-backup' gethrtime 257395831045
1598655907   mmp.c:654:mmp_thread(): MMP suspending pool 'jbod2-backup': gethrtime 257395838153 mmp_last_write 242353194262 mmp_interval 100000000 mmp_fail_intervals 10 mmp_fail_ns 1000000000
1598655907   spa.c:7592:spa_async_request(): spa=jbod2-backup async request task=1

Note, the following also fails to prevent "MMP suspending pool",

[root@node809 ~]# echo 0 > /sys/module/zfs/parameters/zfs_multihost_interval
[root@node809 ~]# echo 0 > /sys/module/zfs/parameters/zfs_multihost_import_intervals
[root@node809 ~]# cat /sys/module/zfs/parameters/zfs_multihost_interval
0
[root@node809 ~]# cat /sys/module/zfs/parameters/zfs_multihost_import_intervals
0
[root@node809 ~]# zpool import jbod2-backup

[behlendorf]

cc: @ofaaland

[ofaaland]

Hi @stuartthebruce

Have you tried
echo 0 > /sys/module/zfs/parameters/zfs_multihost_fail_intervals
before importing?

[stuartthebruce]

Not yet, stay tuned...

[stuartthebruce]

That worked, details on zfs-discuss at https://zfsonlinux.topicbox.com/groups/zfs-discuss/Ta6b683d15084807b/zpool-import-spinning-on-txgquiesce-and-txgsync

For this issue, should "zpool import -o multihost=off" do the equivalent of setting "zfs_multihost_fail_intervals=0"?

I think the larger question of why this was necessary can be dealt with in #10828

[ofaaland]

Hi @stuartthebruce,

For this issue, should "zpool import -o multihost=off" do the equivalent of setting "zfs_multihost_fail_intervals=0"?

I don't think there's a need for this. It would be a workaround for a very specific bug that's already been fixed.

[stuartthebruce]

I don't think there's a need for this. It would be a workaround for a very specific bug that's already been fixed.

Fair enough. If this suggestion is not trivial to implement it may not be worth the effort given the hopefully small probability of ever needing to disable multihost before MMP starts for future bugs (or proper MMP behavior). I will close this once I confirm #10873 fixes this particular problem if it is not closed before then.

[ofaaland]

@stuartthebruce you've probably already seen this, but that fix is in release zfs-0.8.5 and in zfs-2.0.0

[stuartthebruce]

@ofaaland I hadn't seen an option to "disable MMP at import time". If that was added I am willing to give that a try. However, I am also happy with this being closed as "won't fix" now that the root cause MMP problem was fixed (and I can confirm that for 0.8.5, but I have not tested that in 2.0.0 yet).

[ofaaland]

@stuartthebruce Sorry, I wasn't clear. I meant that the bug fix for the MMP problem is in 0.8.5/2.0.0. You're correct, no "disable MMP at import time" option has been added.

[stuartthebruce]

I no longer need this feature and no volunteers to implement it.