Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

don't know what to do with this dmesg #973

Closed
mailinglists35 opened this issue Sep 18, 2012 · 17 comments
Closed

don't know what to do with this dmesg #973

mailinglists35 opened this issue Sep 18, 2012 · 17 comments
Milestone

Comments

@mailinglists35
Copy link

at the time of output I was trying to stress snapshots.

i first created about 200 snapshots then while creating new ones I started to randomly delete previous:

"while true; do zfs-auto-snapshot -p $(uuidgen) //; zfs destroy $(let "snap = $RANDOM % $(zfs list -H -o name -t snap|wc -l)"; zfs list -H -o name -t snap|head -n $snap|tail -n1); done"

during this I started a cp from dir1 to dir2 inside a zfs filesystem.

then during this madness I started zpool scrub :)

at some point I noticed these kernel messages.

Sep 18 13:11:01 homerouter kernel: [15107.938823] SPL: Fixing allocation for task txg_sync (11445) which used GFP flags 0xda0f2bc with PF_NOFS set
Sep 18 13:11:01 homerouter kernel: [15107.938833] SPL: Showing stack for process 11445
Sep 18 13:11:01 homerouter kernel: [15107.938837] Pid: 11445, comm: txg_sync Tainted: P O 3.2.0-3-amd64 #1
Sep 18 13:11:01 homerouter kernel: [15107.938842] Call Trace:
Sep 18 13:11:01 homerouter kernel: [15107.938853] [] ? spl_debug_dumpstack+0x24/0x2a [spl]
Sep 18 13:11:01 homerouter kernel: [15107.938860] [] ? sanitize_flags+0x6e/0x7c [spl]
Sep 18 13:11:01 homerouter kernel: [15107.938868] [] ? kmalloc_nofail+0x1f/0x3d [spl]
Sep 18 13:11:01 homerouter kernel: [15107.938875] [] ? kmem_alloc_debug+0x164/0x2d0 [spl]
Sep 18 13:11:01 homerouter kernel: [15107.938882] [] ? timekeeping_get_ns+0xd/0x2a
Sep 18 13:11:01 homerouter kernel: [15107.938887] [] ? ktime_get_ts+0x5d/0x84
[...]

message is long, uploaded to http://dl.transfer.ro/transfer_ro-18sep-f743bdc8823.zip
the hosting service keeps the file for only 10 days. if asked I can reupload.
system is debian wheezy with ubuntu oneiric daily ppa zfs source
root@homerouter:# dpkg -l|egrep 'zfs|^spl|linux-image'
ii libzfs1 0.6.0.78-0ubuntu1
oneiric1 amd64 Native ZFS filesystem library for Linux
ii linux-image-2.6-amd64 3.2+45 amd64 Linux for 64-bit PCs (dummy package)
ii linux-image-3.2.0-3-amd64 3.2.23-1 amd64 Linux 3.2 for 64-bit PCs
ii linux-image-amd64 3.2+45 amd64 Linux for 64-bit PCs (meta-package)
ii ubuntu-zfs 6precise amd64 Native ZFS filesystem metapackage for Ubuntu.
ii zfs-auto-snapshot 1.0.8-0ubuntu1
oneiric1 all ZFS Automatic Snapshot Service
ii zfs-dkms 0.6.0.78-0ubuntu1oneiric1 amd64 Native ZFS filesystem kernel modules for Linux
ii zfsutils 0.6.0.78-0ubuntu1
oneiric1 amd64 Native ZFS management utilities for Linux

@mailinglists35
Copy link
Author

update
there seems to be something related to scrub
I have even stopped the cp and zfs destroy/create loop.

only running "zpool scrub -s test1" stops kernel message errors.

@mailinglists35
Copy link
Author

when the errors begin, zpool status output appears stalled, ie running it again shows no modification of "scanned" or percent. only the speed decreases (because is the average and "zero" decreases de average)

pool: test1
state: ONLINE
scan: scrub in progress since Tue Sep 18 13:50:34 2012
48.8G scanned out of 54.5G at 156M/s, 0h0m to go
0 repaired, 89.57% done
config:

    NAME        STATE     READ WRITE CKSUM
    test1       ONLINE       0     0     0
      sdg       ONLINE       0     0     0

errors: No known data errors

@behlendorf
Copy link
Contributor

These warnings were addressed in the 0.6.0.76 Ubuntu release. I see you have 0.6.0.78 installed but the older modules must have been loaded. I'd suggest you update to the latest daily source, reboot your machine, ensure the 0.6.0.78 (or newer) modules get loaded and try again.

@mailinglists35
Copy link
Author

I am afraid this has happened with .78:

note the logs are timestamped Sep 18 13:11:01 while the module has been loaded earlier:

root@homerouter:~# fgrep ' module v0.6.0.' /var/log/messages
Sep 17 16:15:25 homerouter kernel: [18851.171470] SPL: Loaded module v0.6.0.78-rc10
Sep 17 16:15:25 homerouter kernel: [18851.255659] ZFS: Loaded module v0.6.0.78-rc10, ZFS pool version 28, ZFS filesystem version 5
Sep 18 10:30:52 homerouter kernel: [ 5499.198379] SPL: Loaded module v0.6.0.78-rc10
Sep 18 10:30:52 homerouter kernel: [ 5499.275886] ZFS: Loaded module v0.6.0.78-rc10, ZFS pool version 28, ZFS filesystem version 5
Sep 19 20:12:46 homerouter kernel: [39638.654961] SPL: Loaded module v0.6.0.80-rc11
Sep 19 20:12:46 homerouter kernel: [39638.691246] ZFS: Loaded module v0.6.0.80-rc11, ZFS pool version 28, ZFS filesystem version 5
Sep 19 20:13:40 homerouter kernel: [39692.694289] ZFS: Unloaded module v0.6.0.80-rc11
Sep 19 20:13:40 homerouter kernel: [39692.760030] SPL: Unloaded module v0.6.0.80-rc11
Sep 19 20:13:47 homerouter kernel: [39699.565587] SPL: Loaded module v0.6.0.80-rc11
Sep 19 20:13:47 homerouter kernel: [39699.601457] ZFS: Loaded module v0.6.0.80-rc11, ZFS pool version 28, ZFS filesystem version 5

nonetheless, I'll retest now with this latest verson.

@behlendorf
Copy link
Contributor

Yes, please do. If your able to recreate the issue we'll absolutely reopen the bug and get it fixed.

@mailinglists35
Copy link
Author

stalled at 5 percent

scan: scrub in progress since Wed Sep 19 20:53:34 2012
6.28G scanned out of 109G at 28.4M/s, 1h1m to go
0 repaired, 5.76% done

do you want me to wait for 100% or shall I cancel the scrub and send what's been dumped so far?

@behlendorf
Copy link
Contributor

Can you check the console via dmesg and see if anything has been logged. If so paste in a full copy of one of the stack traces.

@mailinglists35
Copy link
Author

it eventually finished - dump truncated at 10Mb, hope it's long enough: http://dl.transfer.ro/transfer_ro-19sep-26074c8905.zip

0.6.0.80-0ubuntu1~oneiric1

@behlendorf behlendorf reopened this Sep 19, 2012
behlendorf added a commit to behlendorf/zfs that referenced this issue Sep 19, 2012
This warning indicates the incorrect use of KM_SLEEP in a call
path which must use KM_PUSHPAGE to avoid deadlocking in direct
reclaim.  See commit b8d06fc
for additional details.

  SPL: Fixing allocation for task txg_sync (6093) which
  used GFP flags 0x297bda7c with PF_NOFS set

Signed-off-by: Brian Behlendorf <[email protected]>
Issue openzfs#973
@behlendorf
Copy link
Contributor

@mailinglists35 Your right, you found a few new cases of this related to dedup. The above patch should fix it, you can either roll a custom version or wait a few days for this to make it in to the daily PPA.

@mailinglists35
Copy link
Author

thank you, i'll just wait for normal update and report status.

@mailinglists35
Copy link
Author

I waited and waited for this patch to get into a new daily ppa build but eventually I gave up and downloaded zfsonlinux-spl-a6c6839 & ran "autogen.sh; configure; make" for the first time in years.
shortly after unloading spl v0.6.0.80-rc11 and loading the manually compiled spl.ko and splat.ko, i ran zfs scrub then after a few minutes the attached dump.

this was tested on ubuntu precise amd64

http://dl.transfer.ro/transfer_ro-04oct-d0966a010c71a.zip

@pyavdr
Copy link
Contributor

pyavdr commented Oct 5, 2012

This is maybe solved by commit d75d6f2 ( issue #1009) which was commited yesterday. Pls check it.

@behlendorf
Copy link
Contributor

According to the attached log you loaded the Ubuntu v0.6.0.80-rc11 modules not the latest master source with the fix.

@mailinglists35
Copy link
Author

scrub messages appears to have stopped once I updated to the latest ubuntu-zfs package, thanks.

@mailinglists35
Copy link
Author

is this related, or is another bug? maybe related to openzfs/spl#97? using latest ubuntu-zfs package on ubuntu 12.04 lts amd64
triggered during zfs scrub
scrub appears to be dead. "btrace /dev/sdc" shows no disk activity.
root@mailhost:~# zpool status
pool: fulldisk
state: ONLINE
scan: scrub in progress since Thu Oct 11 18:00:49 2012
49.8G scanned out of 147G at 11.3M/s, 2h26m to go
0 repaired, 33.95% done
config:

    NAME        STATE     READ WRITE CKSUM
    fulldisk    ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        sdc     ONLINE       0     0     0
        sdd     ONLINE       0     0     0

errors: No known data errors

kernel:

Oct 11 18:24:27 mailhost kernel: [ 1965.517305] VERIFY(!RW_LOCK_HELD(&l->l_rwlock)) failed
Oct 11 18:24:27 mailhost kernel: [ 1965.517336] SPLError: 337:0:(zap.c:444:zap_leaf_pageout()) SPL PANIC
Oct 11 18:24:27 mailhost kernel: [ 1965.517361] SPL: Showing stack for process 337
Oct 11 18:24:27 mailhost kernel: [ 1965.517364] Pid: 337, comm: arc_adapt Tainted: P IO 3.2.0-31-generic #50-Ubuntu
Oct 11 18:24:27 mailhost kernel: [ 1965.517367] Call Trace:
Oct 11 18:24:27 mailhost kernel: [ 1965.517382] [] spl_debug_dumpstack+0x27/0x40 [spl]
Oct 11 18:24:27 mailhost kernel: [ 1965.517390] [] spl_debug_bug+0x82/0xe0 [spl]
Oct 11 18:24:27 mailhost kernel: [ 1965.517430] [] zap_leaf_pageout+0x73/0xc0 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517449] [] dbuf_evict_user+0x49/0x80 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517468] [] dbuf_clear+0x29/0x180 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517486] [] dbuf_evict+0x16/0x30 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517505] [] dbuf_do_evict+0x43/0x70 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517521] [] arc_do_user_evicts+0x50/0xd0 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517543] [] ? arc_adjust_meta+0x1e0/0x1e0 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517567] [] arc_adapt_thread+0x7e/0xd0 [zfs]
Oct 11 18:24:27 mailhost kernel: [ 1965.517579] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:24:27 mailhost kernel: [ 1965.517590] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:24:27 mailhost kernel: [ 1965.517597] [] kthread+0x8c/0xa0
Oct 11 18:24:27 mailhost kernel: [ 1965.517604] [] kernel_thread_helper+0x4/0x10
Oct 11 18:24:27 mailhost kernel: [ 1965.517610] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:24:27 mailhost kernel: [ 1965.517616] [] ? gs_change+0x13/0x13
Oct 11 18:24:27 mailhost kernel: [ 1965.517661] SPL: Dumping log to /tmp/spl-log.1349969067.337
Oct 11 18:27:41 mailhost kernel: [ 2160.130986] INFO: task arc_adapt:337 blocked for more than 120 seconds.
Oct 11 18:27:41 mailhost kernel: [ 2160.131013] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:27:41 mailhost kernel: [ 2160.131052] arc_adapt D ffffffff81806200 0 337 2 0x00000000
Oct 11 18:27:41 mailhost kernel: [ 2160.131057] ffff880076719d60 0000000000000046 0000000000000000 0000000000000286
Oct 11 18:27:41 mailhost kernel: [ 2160.131063] ffff880076719fd8 ffff880076719fd8 ffff880076719fd8 00000000000137c0
Oct 11 18:27:41 mailhost kernel: [ 2160.131068] ffffffff81c0d020 ffff88007568ae00 ffffffff00000001 0000000000000000
Oct 11 18:27:41 mailhost kernel: [ 2160.131073] Call Trace:
Oct 11 18:27:41 mailhost kernel: [ 2160.131082] [] schedule+0x3f/0x60
Oct 11 18:27:41 mailhost kernel: [ 2160.131094] [] spl_debug_bug+0xbd/0xe0 [spl]
Oct 11 18:27:41 mailhost kernel: [ 2160.131132] [] zap_leaf_pageout+0x73/0xc0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131151] [] dbuf_evict_user+0x49/0x80 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131170] [] dbuf_clear+0x29/0x180 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131189] [] dbuf_evict+0x16/0x30 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131207] [] dbuf_do_evict+0x43/0x70 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131223] [] arc_do_user_evicts+0x50/0xd0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131241] [] ? arc_adjust_meta+0x1e0/0x1e0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131257] [] arc_adapt_thread+0x7e/0xd0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131266] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:27:41 mailhost kernel: [ 2160.131274] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:27:41 mailhost kernel: [ 2160.131279] [] kthread+0x8c/0xa0
Oct 11 18:27:41 mailhost kernel: [ 2160.131284] [] kernel_thread_helper+0x4/0x10
Oct 11 18:27:41 mailhost kernel: [ 2160.131288] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:27:41 mailhost kernel: [ 2160.131292] [] ? gs_change+0x13/0x13
Oct 11 18:27:41 mailhost kernel: [ 2160.131309] INFO: task txg_sync:11372 blocked for more than 120 seconds.
Oct 11 18:27:41 mailhost kernel: [ 2160.131334] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:27:41 mailhost kernel: [ 2160.131371] txg_sync D ffffffff81806200 0 11372 2 0x00000000
Oct 11 18:27:41 mailhost kernel: [ 2160.131376] ffff880039ee3690 0000000000000046 0000000000000282 ffff88003adb70e0
Oct 11 18:27:41 mailhost kernel: [ 2160.131380] ffff880039ee3fd8 ffff880039ee3fd8 ffff880039ee3fd8 00000000000137c0
Oct 11 18:27:41 mailhost kernel: [ 2160.131385] ffff880078c21700 ffff88003a6a1700 0000000000000000 ffff88003b173498
Oct 11 18:27:41 mailhost kernel: [ 2160.131390] Call Trace:
Oct 11 18:27:41 mailhost kernel: [ 2160.131394] [] schedule+0x3f/0x60
Oct 11 18:27:41 mailhost kernel: [ 2160.131399] [] __mutex_lock_slowpath+0xd7/0x150
Oct 11 18:27:41 mailhost kernel: [ 2160.131404] [] mutex_lock+0x2a/0x50
Oct 11 18:27:41 mailhost kernel: [ 2160.131422] [] dbuf_find+0xac/0x100 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131441] [] __dbuf_hold_impl+0xa9/0x490 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131460] [] dbuf_hold_impl+0x86/0xc0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131479] [] dbuf_hold+0x20/0x30 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131499] [] dmu_buf_hold+0x97/0x1d0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131530] [] zap_get_leaf_byblk+0x4f/0x2d0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131560] [] ? zap_match.part.5+0x80/0x80 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131589] [] zap_deref_leaf+0x6d/0x80 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131618] [] fzap_cursor_retrieve+0xdf/0x280 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131646] [] zap_cursor_retrieve+0x6b/0x320 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131665] [] ? dbuf_read+0x5cb/0x840 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131689] [] ? dnode_hold_impl+0x2d9/0x5b0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131718] [] zap_value_search+0x96/0xf0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131742] [] dsl_dataset_get_snapname+0x87/0xa0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131767] [] ? dsl_dir_name+0x6e/0xb0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131790] [] dsl_dataset_name+0x38/0x120 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131816] [] dsl_scan_visitds+0x102/0x4d0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131840] [] ? dsl_dataset_rele+0x43/0x110 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131865] [] dsl_scan_sync+0x227/0x8e0 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131894] [] spa_sync+0x3f1/0xa00 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131924] [] txg_sync_thread+0x286/0x450 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131954] [] ? txg_init+0x250/0x250 [zfs]
Oct 11 18:27:41 mailhost kernel: [ 2160.131962] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:27:41 mailhost kernel: [ 2160.131970] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:27:41 mailhost kernel: [ 2160.131973] [] kthread+0x8c/0xa0
Oct 11 18:27:41 mailhost kernel: [ 2160.131978] [] kernel_thread_helper+0x4/0x10
Oct 11 18:27:41 mailhost kernel: [ 2160.131981] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:27:41 mailhost kernel: [ 2160.131985] [] ? gs_change+0x13/0x13
Oct 11 18:29:41 mailhost kernel: [ 2280.104509] INFO: task arc_adapt:337 blocked for more than 120 seconds.
Oct 11 18:29:41 mailhost kernel: [ 2280.104537] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:29:41 mailhost kernel: [ 2280.104575] arc_adapt D ffffffff81806200 0 337 2 0x00000000
Oct 11 18:29:41 mailhost kernel: [ 2280.104581] ffff880076719d60 0000000000000046 0000000000000000 0000000000000286
Oct 11 18:29:41 mailhost kernel: [ 2280.104586] ffff880076719fd8 ffff880076719fd8 ffff880076719fd8 00000000000137c0
Oct 11 18:29:41 mailhost kernel: [ 2280.104591] ffffffff81c0d020 ffff88007568ae00 ffffffff00000001 0000000000000000
Oct 11 18:29:41 mailhost kernel: [ 2280.104596] Call Trace:
Oct 11 18:29:41 mailhost kernel: [ 2280.104605] [] schedule+0x3f/0x60
Oct 11 18:29:41 mailhost kernel: [ 2280.104617] [] spl_debug_bug+0xbd/0xe0 [spl]
Oct 11 18:29:41 mailhost kernel: [ 2280.104656] [] zap_leaf_pageout+0x73/0xc0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104675] [] dbuf_evict_user+0x49/0x80 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104694] [] dbuf_clear+0x29/0x180 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104713] [] dbuf_evict+0x16/0x30 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104731] [] dbuf_do_evict+0x43/0x70 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104747] [] arc_do_user_evicts+0x50/0xd0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104764] [] ? arc_adjust_meta+0x1e0/0x1e0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104781] [] arc_adapt_thread+0x7e/0xd0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104789] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:29:41 mailhost kernel: [ 2280.104797] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:29:41 mailhost kernel: [ 2280.104802] [] kthread+0x8c/0xa0
Oct 11 18:29:41 mailhost kernel: [ 2280.104807] [] kernel_thread_helper+0x4/0x10
Oct 11 18:29:41 mailhost kernel: [ 2280.104811] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:29:41 mailhost kernel: [ 2280.104815] [] ? gs_change+0x13/0x13
Oct 11 18:29:41 mailhost kernel: [ 2280.104833] INFO: task txg_sync:11372 blocked for more than 120 seconds.
Oct 11 18:29:41 mailhost kernel: [ 2280.104857] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:29:41 mailhost kernel: [ 2280.104895] txg_sync D ffffffff81806200 0 11372 2 0x00000000
Oct 11 18:29:41 mailhost kernel: [ 2280.104899] ffff880039ee3690 0000000000000046 0000000000000282 ffff88003adb70e0
Oct 11 18:29:41 mailhost kernel: [ 2280.104904] ffff880039ee3fd8 ffff880039ee3fd8 ffff880039ee3fd8 00000000000137c0
Oct 11 18:29:41 mailhost kernel: [ 2280.104908] ffff880078c21700 ffff88003a6a1700 0000000000000000 ffff88003b173498
Oct 11 18:29:41 mailhost kernel: [ 2280.104913] Call Trace:
Oct 11 18:29:41 mailhost kernel: [ 2280.104917] [] schedule+0x3f/0x60
Oct 11 18:29:41 mailhost kernel: [ 2280.104922] [] __mutex_lock_slowpath+0xd7/0x150
Oct 11 18:29:41 mailhost kernel: [ 2280.104927] [] mutex_lock+0x2a/0x50
Oct 11 18:29:41 mailhost kernel: [ 2280.104946] [] dbuf_find+0xac/0x100 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104964] [] __dbuf_hold_impl+0xa9/0x490 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.104983] [] dbuf_hold_impl+0x86/0xc0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105002] [] dbuf_hold+0x20/0x30 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105022] [] dmu_buf_hold+0x97/0x1d0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105053] [] zap_get_leaf_byblk+0x4f/0x2d0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105083] [] ? zap_match.part.5+0x80/0x80 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105112] [] zap_deref_leaf+0x6d/0x80 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105141] [] fzap_cursor_retrieve+0xdf/0x280 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105170] [] zap_cursor_retrieve+0x6b/0x320 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105189] [] ? dbuf_read+0x5cb/0x840 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105212] [] ? dnode_hold_impl+0x2d9/0x5b0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105241] [] zap_value_search+0x96/0xf0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105265] [] dsl_dataset_get_snapname+0x87/0xa0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105290] [] ? dsl_dir_name+0x6e/0xb0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105314] [] dsl_dataset_name+0x38/0x120 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105339] [] dsl_scan_visitds+0x102/0x4d0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105363] [] ? dsl_dataset_rele+0x43/0x110 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105389] [] dsl_scan_sync+0x227/0x8e0 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105417] [] spa_sync+0x3f1/0xa00 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105447] [] txg_sync_thread+0x286/0x450 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105476] [] ? txg_init+0x250/0x250 [zfs]
Oct 11 18:29:41 mailhost kernel: [ 2280.105485] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:29:41 mailhost kernel: [ 2280.105492] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:29:41 mailhost kernel: [ 2280.105496] [] kthread+0x8c/0xa0
Oct 11 18:29:41 mailhost kernel: [ 2280.105500] [] kernel_thread_helper+0x4/0x10
Oct 11 18:29:41 mailhost kernel: [ 2280.105504] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:29:41 mailhost kernel: [ 2280.105508] [] ? gs_change+0x13/0x13
Oct 11 18:31:41 mailhost kernel: [ 2400.078035] INFO: task arc_adapt:337 blocked for more than 120 seconds.
Oct 11 18:31:41 mailhost kernel: [ 2400.078064] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:31:41 mailhost kernel: [ 2400.078102] arc_adapt D ffffffff81806200 0 337 2 0x00000000
Oct 11 18:31:41 mailhost kernel: [ 2400.078107] ffff880076719d60 0000000000000046 0000000000000000 0000000000000286
Oct 11 18:31:41 mailhost kernel: [ 2400.078113] ffff880076719fd8 ffff880076719fd8 ffff880076719fd8 00000000000137c0
Oct 11 18:31:41 mailhost kernel: [ 2400.078118] ffffffff81c0d020 ffff88007568ae00 ffffffff00000001 0000000000000000
Oct 11 18:31:41 mailhost kernel: [ 2400.078123] Call Trace:
Oct 11 18:31:41 mailhost kernel: [ 2400.078132] [] schedule+0x3f/0x60
Oct 11 18:31:41 mailhost kernel: [ 2400.078143] [] spl_debug_bug+0xbd/0xe0 [spl]
Oct 11 18:31:41 mailhost kernel: [ 2400.078181] [] zap_leaf_pageout+0x73/0xc0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078201] [] dbuf_evict_user+0x49/0x80 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078220] [] dbuf_clear+0x29/0x180 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078238] [] dbuf_evict+0x16/0x30 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078256] [] dbuf_do_evict+0x43/0x70 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078272] [] arc_do_user_evicts+0x50/0xd0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078289] [] ? arc_adjust_meta+0x1e0/0x1e0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078306] [] arc_adapt_thread+0x7e/0xd0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078314] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:31:41 mailhost kernel: [ 2400.078323] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:31:41 mailhost kernel: [ 2400.078327] [] kthread+0x8c/0xa0
Oct 11 18:31:41 mailhost kernel: [ 2400.078332] [] kernel_thread_helper+0x4/0x10
Oct 11 18:31:41 mailhost kernel: [ 2400.078336] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:31:41 mailhost kernel: [ 2400.078340] [] ? gs_change+0x13/0x13
Oct 11 18:31:41 mailhost kernel: [ 2400.078358] INFO: task txg_sync:11372 blocked for more than 120 seconds.
Oct 11 18:31:41 mailhost kernel: [ 2400.078383] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:31:41 mailhost kernel: [ 2400.078420] txg_sync D ffffffff81806200 0 11372 2 0x00000000
Oct 11 18:31:41 mailhost kernel: [ 2400.078425] ffff880039ee3690 0000000000000046 0000000000000282 ffff88003adb70e0
Oct 11 18:31:41 mailhost kernel: [ 2400.078429] ffff880039ee3fd8 ffff880039ee3fd8 ffff880039ee3fd8 00000000000137c0
Oct 11 18:31:41 mailhost kernel: [ 2400.078434] ffff880078c21700 ffff88003a6a1700 0000000000000000 ffff88003b173498
Oct 11 18:31:41 mailhost kernel: [ 2400.078438] Call Trace:
Oct 11 18:31:41 mailhost kernel: [ 2400.078443] [] schedule+0x3f/0x60
Oct 11 18:31:41 mailhost kernel: [ 2400.078448] [] __mutex_lock_slowpath+0xd7/0x150
Oct 11 18:31:41 mailhost kernel: [ 2400.078452] [] mutex_lock+0x2a/0x50
Oct 11 18:31:41 mailhost kernel: [ 2400.078471] [] dbuf_find+0xac/0x100 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078490] [] __dbuf_hold_impl+0xa9/0x490 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078508] [] dbuf_hold_impl+0x86/0xc0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078527] [] dbuf_hold+0x20/0x30 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078547] [] dmu_buf_hold+0x97/0x1d0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078579] [] zap_get_leaf_byblk+0x4f/0x2d0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078608] [] ? zap_match.part.5+0x80/0x80 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078637] [] zap_deref_leaf+0x6d/0x80 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078666] [] fzap_cursor_retrieve+0xdf/0x280 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078694] [] zap_cursor_retrieve+0x6b/0x320 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078713] [] ? dbuf_read+0x5cb/0x840 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078736] [] ? dnode_hold_impl+0x2d9/0x5b0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078765] [] zap_value_search+0x96/0xf0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078789] [] dsl_dataset_get_snapname+0x87/0xa0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078814] [] ? dsl_dir_name+0x6e/0xb0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078837] [] dsl_dataset_name+0x38/0x120 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078863] [] dsl_scan_visitds+0x102/0x4d0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078887] [] ? dsl_dataset_rele+0x43/0x110 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078913] [] dsl_scan_sync+0x227/0x8e0 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078941] [] spa_sync+0x3f1/0xa00 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.078971] [] txg_sync_thread+0x286/0x450 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.079000] [] ? txg_init+0x250/0x250 [zfs]
Oct 11 18:31:41 mailhost kernel: [ 2400.079009] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:31:41 mailhost kernel: [ 2400.079016] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:31:41 mailhost kernel: [ 2400.079020] [] kthread+0x8c/0xa0
Oct 11 18:31:41 mailhost kernel: [ 2400.079024] [] kernel_thread_helper+0x4/0x10
Oct 11 18:31:41 mailhost kernel: [ 2400.079028] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:31:41 mailhost kernel: [ 2400.079032] [] ? gs_change+0x13/0x13
Oct 11 18:33:41 mailhost kernel: [ 2520.051554] INFO: task arc_adapt:337 blocked for more than 120 seconds.
Oct 11 18:33:41 mailhost kernel: [ 2520.051583] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:33:41 mailhost kernel: [ 2520.051621] arc_adapt D ffffffff81806200 0 337 2 0x00000000
Oct 11 18:33:41 mailhost kernel: [ 2520.051627] ffff880076719d60 0000000000000046 0000000000000000 0000000000000286
Oct 11 18:33:41 mailhost kernel: [ 2520.051632] ffff880076719fd8 ffff880076719fd8 ffff880076719fd8 00000000000137c0
Oct 11 18:33:41 mailhost kernel: [ 2520.051637] ffffffff81c0d020 ffff88007568ae00 ffffffff00000001 0000000000000000
Oct 11 18:33:41 mailhost kernel: [ 2520.051642] Call Trace:
Oct 11 18:33:41 mailhost kernel: [ 2520.051651] [] schedule+0x3f/0x60
Oct 11 18:33:41 mailhost kernel: [ 2520.051663] [] spl_debug_bug+0xbd/0xe0 [spl]
Oct 11 18:33:41 mailhost kernel: [ 2520.051701] [] zap_leaf_pageout+0x73/0xc0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051721] [] dbuf_evict_user+0x49/0x80 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051740] [] dbuf_clear+0x29/0x180 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051758] [] dbuf_evict+0x16/0x30 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051776] [] dbuf_do_evict+0x43/0x70 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051792] [] arc_do_user_evicts+0x50/0xd0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051809] [] ? arc_adjust_meta+0x1e0/0x1e0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051826] [] arc_adapt_thread+0x7e/0xd0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.051835] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:33:41 mailhost kernel: [ 2520.051843] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:33:41 mailhost kernel: [ 2520.051847] [] kthread+0x8c/0xa0
Oct 11 18:33:41 mailhost kernel: [ 2520.051852] [] kernel_thread_helper+0x4/0x10
Oct 11 18:33:41 mailhost kernel: [ 2520.051856] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:33:41 mailhost kernel: [ 2520.051860] [] ? gs_change+0x13/0x13
Oct 11 18:33:41 mailhost kernel: [ 2520.051878] INFO: task txg_sync:11372 blocked for more than 120 seconds.
Oct 11 18:33:41 mailhost kernel: [ 2520.051902] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:33:41 mailhost kernel: [ 2520.051940] txg_sync D ffffffff81806200 0 11372 2 0x00000000
Oct 11 18:33:41 mailhost kernel: [ 2520.051944] ffff880039ee3690 0000000000000046 0000000000000282 ffff88003adb70e0
Oct 11 18:33:41 mailhost kernel: [ 2520.051949] ffff880039ee3fd8 ffff880039ee3fd8 ffff880039ee3fd8 00000000000137c0
Oct 11 18:33:41 mailhost kernel: [ 2520.051954] ffff880078c21700 ffff88003a6a1700 0000000000000000 ffff88003b173498
Oct 11 18:33:41 mailhost kernel: [ 2520.051958] Call Trace:
Oct 11 18:33:41 mailhost kernel: [ 2520.051963] [] schedule+0x3f/0x60
Oct 11 18:33:41 mailhost kernel: [ 2520.051967] [] __mutex_lock_slowpath+0xd7/0x150
Oct 11 18:33:41 mailhost kernel: [ 2520.051972] [] mutex_lock+0x2a/0x50
Oct 11 18:33:41 mailhost kernel: [ 2520.051990] [] dbuf_find+0xac/0x100 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052009] [] __dbuf_hold_impl+0xa9/0x490 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052028] [] dbuf_hold_impl+0x86/0xc0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052047] [] dbuf_hold+0x20/0x30 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052067] [] dmu_buf_hold+0x97/0x1d0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052098] [] zap_get_leaf_byblk+0x4f/0x2d0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052128] [] ? zap_match.part.5+0x80/0x80 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052158] [] zap_deref_leaf+0x6d/0x80 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052186] [] fzap_cursor_retrieve+0xdf/0x280 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052214] [] zap_cursor_retrieve+0x6b/0x320 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052233] [] ? dbuf_read+0x5cb/0x840 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052257] [] ? dnode_hold_impl+0x2d9/0x5b0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052286] [] zap_value_search+0x96/0xf0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052310] [] dsl_dataset_get_snapname+0x87/0xa0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052335] [] ? dsl_dir_name+0x6e/0xb0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052358] [] dsl_dataset_name+0x38/0x120 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052384] [] dsl_scan_visitds+0x102/0x4d0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052408] [] ? dsl_dataset_rele+0x43/0x110 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052433] [] dsl_scan_sync+0x227/0x8e0 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052462] [] spa_sync+0x3f1/0xa00 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052491] [] txg_sync_thread+0x286/0x450 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052521] [] ? txg_init+0x250/0x250 [zfs]
Oct 11 18:33:41 mailhost kernel: [ 2520.052529] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:33:41 mailhost kernel: [ 2520.052537] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:33:41 mailhost kernel: [ 2520.052541] [] kthread+0x8c/0xa0
Oct 11 18:33:41 mailhost kernel: [ 2520.052545] [] kernel_thread_helper+0x4/0x10
Oct 11 18:33:41 mailhost kernel: [ 2520.052549] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:33:41 mailhost kernel: [ 2520.052553] [] ? gs_change+0x13/0x13
Oct 11 18:35:41 mailhost kernel: [ 2640.025070] INFO: task arc_adapt:337 blocked for more than 120 seconds.
Oct 11 18:35:41 mailhost kernel: [ 2640.025098] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:35:41 mailhost kernel: [ 2640.025136] arc_adapt D ffffffff81806200 0 337 2 0x00000000
Oct 11 18:35:41 mailhost kernel: [ 2640.025142] ffff880076719d60 0000000000000046 0000000000000000 0000000000000286
Oct 11 18:35:41 mailhost kernel: [ 2640.025148] ffff880076719fd8 ffff880076719fd8 ffff880076719fd8 00000000000137c0
Oct 11 18:35:41 mailhost kernel: [ 2640.025153] ffffffff81c0d020 ffff88007568ae00 ffffffff00000001 0000000000000000
Oct 11 18:35:41 mailhost kernel: [ 2640.025157] Call Trace:
Oct 11 18:35:41 mailhost kernel: [ 2640.025167] [] schedule+0x3f/0x60
Oct 11 18:35:41 mailhost kernel: [ 2640.025179] [] spl_debug_bug+0xbd/0xe0 [spl]
Oct 11 18:35:41 mailhost kernel: [ 2640.025218] [] zap_leaf_pageout+0x73/0xc0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025237] [] dbuf_evict_user+0x49/0x80 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025256] [] dbuf_clear+0x29/0x180 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025275] [] dbuf_evict+0x16/0x30 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025293] [] dbuf_do_evict+0x43/0x70 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025309] [] arc_do_user_evicts+0x50/0xd0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025326] [] ? arc_adjust_meta+0x1e0/0x1e0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025343] [] arc_adapt_thread+0x7e/0xd0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025351] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:35:41 mailhost kernel: [ 2640.025359] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:35:41 mailhost kernel: [ 2640.025364] [] kthread+0x8c/0xa0
Oct 11 18:35:41 mailhost kernel: [ 2640.025369] [] kernel_thread_helper+0x4/0x10
Oct 11 18:35:41 mailhost kernel: [ 2640.025373] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:35:41 mailhost kernel: [ 2640.025377] [] ? gs_change+0x13/0x13
Oct 11 18:35:41 mailhost kernel: [ 2640.025395] INFO: task txg_sync:11372 blocked for more than 120 seconds.
Oct 11 18:35:41 mailhost kernel: [ 2640.025419] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 18:35:41 mailhost kernel: [ 2640.025457] txg_sync D ffffffff81806200 0 11372 2 0x00000000
Oct 11 18:35:41 mailhost kernel: [ 2640.025461] ffff880039ee3690 0000000000000046 0000000000000282 ffff88003adb70e0
Oct 11 18:35:41 mailhost kernel: [ 2640.025466] ffff880039ee3fd8 ffff880039ee3fd8 ffff880039ee3fd8 00000000000137c0
Oct 11 18:35:41 mailhost kernel: [ 2640.025471] ffff880078c21700 ffff88003a6a1700 0000000000000000 ffff88003b173498
Oct 11 18:35:41 mailhost kernel: [ 2640.025475] Call Trace:
Oct 11 18:35:41 mailhost kernel: [ 2640.025480] [] schedule+0x3f/0x60
Oct 11 18:35:41 mailhost kernel: [ 2640.025485] [] __mutex_lock_slowpath+0xd7/0x150
Oct 11 18:35:41 mailhost kernel: [ 2640.025489] [] mutex_lock+0x2a/0x50
Oct 11 18:35:41 mailhost kernel: [ 2640.025508] [] dbuf_find+0xac/0x100 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025527] [] __dbuf_hold_impl+0xa9/0x490 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025545] [] dbuf_hold_impl+0x86/0xc0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025564] [] dbuf_hold+0x20/0x30 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025584] [] dmu_buf_hold+0x97/0x1d0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025615] [] zap_get_leaf_byblk+0x4f/0x2d0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025645] [] ? zap_match.part.5+0x80/0x80 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025674] [] zap_deref_leaf+0x6d/0x80 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025702] [] fzap_cursor_retrieve+0xdf/0x280 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025731] [] zap_cursor_retrieve+0x6b/0x320 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025750] [] ? dbuf_read+0x5cb/0x840 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025773] [] ? dnode_hold_impl+0x2d9/0x5b0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025802] [] zap_value_search+0x96/0xf0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025826] [] dsl_dataset_get_snapname+0x87/0xa0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025851] [] ? dsl_dir_name+0x6e/0xb0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025874] [] dsl_dataset_name+0x38/0x120 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025900] [] dsl_scan_visitds+0x102/0x4d0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025924] [] ? dsl_dataset_rele+0x43/0x110 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025949] [] dsl_scan_sync+0x227/0x8e0 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.025978] [] spa_sync+0x3f1/0xa00 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.026007] [] txg_sync_thread+0x286/0x450 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.026037] [] ? txg_init+0x250/0x250 [zfs]
Oct 11 18:35:41 mailhost kernel: [ 2640.026045] [] thread_generic_wrapper+0x78/0x90 [spl]
Oct 11 18:35:41 mailhost kernel: [ 2640.026053] [] ? __thread_create+0x310/0x310 [spl]
Oct 11 18:35:41 mailhost kernel: [ 2640.026057] [] kthread+0x8c/0xa0
Oct 11 18:35:41 mailhost kernel: [ 2640.026061] [] kernel_thread_helper+0x4/0x10
Oct 11 18:35:41 mailhost kernel: [ 2640.026065] [] ? flush_kthread_worker+0xa0/0xa0
Oct 11 18:35:41 mailhost kernel: [ 2640.026068] [] ? gs_change+0x13/0x13

@mailinglists35
Copy link
Author

referenced log file /tmp/spl-log.1349969067.337
http://dl.transfer.ro/transfer_ro-11oct-1205aa2b23387ba.zip

@behlendorf
Copy link
Contributor

Yes, your second issues certainly looks like openzfs/spl#97 . So it's a different bug.

pcd1193182 pushed a commit to pcd1193182/zfs that referenced this issue Sep 26, 2023
…#973)

Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](seanmonstar/num_cpus@v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants