Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zpool status hangs on SSD pool #5938

Closed
fcrg opened this issue Mar 28, 2017 · 5 comments
Closed

zpool status hangs on SSD pool #5938

fcrg opened this issue Mar 28, 2017 · 5 comments

Comments

@fcrg
Copy link

fcrg commented Mar 28, 2017

System information

Type Version/Name
Distribution Name gentoo
Distribution Version 4.7.10
Linux Kernel
Architecture x86_64
ZFS Version PR 5925 ; dweeezil:ntrim2
SPL Version spl-0.7.0-rc3

Describe the problem you're observing

I am using a pool of 8 SSD disks that i have created with zfs-0.7.0-rc2.

4x SanDisk SD8SBAT2 3000 + 4x Samsung SSD 750

The console hangs with the following sequence:

  1. start trim
  2. show status
  3. show trimstats
  4. show status

Describe how to reproduce the problem

zpool trim zfs-d3b2ea14-cb89-4d41-9fd5-5da8e1a6cb15

zpool status zfs-d3b2ea14-cb89-4d41-9fd5-5da8e1a6cb15
pool: zfs-d3b2ea14-cb89-4d41-9fd5-5da8e1a6cb15
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: scrub repaired 0 in 0h0m with 0 errors on Tue Mar 28 14:41:13 2017
trim: 4.48% started: Tue Mar 28 15:34:06 2017 (rate: max)
config:

NAME                        STATE     READ WRITE CKSUM
zfs-d3b2ea14-cb89-4d41-9fd5-5da8e1a6cb15  ONLINE       0     0     0
  raidz2-0                  ONLINE       0     0     0
    zfs-0x5001b444a4534e08  ONLINE       0     0     0
    zfs-0x5001b444a4534963  ONLINE       0     0     0
    zfs-0x5001b444a4534eca  ONLINE       0     0     0
    zfs-0x5001b444a4534edf  ONLINE       0     0     0
    zfs-0x5002538d701c9bc3  ONLINE       0     0     0
    zfs-0x5002538d701ca675  ONLINE       0     0     0
    zfs-0x5002538d701c5128  ONLINE       0     0     0
    zfs-0x5002538d701ca602  ONLINE       0     0     0

errors: No known data errors

cat /proc/spl/kstat/zfs/zfs-d3b2ea14-cb89-4d41-9fd/trimstats
37 1 0x01 5 240 139158676930 404906656552
name type data
extents 4 3390
bytes 4 439688423936
extents_skipped 4 112
bytes_skipped 4 2281984
auto_slow 4 0

zpool status zfs-d3b2ea14-cb89-4d41-9fd5-5da8e1a6cb15

Include any warning/errors/backtraces from the system logs

 kernel: [  379.658533] BUG: unable to handle kernel NULL pointer dereference at 0000000000000388
 kernel: [  379.658538] IP: [<ffffffff81271389>] zio_wait+0x19/0x119
 kernel: [  379.658542] PGD 469aed067 PUD 45fa62067 PMD 0 
 kernel: [  379.658545] Oops: 0002 [#1] SMP
 kernel: [  379.658547] Modules linked in: nfsd nvidia_uvm(PO) mhvtl(O) scst_local(O) scst_user(O) qla2x00tgt(O) qla2xxx_scst(O) iscsi_scst(O) scsi_transport_iscsi scst_changer(O) scst_tape(O) scst(O) sg bonding nvidia_drm(PO) nvidia_modeset(PO) nvidia(PO) iTCO_wdt iTCO_vendor_support x86_pkg_temp_thermal coretemp i2c_i801 asix ixgbe lpc_ich mpt3sas mfd_core mdio
 kernel: [  379.658560] CPU: 7 PID: 25587 Comm: spa_async Tainted: P           O    4.7.10-gentoo-FAST #3
 kernel: [  379.658561] Hardware name: Supermicro X9SRG-F/X9SRG-F, BIOS 3.00 08/01/2013
 kernel: [  379.658563] task: ffff880078cb6800 ti: ffff880386904000 task.ti: ffff880386904000
 kernel: [  379.658563] RIP: 0010:[<ffffffff81271389>]  [<ffffffff81271389>] zio_wait+0x19/0x119
 kernel: [  379.658565] RSP: 0018:ffff880386907d48  EFLAGS: 00010286
 kernel: [  379.658566] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff8803e95d4000
 kernel: [  379.658567] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000
 kernel: [  379.658568] RBP: ffff880386907d60 R08: 0000000000000001 R09: ffffffff8115a86e
 kernel: [  379.658569] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880078cb6800
 kernel: [  379.658569] R13: 00000000ffffffff R14: 00000037a3fc0000 R15: ffff880078d9ea08
 kernel: [  379.658571] FS:  0000000000000000(0000) GS:ffff88047fdc0000(0000) knlGS:0000000000000000
 kernel: [  379.658572] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 kernel: [  379.658573] CR2: 0000000000000388 CR3: 000000045fb00000 CR4: 00000000001406e0
 kernel: [  379.658573] Stack:
 kernel: [  379.658574]  ffff8803e9070000 00000037a3fc0000 ffff880078d9e000 ffff880386907db0
 kernel: [  379.658575]  ffffffff81232307 ffff880386907d98 00000037a4440000 00000037a4440000
 kernel: [  379.658577]  0000000000000009 ffff8803e9070000 ffff880078d9e000 ffff880078d9e000
 kernel: [  379.658578] Call Trace:
 kernel: [  379.658582]  [<ffffffff81232307>] vdev_open+0x3a0/0x456
 kernel: [  379.658584]  [<ffffffff81232b7d>] vdev_reopen+0x2e/0x9c
 kernel: [  379.658587]  [<ffffffff81216283>] spa_async_probe+0x2a/0x58
 kernel: [  379.658588]  [<ffffffff812162a8>] spa_async_probe+0x4f/0x58
 kernel: [  379.658589]  [<ffffffff812162a8>] spa_async_probe+0x4f/0x58
 kernel: [  379.658591]  [<ffffffff8121c0c6>] spa_async_thread+0x1dc/0x28c
 kernel: [  379.658593]  [<ffffffff8121beea>] ? spa_vdev_resilver_done+0xf4/0xf4
 kernel: [  379.658594]  [<ffffffff8121beea>] ? spa_vdev_resilver_done+0xf4/0xf4
 kernel: [  379.658597]  [<ffffffff8115c32e>] thread_generic_wrapper+0x69/0x73
 kernel: [  379.658598]  [<ffffffff8115c2c5>] ? __thread_exit+0x12/0x12
 kernel: [  379.658601]  [<ffffffff810a0ce7>] kthread+0xa0/0xa8
 kernel: [  379.658603]  [<ffffffff818cf9cf>] ret_from_fork+0x1f/0x40
 kernel: [  379.658605]  [<ffffffff810a0c47>] ? init_completion+0x24/0x24
 kernel: [  379.658606] Code: 41 5c 41 5d 41 5e 41 5f 5d c3 55 31 c0 48 89 e5 5d c3 55 48 89 e5 41 55 41 83 cd ff 41 54 65 4c 8b 24 25 40 c3 00 00 53 48 89 fb <4c> 89 a7 88 03 00 00 e8 b6 ce ff ff 4c 89 a3 80 03 00 00 48 89 
 kernel: [  379.658622] RIP  [<ffffffff81271389>] zio_wait+0x19/0x119
 kernel: [  379.658623]  RSP <ffff880386907d48>
 kernel: [  379.658624] CR2: 0000000000000388
 kernel: [  379.658626] ---[ end trace 7514d80dc343f34b ]---
@dweeezil
Copy link
Contributor

@fcrg If you can repeat this, could you please post the output of zpool events -v.

@fcrg
Copy link
Author

fcrg commented Mar 29, 2017

The output from todays test:

controller-7a76c143 ~ # zpool events -v
TIME                           CLASS
Mar 29 2017 08:43:06.471297036 sysevent.fs.zfs.config_sync
        version = 0x0
        class = "sysevent.fs.zfs.config_sync"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58db577a 0x1c176c0c 
        eid = 0x1

Mar 29 2017 08:43:06.471297036 sysevent.fs.zfs.pool_import
        version = 0x0
        class = "sysevent.fs.zfs.pool_import"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58db577a 0x1c176c0c 
        eid = 0x2

Mar 29 2017 08:43:06.522297037 sysevent.fs.zfs.config_sync
        version = 0x0
        class = "sysevent.fs.zfs.config_sync"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58db577a 0x1f219ecd 
        eid = 0x3

Mar 29 2017 09:01:43.741911934 sysevent.fs.zfs.scrub_start
        version = 0x0
        class = "sysevent.fs.zfs.scrub_start"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58db5bd7 0x2c38ad7e 
        eid = 0x4

Mar 29 2017 09:01:44.186907314 sysevent.fs.zfs.scrub_finish
        version = 0x0
        class = "sysevent.fs.zfs.scrub_finish"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58db5bd8 0xb23fab2 
        eid = 0x5

Mar 29 2017 09:02:11.425624516 sysevent.fs.zfs.trim_start
        version = 0x0
        class = "sysevent.fs.zfs.trim_start"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58db5bf3 0x195e83c4 
        eid = 0x6
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781518] BUG: unable to handle kernel NULL pointer dereference at 0000000000000388
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781521] IP: [<ffffffff81271389>] zio_wait+0x19/0x119
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781526] PGD 469b56067 PUD 469901067 PMD 0 
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781528] Oops: 0002 [#1] SMP
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781530] Modules linked in: nfsd nvidia_uvm(PO) mhvtl(O) scst_local(O) scst_user(O) qla2x00tgt(O) qla2xxx_scst(O) iscsi_scst(O) scsi_transport_iscsi scst_changer(O) scst_tape(O) scst(O) sg bonding nvidia_drm(PO) nvidia_modeset(PO) asix nvidia(PO) iTCO_wdt iTCO_vendor_support mpt3sas i2c_i801 ixgbe x86_pkg_temp_thermal coretemp lpc_ich mdio mfd_core
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781541] CPU: 4 PID: 31263 Comm: spa_async Tainted: P           O    4.7.10-gentoo-FAST #3
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781542] Hardware name: Supermicro X9SRG-F/X9SRG-F, BIOS 3.00 08/01/2013
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781543] task: ffff8803d21d0d00 ti: ffff8803b8090000 task.ti: ffff8803b8090000
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781544] RIP: 0010:[<ffffffff81271389>]  [<ffffffff81271389>] zio_wait+0x19/0x119
2017-03-29T09:02:29.198987+02:00 controller-7a76c143 kernel: [ 1301.781546] RSP: 0018:ffff8803b8093d48  EFLAGS: 00010286
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781547] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff8804349f0000
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781547] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781548] RBP: ffff8803b8093d60 R08: 0000000000000001 R09: ffffffff8115a86e
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781549] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8803d21d0d00
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781550] R13: 00000000ffffffff R14: 00000037a3fc0000 R15: ffff88045d8e4a08
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781551] FS:  0000000000000000(0000) GS:ffff88047fd00000(0000) knlGS:0000000000000000
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781552] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781553] CR2: 0000000000000388 CR3: 0000000465a33000 CR4: 00000000001406e0
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781554] Stack:
2017-03-29T09:02:29.199182+02:00 controller-7a76c143 kernel: [ 1301.781554]  ffff8803ece64000 00000037a3fc0000 ffff88045d8e4000 ffff8803b8093db0
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781556]  ffffffff81232307 ffff8803b8093d98 00000037a4440000 00000037a4440000
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781557]  0000000000000009 ffff8803ece64000 ffff88045d8e4000 ffff88045d8e4000
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781559] Call Trace:
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781562]  [<ffffffff81232307>] vdev_open+0x3a0/0x456
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781563]  [<ffffffff81232b7d>] vdev_reopen+0x2e/0x9c
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781565]  [<ffffffff81216283>] spa_async_probe+0x2a/0x58
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781567]  [<ffffffff812162a8>] spa_async_probe+0x4f/0x58
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781568]  [<ffffffff812162a8>] spa_async_probe+0x4f/0x58
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781570]  [<ffffffff8121c0c6>] spa_async_thread+0x1dc/0x28c
2017-03-29T09:02:29.199285+02:00 controller-7a76c143 kernel: [ 1301.781572]  [<ffffffff8121beea>] ? spa_vdev_resilver_done+0xf4/0xf4
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781573]  [<ffffffff8121beea>] ? spa_vdev_resilver_done+0xf4/0xf4
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781576]  [<ffffffff8115c32e>] thread_generic_wrapper+0x69/0x73
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781577]  [<ffffffff8115c2c5>] ? __thread_exit+0x12/0x12
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781580]  [<ffffffff810a0ce7>] kthread+0xa0/0xa8
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781582]  [<ffffffff818cf9cf>] ret_from_fork+0x1f/0x40
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781583]  [<ffffffff810a0c47>] ? init_completion+0x24/0x24
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781584] Code: 41 5c 41 5d 41 5e 41 5f 5d c3 55 31 c0 48 89 e5 5d c3 55 48 89 e5 41 55 41 83 cd ff 41 54 65 4c 8b 24 25 40 c3 00 00 53 48 89 fb <4c> 89 a7 88 03 00 00 e8 b6 ce ff ff 4c 89 a3 80 03 00 00 48 89 
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781598] RIP  [<ffffffff81271389>] zio_wait+0x19/0x119
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781599]  RSP <ffff8803b8093d48>
2017-03-29T09:02:29.199371+02:00 controller-7a76c143 kernel: [ 1301.781600] CR2: 0000000000000388
2017-03-29T09:02:29.199425+02:00 controller-7a76c143 kernel: [ 1301.781938] ---[ end trace 5e9ebae1f59afb8b ]---

@dweeezil
Copy link
Contributor

dweeezil commented Apr 2, 2017

@fcrg Could you please try this with dweeezil:ntrim2-next to see whether it still happens. I can't reproduce the problem, however, I've only tried it with file vdevs so far. The ntrim2-next branch incorporates a few more upstream fixes, one if which does involve raidz. I've also updated the tracepoint support which I believe you discovered wasn't working.

@fcrg
Copy link
Author

fcrg commented Apr 3, 2017

@dweeezil I have updated the sources to dweeezil:ntrim2-next .
The compile issue (#5937) is resolved.

The issue i have seen today with the same workflow (call zpool status during trim)

Apr  3 2017 14:27:27.909392306 sysevent.fs.zfs.trim_start
        version = 0x0
        class = "sysevent.fs.zfs.trim_start"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58e23faf 0x363439b2 
        eid = 0xe051

Apr  3 2017 14:28:36.007721520 sysevent.fs.zfs.trim_finish
        version = 0x0
        class = "sysevent.fs.zfs.trim_finish"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58e23ff4 0x75d230 
        eid = 0xe052

Apr  3 2017 14:29:57.674912650 sysevent.fs.zfs.trim_start
        version = 0x0
        class = "sysevent.fs.zfs.trim_start"
        pool_guid = 0x6e22b247b4324b63
        pool_context = 0x0
        time = 0x58e24045 0x283a598a 
        eid = 0xe053


2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062721] BUG: unable to handle kernel NULL pointer dereference at           (null)
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062724] IP: [<ffffffff810b56a5>] prepare_to_wait_exclusive+0x3f/0x71
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062730] PGD 68167067 PUD 5edf9067 PMD 0 
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062732] Oops: 0002 [#1] SMP
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062734] Modules linked in: nfsd nvidia_uvm(PO) mhvtl(O) scst_local(O) scst_user(O) qla2x00tgt(O) qla2xxx_scst(O) iscsi_scst(O) scsi_transport_iscsi scst_changer(O) scst_tape(O) scst(O) sg bonding nvidia_drm(PO) nvidia_modeset(PO) nvidia(PO) iTCO_wdt iTCO_vendor_support x86_pkg_temp_thermal coretemp i2c_i801 ixgbe mpt3sas asix lpc_ich mfd_core mdio
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062745] CPU: 3 PID: 10661 Comm: zfs-d3b2ea14-cb Tainted: P           O    4.7.10-gentoo-FAST #8
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062746] Hardware name: Supermicro X9SRG-F/X9SRG-F, BIOS 3.00 08/01/2013
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062748] task: ffff8803cdd70d00 ti: ffff88026de0c000 task.ti: ffff88026de0c000
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062749] RIP: 0010:[<ffffffff810b56a5>]  [<ffffffff810b56a5>] prepare_to_wait_exclusive+0x3f/0x71
2017-04-03T14:31:37.295141+02:00 controller-3f927ced kernel: [ 4623.062751] RSP: 0018:ffff88026de0fc88  EFLAGS: 00010046
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062752] RAX: 0000000000000202 RBX: ffff88026de0fcb8 RCX: ffff88026de0fcd0
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062753] RDX: ffff880433736160 RSI: ffff88026de0fcb8 RDI: 0000000000000000
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062753] RBP: ffff88026de0fca8 R08: 0000000000000280 R09: 0000000000000000
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062754] R10: ffffea0009a50200 R11: 0000000000000000 R12: ffff880433736158
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062755] R13: 0000000000000002 R14: 0000000000000000 R15: ffff88026de0fddc
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062756] FS:  0000000000000000(0000) GS:ffff88047fcc0000(0000) knlGS:0000000000000000
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062757] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062758] CR2: 0000000000000000 CR3: 000000005438f000 CR4: 00000000001406e0
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062759] Stack:
2017-04-03T14:31:37.295358+02:00 controller-3f927ced kernel: [ 4623.062760]  0000000000000000 ffff880433736150 ffff880433736000 ffff880433736158
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062761]  ffff88026de0fd00 ffffffff8116305b 0000000000000001 ffff8803cdd70d00
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062763]  ffffffff810b59c5 ffff880433736160 0000000000000000 ffff880433736000
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062764] Call Trace:
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062767]  [<ffffffff8116305b>] cv_wait_common+0x6e/0xef
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062769]  [<ffffffff810b59c5>] ? finish_wait+0x60/0x60
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062770]  [<ffffffff811630ec>] __cv_wait+0x10/0x12
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062773]  [<ffffffff81237099>] metaslab_trim_all+0x131/0x369
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062775]  [<ffffffff81914e29>] ? mutex_unlock+0x11/0x13
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062778]  [<ffffffff81250c8c>] ? spa_config_enter+0x145/0x19e
2017-04-03T14:31:37.295515+02:00 controller-3f927ced kernel: [ 4623.062780]  [<ffffffff812642fd>] vdev_man_trim+0xfc/0x397
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062783]  [<ffffffff81160450>] taskq_thread+0x281/0x395
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062785]  [<ffffffff810a862d>] ? wake_up_q+0x42/0x42
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062787]  [<ffffffff811601cf>] ? taskq_thread_should_stop+0x60/0x60
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062789]  [<ffffffff810a18a7>] kthread+0xa0/0xa8
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062791]  [<ffffffff8191688f>] ret_from_fork+0x1f/0x40
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062792]  [<ffffffff810a1807>] ? init_completion+0x24/0x24
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062793] Code: 0e 01 e8 0e 0f 86 00 48 8b 4b 18 48 8d 53 18 48 39 d1 75 1a 49 8b 7c 24 10 49 8d 54 24 08 49 89 4c 24 10 48 89 53 18 48 89 7b 20 <48> 89 0f 4c 89 6d e0 48 8b 55 e0 65 48 8b 0c 25 40 c3 00 00 48 
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062807] RIP  [<ffffffff810b56a5>] prepare_to_wait_exclusive+0x3f/0x71
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062809]  RSP <ffff88026de0fc88>
2017-04-03T14:31:37.295578+02:00 controller-3f927ced kernel: [ 4623.062809] CR2: 0000000000000000
2017-04-03T14:31:37.295692+02:00 controller-3f927ced kernel: [ 4623.063149] ---[ end trace 675707b65af2526d ]---

Reading symbols from vmlinux...done.
(gdb) disassemble prepare_to_wait_exclusive
Dump of assembler code for function prepare_to_wait_exclusive:
   0xffffffff810b5666 <+0>:	push   %rbp
   0xffffffff810b5667 <+1>:	mov    %rsp,%rbp
   0xffffffff810b566a <+4>:	push   %r13
   0xffffffff810b566c <+6>:	movslq %edx,%r13
   0xffffffff810b566f <+9>:	push   %r12
   0xffffffff810b5671 <+11>:	mov    %rdi,%r12
   0xffffffff810b5674 <+14>:	push   %rbx
   0xffffffff810b5675 <+15>:	mov    %rsi,%rbx
   0xffffffff810b5678 <+18>:	push   %rcx
   0xffffffff810b5679 <+19>:	orl    $0x1,(%rsi)
   0xffffffff810b567c <+22>:	callq  0xffffffff8191658f <_raw_spin_lock_irqsave>
   0xffffffff810b5681 <+27>:	mov    0x18(%rbx),%rcx
   0xffffffff810b5685 <+31>:	lea    0x18(%rbx),%rdx
   0xffffffff810b5689 <+35>:	cmp    %rdx,%rcx
   0xffffffff810b568c <+38>:	jne    0xffffffff810b56a8 <prepare_to_wait_exclusive+66>
   0xffffffff810b568e <+40>:	mov    0x10(%r12),%rdi
   0xffffffff810b5693 <+45>:	lea    0x8(%r12),%rdx
   0xffffffff810b5698 <+50>:	mov    %rcx,0x10(%r12)
   0xffffffff810b569d <+55>:	mov    %rdx,0x18(%rbx)
   0xffffffff810b56a1 <+59>:	mov    %rdi,0x20(%rbx)
   0xffffffff810b56a5 <+63>:	mov    %rcx,(%rdi)
   0xffffffff810b56a8 <+66>:	mov    %r13,-0x20(%rbp)
   0xffffffff810b56ac <+70>:	mov    -0x20(%rbp),%rdx
   0xffffffff810b56b0 <+74>:	mov    %gs:0xc340,%rcx
   0xffffffff810b56b9 <+83>:	xchg   %rdx,(%rcx)
   0xffffffff810b56bc <+86>:	mov    %rax,%rsi
   0xffffffff810b56bf <+89>:	mov    %r12,%rdi
   0xffffffff810b56c2 <+92>:	mov    %rdx,-0x20(%rbp)
   0xffffffff810b56c6 <+96>:	mov    -0x20(%rbp),%rdx
   0xffffffff810b56ca <+100>:	callq  0xffffffff81916440 <_raw_spin_unlock_irqrestore>
   0xffffffff810b56cf <+105>:	pop    %rax
   0xffffffff810b56d0 <+106>:	pop    %rbx
   0xffffffff810b56d1 <+107>:	pop    %r12
   0xffffffff810b56d3 <+109>:	pop    %r13
   0xffffffff810b56d5 <+111>:	pop    %rbp
   0xffffffff810b56d6 <+112>:	retq   
End of assembler dump.


@fcrg
Copy link
Author

fcrg commented Apr 4, 2017

@dweeezil
The crashes are resolved with dweeezil:ntrim2-next.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants