Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kernel panic in ZFS when running qemu-img info on a disk image in a ZFS snapshot #6478

Closed
flynnjFIU opened this issue Aug 8, 2017 · 1 comment
Closed

Kernel panic in ZFS when running qemu-img info on a disk image in a ZFS snapshot #6478

flynnjFIU opened this issue Aug 8, 2017 · 1 comment

Comments

@flynnjFIU
Copy link

System information

Type Version/Name
Distribution Name CentOS
Distribution Version 6
Linux Kernel 2.6.32-696.1.1.el6.x86_64
Architecture x86_64
ZFS Version 0.7.0-1
SPL Version 0.7.0-1

Describe the problem you're observing

Instant kernel panic when you run qemu-img info on a qcow2 disk image contained in a ZFS snapshot.

Describe how to reproduce the problem

Create a qcow2 disk image, then snapshot the filesystem. Navigate to the snapshot and run "qemu-img info" on the snapshot. Kernel panics.

Include any warning/errors/backtraces from the system logs

Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: BUG: unable to handle kernel NULL pointer dereference at 000000000000013e
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: IP: [<ffffffffa03d96f6>] zil_commit+0x16/0x890 [zfs]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: PGD 0 
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Oops: 0000 [#1] SMP 
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: last sysfs file: /sys/devices/system/cpu/online
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: CPU 14 
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Modules linked in: ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle ipt_REJECT xt_multiport nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables nfs fscache nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs bridge stp llc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 vhost_net macvtap macvlan tun kvm_intel kvm uinput ipmi_devintf zfs(P)(U) zcommon(P)(U) znvpair(P)(U) icp(P)(U) spl(U) zlib_deflate zavl(P)(U) zunicode(P)(U) power_meter acpi_ipmi ipmi_si ipmi_msghandler microcode iTCO_wdt iTCO_vendor_support dcdbas sg joydev sb_edac edac_core lpc_ich mfd_core shpchp tg3 ptp pps_core ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif ahci megaraid_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: 
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Pid: 25404, comm: qemu-img Tainted: P           -- ------------    2.6.32-696.6.3.el6.x86_64 #1 Dell Inc. PowerEdge R420/0JD6X3
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: RIP: 0010:[<ffffffffa03d96f6>]  [<ffffffffa03d96f6>] zil_commit+0x16/0x890 [zfs]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: RSP: 0018:ffff8810222c3b98  EFLAGS: 00010292
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: RAX: ffff881014271a08 RBX: 0000000000089000 RCX: 000000000000000f
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: RDX: 0000000000000800 RSI: 0000000000000030 RDI: 0000000000000000
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: RBP: ffff8810222c3c28 R08: ffff8810222c3e48 R09: 0000000000000000
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: R10: ffff8810222c3fd8 R11: 0000000000000000 R12: ffff881014271000
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: R13: ffff8810113172a0 R14: ffff8810113172a0 R15: ffff8810222c3cd8
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: FS:  00007f7e9325b700(0000) GS:ffff8800462e0000(0000) knlGS:0000000000000000
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: CR2: 000000000000013e CR3: 00000010202a1000 CR4: 00000000000407e0
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Process qemu-img (pid: 25404, threadinfo ffff8810222c0000, task ffff88101f321520)
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Stack:
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: ffff881020f6ddc0 ffff8810222c3c58 ffff8810222c3c18 ffffffff8112f943
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: <d> ffff8810222c3bd8 ffffffff8154bdde ffff8810142719e8 ffffffffa042b035
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: <d> ffff8810222c3c08 ffffffffa036a8f2 0000000000089000 ffff881014271000
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Call Trace:
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8112f943>] ? filemap_fault+0xd3/0x500
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8154bdde>] ? mutex_lock+0x1e/0x50
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffffa036a8f2>] ? rrw_enter_read_impl+0xd2/0x180 [zfs]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffffa03d1ecc>] zfs_read+0x40c/0x440 [zfs]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffffa03e7f92>] zpl_read_common_iovec.clone.1+0x72/0xc0 [zfs]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffffa03e809e>] zpl_aio_read+0xbe/0xf0 [zfs]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff81199d7a>] do_sync_read+0xfa/0x140
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff810a6930>] ? autoremove_wake_function+0x0/0x40
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8113ed90>] ? __free_pages+0x60/0xa0
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8113ee19>] ? free_pages+0x49/0x50
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8123ae36>] ? security_file_permission+0x16/0x20
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8119a675>] vfs_read+0xb5/0x1a0
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8119a7da>] sys_pread64+0x7a/0x90
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Code: e8 f0 25 17 e1 66 ff 03 66 66 90 eb a6 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 83 ec 68 0f 1f 44 00 00 <80> bf 3e 01 00 00 02 49 89 fe 0f 84 9a 00 00 00 f0 48 83 05 91 
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: RIP  [<ffffffffa03d96f6>] zil_commit+0x16/0x890 [zfs]
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: RSP <ffff8810222c3b98>
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: CR2: 000000000000013e
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: ---[ end trace 5914d97150a25ab4 ]---
Aug  8 12:09:21 cgs4285-a.cs.fiu.edu kernel: Kernel panic - not syncing: Fatal exception
@tuxoko
Copy link
Contributor

tuxoko commented Aug 10, 2017

The offending code is here.
https://github.com/zfsonlinux/zfs/blob/46364cb2f35545a7fc915df9593b719a94c43a83/module/zfs/zfs_vnops.c#L487

An obvious fix would be to check z_log is NULL. Though I wonder why we need to do zil_commit for zfs_read.

@tuxoko tuxoko mentioned this issue Aug 10, 2017
Merged
13 tasks
tonyhutter pushed a commit that referenced this issue Aug 22, 2017
@tuxoko @tonyhutter
Fix NULL pointer when O_SYNC read in snapshot
aec4318 
When doing read on a file open with O_SYNC, it will trigger zil_commit.
However for snapshot, there's no zil, so we shouldn't be doing that.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Signed-off-by: Chunwei Chen <[email protected]>
Closes #6478 
Closes #6494
Fabian-Gruenbichler pushed a commit to Fabian-Gruenbichler/zfs that referenced this issue Sep 29, 2017
@tuxoko @Fabian-Gruenbichler
Fix NULL pointer when O_SYNC read in snapshot
cfbf6f6 
When doing read on a file open with O_SYNC, it will trigger zil_commit.
However for snapshot, there's no zil, so we shouldn't be doing that.

Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Signed-off-by: Chunwei Chen <[email protected]>
Closes openzfs#6478 
Closes openzfs#6494
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tuxoko @flynnjFIU