dom0 kernel panic with linux 5.16.5 on HP ProBook 445 G7

[marmarek]

How to file a helpful issue

Qubes OS release

R4.1

Brief summary

Booting Qubes-20220205-kernel-latest-x86_64.iso panics on HP ProBook 445 G7. The same kernel boots fine on KVM (other openQA tests). 5.15.14 works fine too.

https://openqa.qubes-os.org/tests/30805

Steps to reproduce

Take Qubes-20220205-kernel-latest-x86_64.iso and boot on the affected hardware.

Expected behavior

Installer starts fine.

Actual behavior

Few OOPSes and finally kernel panic. Specific messages vary, but it always first starts with general protection fault, probably for non-canonical address in __kmalloc or kmem_cache_alloc_trace. Example message (extracted from pstore (EFI vars)):

<4>[   24.376310] general protection fault, probably for non-canonical address 0x6fcfc5a650a2a633: 0000 [#1] PREEMPT
 SMP NOPTI
<4>[   24.377403] CPU: 3 PID: 1536 Comm: systemd-udevd Not tainted 5.16.5-1.fc32.qubes.x86_64 #1
<4>[   24.378387] Hardware name: HP HP ProBook 445 G7/8730, BIOS S79 Ver. 01.03.01 10/23/2020
<4>[   24.379066] hp_wmi: query 0xd returned error 0x5
<4>[   24.379382] RIP: e030:kmem_cache_alloc_trace+0xf6/0x2f0
<6>[   24.380242] input: HP WMI hotkeys as /devices/virtual/input/input17
<4>[   24.381171] Code: 7e 48 8b 51 08 48 8b 01 48 83 79 10 00 48 89 04 24 0f 84 a0 01 00 00 48 85 c0 0f 84 97 01 00
 00 8b 4d 28 48 8b 7d 00 48 01 c1 <48> 8b 19 48 89 ce 48 33 9d b8 00 00 00 48 0f ce 48 31 f3 40 f6 c7
<4>[   24.381176] RSP: e02b:ffffc90040d1bc20 EFLAGS: 00010206
<4>[   24.381180] RAX: 6fcfc5a650a2a233 RBX: 0000000000000dc0 RCX: 6fcfc5a650a2a633
<4>[   24.381183] RDX: 0000000000a24003 RSI: 0000000000000dc0 RDI: 0000000000036180
Oops#1 Part2
<4>[   24.387218] RBP: ffff888100040c00 R08: 0000000000000000 R09: ffff8881001e7910
<4>[   24.388336] R10: ffff888104943b78 R11: 0000000000000000 R12: 0000000000000000
<4>[   24.389259] hp_wmi: query 0x1b returned error 0x5
<4>[   24.389356] R13: 0000000000000dc0 R14: 0000000000000568 R15: ffffffff819b288e
<4>[   24.391184] FS:  00007b275584bb80(0000) GS:ffff8881350c0000(0000) knlGS:0000000000000000
<4>[   24.392304] CS:  e030 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[   24.393336] CR2: 000058cedd37cac0 CR3: 000000011ba60000 CR4: 0000000000050660
<4>[   24.394366] Call Trace:
<4>[   24.395375]  <TASK>
<4>[   24.396450]  input_allocate_device+0x1e/0xf0
<4>[   24.397446]  pcspkr_probe+0x13/0xa0 [pcspkr]
<4>[   24.398432]  platform_probe+0x3f/0x90
<4>[   24.399411]  call_driver_probe+0x24/0xc0
<4>[   24.400386]  really_probe+0x1e7/0x310
<4>[   24.401357]  __driver_probe_device+0xfe/0x180
<4>[   24.402422]  driver_probe_device+0x1e/0x90
<4>[   24.402868]  __driver_attach+0xc0/0x1c0
<4>[   24.403208]  ? __device_attach_driver+0xe0/0xe0
<4>[   24.403552]  ? __device_attach_driver+0xe0/0xe0
<4>[   24.403892]  bus_for_each_dev+0x89/0xd0
<4>[   24.404231]  bus_add_driver+0x149/0x1e0
<4>[   24.404571]  driver_register+0x8f/0xe0
<4>[   24.404906]  ? 0xffffffffc0371000
<4>[   24.405233]  do_one_initcall+0x57/0x230
<4>[   24.405560]  do_init_module+0x5c/0x270
<4>[   24.405880]  __do_sys_finit_module+0xae/0x110
<4>[   24.406201]  do_syscall_64+0x3b/0x90
<4>[   24.406524]  entry_SYSCALL_64_after_hwframe+0x44/0xae
<4>[   24.406847] RIP: 0033:0x7b275697813d
<4>[   24.407167] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 2b 6d 0c 00 f7 d8 64 89 01 48
Oops#1 Part1
<4>[   24.407859] RSP: 002b:00007ffcdeed5e38 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
<4>[   24.408203] RAX: ffffffffffffffda RBX: 000058cedd3830e0 RCX: 00007b275697813d
<4>[   24.408552] RDX: 0000000000000000 RSI: 00007b27565d595d RDI: 0000000000000006
<4>[   24.408899] RBP: 0000000000020000 R08: 0000000000000000 R09: 0000000000000007
<4>[   24.409239] R10: 0000000000000006 R11: 0000000000000246 R12: 0000000000000000
<4>[   24.409584] R13: 00007b27565d595d R14: 000058cedd3d6170 R15: 000058cedd3e1000
<4>[   24.409920]  </TASK>
<4>[   24.410240] Modules linked in: pcspkr(+) hp_wmi platform_profile sparse_keymap wmi_bmof fjes(-) acpi_cpufreq(-) pcc_cpufreq(-) iwlwifi ucsi_acpi i2c_piix4 typec_ucsi joydev cfg80211 typec hp_accel lis3lv02d wireless_hotkey fuse binfmt_misc loop rfkill hid_multitouch rtsx_pci_sdmmc amdgpu mmc_core iommu_v2 crct10dif_pclmul crc32_pclmul gpu_sched i2c_algo_bit crc32c_intel 8021q drm_kms_helper garp ghash_clmulni_intel mrp serio_raw stp xhci_pci llc xhci_pci_renesas cec drm_ttm_helper ttm xhci_hcd drm rtsx_pci ccp r8169 wmi video i2c_hid_acpi i2c_hid sunrpc sha256_ssse3 dm_crypt trusted asn1_encoder dm_round_robin dm_multipath linear raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx raid1 raid0 iscsi_ibft scsi_dh_rdac scsi_dh_hp_sw scsi_dh_emc scsi_dh_alua squashfs cramfs be2iscsi bnx2i cnic uio cxgb4i cxgb4 tls cxgb3i cxgb3 mdio libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi edd xen_acpi_processor xen_privcmd xen_pciback xen_blkback
<4>[   24.410284]  xen_gntalloc xen_gntdev scsi_transport_iscsi xen_evtchn
<4>[   24.413515] ---[ end trace 02bd565c23426e94 ]---

Booting the same kernel without Xen crashes too. Blacklisting hp_wmi does not change anything.

[marmarek]

Bisected down to torvalds/linux@6b5ad4bd0d78

[DemiMarie]

Is this still an issue?

[marmarek]

I haven't seen this issue for a long time.