ZFSBootMenu does not boot

[curiousercreative]

• Model: galp5
• BIOS version: 2022-11-21_2021-07-20_93c2809-106-gb337ac6-dirty
• EC version: 2a8befc195cd821232d2934593d29337e4eddb3e (Nov 18, 2022)

Cannot boot into the portable ZFSBootMenu EFI

Steps to reproduce

1. Follow the steps here to create a portable bootloader on thumb drive
2. Reboot device
3. Press ESC to enter firmware interface
4. Select One time boot
5. Select the thumb drive with portable bootloader installed

Expected behavior

A bootloader UI should load

Actual behavior

Black screen indefinitely, CPU seems to get warm

Notes

• The portable bootloader worked fine on my desktop (HP Z840) as a UEFI one time boot.
• Overwriting the BOOTX64.EFI file on my internal nvme ESP results in the same behavior.
• Adding an entry to the systemd-boot loader results in the same behavior.
• Does not boot via scripts/qemu.sh either

May relate to zbm-dev/zfsbootmenu#431

[curiousercreative]

Here's the qemu script and the accompanying output, the final error I'll share here as well:

[DEBUG]  Loading segment from ROM address 0xff826788
[DEBUG]    Entry Point 0x008033c0
[DEBUG]  BS: BS_PAYLOAD_LOAD run times (exec / console): 83 / 9 ms
[DEBUG]  ICH-NM10-PCH: watchdog disabled
[DEBUG]  Jumping to boot code at 0x008033c0(0x7ff95000)
!!!! X64 Exception Type - 06(#UD - Invalid Opcode)  CPU Apic ID - 00000000 !!!!
RIP  - 000000007FD1C000, CS  - 0000000000000038, RFLAGS - 0000000000010082
RAX  - 000000017F812298, RCX - 000000007FF53018, RDX - 000000003FFFC000
RBX  - 000000017F2A71F0, RSP - 0000000002FE4588, RBP - 0000000002FE45B0
RSI  - 000000007FF53018, RDI - 000000017F812298
R8   - 000000007B455590, R9  - 0000000000001000, R10 - 0000000000000005
R11  - 0000000000000000, R12 - 0000000000000000, R13 - 000000017F96C740
R14  - 000000017F96A80E, R15 - 0000000000000000
DS   - 0000000000000030, ES  - 0000000000000030, FS  - 0000000000000030
GS   - 0000000000000030, SS  - 0000000000000030
CR0  - 0000000080010011, CR2 - 0000000000000000, CR3 - 0000000002801000
CR4  - 0000000000000228, CR8 - 0000000000000000
DR0  - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3  - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 00000000000F0400
GDTR - 000000007FF4E000 0000000000000047, LDTR - 0000000000000000
IDTR - 000000017FB69018 0000000000000FFF,   TR - 0000000000000000
FXSAVE_STATE - 0000000002FE41E0
!!!! Can't find image information. !!!!

[ahesford]

Can you boot the separate kernel and initramfs corresponding to the release that's causing you trouble?

[curiousercreative]

Can you boot the separate kernel and initramfs corresponding to the release that's causing you trouble?

@ahesford I haven't tried that and don't quite know how, but happy to give it a shot if you can point me in the right direction. I did confirm that the latest void Linux live boots without issue (glibc, musl, xfce variants) and I also confirmed that zbm 1.10 efi fails to boot.

[ahesford]

You can grab a tarball of the release components with

curl -LJO https://get.zfsboot.menu/components/release

Because you mentioned trying systemd-boot, you can just unpack the tarball (which should contain a kernel and initramfs image) to your EFI System Partition and configure systemd-boot to use them.

[curiousercreative]

@ahesford ah, OK. I can try that on my physical machine, but won't get any firmware logs (don't have the hardware). Maybe I can also clone my ESP to get systemd-boot to run in qemu with logging as well

[curiousercreative]

@ahesford hey, that works! I was able to boot into zbm via systemd-boot by creating an entry as follows:

title ZFSBootMenu
linux /EFI/zbm/vmlinuz-bootmenu
initrd /EFI/zbm/initramfs-bootmenu.img

Chain loading the efi still not working, though now I at least get an error message from systemd-boot about it rather than qemu firmware logs.

Execution of embedded linux image failed: Load Error
Failed to execute ZFSBootMenu (\EFI/zbm/zbm.efi): Load Error

Here's the loader entry for failed efi:

title ZFSBootMenu
efi /EFI/zbm/zbm.efi

[curiousercreative]

@ahesford also FWIW my machines run Pop!_OS which currently packages Linux 6.6.6 generic and of course has no trouble booting.

$ uname -r
6.6.6-76060606-generic

[ahesford]

Unfortunately, there seems to be some incompatibility with the UEFI stub that we use to build EFI bundles and certain hardware or firmware. Your best bet for now is to continue using the separate components, which you can boot with systemd-boot or rEFInd. In these kinds of situtations, it's often convenient to create two menu entries: a default one that passes zbm.skip to the kernel so that it doesn't present the ZBM menu at all, and another that passes zbm.show to force the menu to appear. This lets systemd-boot or rEFInd take over countdown duty, letting you boot a default option but giving you a fallback if necessary.

If you are adventurous, you could try building a custom EFI bundle using a different UEFI stub (e.g., a more recent version that comes with Arch Linux) to see if this problem persists. If not, we can investigate switching to a more compatible stub.

[curiousercreative]

For others running Pop! and interested in moving to root on ZFS, I took notes for how I migrated an existing Pop! install and in particular, these notes for how to get ZFSBootMenu chainloaded via systemd-boot. It's basically a modified version of the ZFSBootMenu guide for Ubuntu install from scratch.