How does this work?

[dsedivec]

Hi! I see lots of advice on the Internet to use this container when you need to, say, build or run cross-arch images, like building linux/arm64 images from an x86_64 host. However, I cannot for the life of me understand how this container works.

When called with --install, the Go code looks like it just sets up the binfmt_misc mappings via /proc/sys/fs/binfmt_misc/register. I see the /usr/bin/qemu-* binaries included in this image, but I don't see that they're used anywhere. I expected that this container would be using dark magic to copy the qemu binaries somewhere that the host would be able to use them after the container exits.

What am I missing?

Edit: I just noticed that the README says you're providing "helper for BuildKit's automatic emulation support" and that "These binaries are BuildKit specific and should not be installed in kernel with binfmt_misc." (Emphasis added.) The linked PR is also closed, though, and I didn't see a new one re-opened. Are the included qemu images perhaps meant for use by BuildKit in some future version of BuildKit? If so, does that mean that Docker will always somehow have the qemu binaries available, and running binfmt --install from your container just makes sure all the binfmt_misc handlers point to the stock Docker qemu binaries?

Edit 2: Is it just my imagination, or is your personal repository used by 80% of the container/Docker/K8s ecosystem? :) I'm finding this repo referred to from everywhere. It looks something like the "package" included in LinuxKit, except yours is up-to-date and LinuxKit's seems to include very old qemu.

[tonistiigi]

This repo packages upstream qemu-user with some additional patches so it works well in containers with a container-friendly installer.

It can be built in two modes. One is "mainline" that is for installing in your kernel. Another is buildkit-specific that is only used by buildkit that directly invokes the binaries(if you don't want to install anything in kernel). Unless you are a buildkit maintainer you only care about the mainline.

I see the /usr/bin/qemu-* binaries included in this image, but I don't see that they're used anywhere.

The binaries are set as a target to the register script. Flag F is used https://en.wikipedia.org/wiki/Binfmt_misc that effectively locks the file/fd to the kernel at install time so the path doesn't matter after installation has completed.

[dsedivec]

Thank you very much! Excellent description, and now that I know about the F flag—my mistake for missing that in the documentation—this gets me unstuck.