[RFC] many: make bootstrap container usage a manifest option

[mvo5]

[draft as a) it lacks tests b) there are multiple ways of doing this and this outlines the one I like best but that may not match with the others of course]

This PR adds the option to use a bootstrap container when generating the manifest. This allows a seamless experience when using e.g. image-builder:

$ image-builder build --arch=riscv64 minimal-raw --distro fedora-42

(or --arch=aarch64 etc, as long as there is an upstream container with python3 for the given distro)

The approach is to make the distro bootstrap container ref part of the manifest struct just like we set "Distro" there. This is not ideal, it would be much nicer if manifest would hold the full distro.Distro and this way we could simply run something like distro.BootstrapContainerRef() when needed. However because of circular inputs that is not possible.

The other option would be to change the signature of image.ImageKind.InstantiateManifest, e.g.

- InstantiateManifest(m *manifest.Manifest, repos []rpmmd.RepoConfig, runner runner.Runner, rng *rand.Rand) (*artifact.Artifact, error)
+ InstantiateManifest(m *manifest.Manifest, repos newpkg.RepoAndBootrapConfig, runner runner.Runner, rng *rand.Rand) (*artifact.Artifact, error)

here it would be either a new rpmmd.RepoAndBootstrapConfig with a struct that holds []rpmmd.RepoConfig and BootstrapRef - but that feels a bit out of place in rpmmd (because of the container ref that is unreleated to rpm repos). The RepoAndBootstrapConfig cannot be part of manifest,image or distro because of circular imports.

Alternatively:

- InstantiateManifest(m *manifest.Manifest, repos []rpmmd.RepoConfig, runner runner.Runner, rng *rand.Rand) (*artifact.Artifact, error)
+ InstantiateManifest(m *manifest.Manifest, repos []rpmmd.RepoConfig, bootstrapRef string, runner runner.Runner, rng *rand.Rand) (*artifact.Artifact, error)

that just adds a boostrapRef arg. That seems ok but requires a lot of (mechanical) changes.

Note that I think eventually the bootstrap container ref will be part of the "distro.yaml" and the currently (slightly ugly) helper bootstrapContainerFor() will go away.

The matching and small image-builder-cli changes are in https://github.com/osbuild/image-builder-cli/compare/main...mvo5:cross-arch-for-all-2?expand=1 (and the cross-build test in there passes locally but takes ~30min for all 4 tested arches and it lacks a check that inside the generate container image we actually have binaries of the expected architecture)

Note that cross-arch building will not support selinux in the buildroot yet - see the second commit about how we can still archive this in the future.

[achilleas-k]

I think the bootstrap container ref should be treated like any other source during the manifest instantiation and serialization flow. I don't understand why it needs to be part of the InstantiateManifest() call.

So, right now we have:
(setting aside our plans to simplify this flow for now)

// options contains ostree refs, so ignore the inconsistencies for now
img, err := t.image(w, t, bp, options, staticPackageSets, containerSources, rng)

mf := manifest.New()
img.InstantiateManifest(&mf, repos, runner, rng)

// get source specs
pkgSets := mf.GetPackageSetChains()
containerSources := mf.GetContainers()
commitSources := mf.GetCommits()

// resolve each
packages := depsolve(packages)
containers := resolveContainers(containerSources)
commits := resolveCommits(commitSources)

finalmf := mf.Serialize(packages, containers, commits)

I think the bootstrap container should come from outside this flow, just like we do with repositories. It can be a config file we embed and can override externally. Exactly like we do with repositories.

Unlike repositories though, I like that it goes into the image function like you did in the PR. In fact, maybe repositories should go into the image function too (but, again, let's ignore that for now).