Support for custom/relocatable install prefixes (Conda packages)

[mbargull]

I've packaged podman and buildah (plus crun, runc, conmon, fuse-overlayfs, and dependencies) as Conda packages for conda-forge over at https://github.com/conda-forge/staged-recipes/pulls/mbargull. I also updated our build of skopeo over at https://github.com/conda-forge/skopeo-feedstock. (NB: The build PRs for podman, buildah, and updated skopeo are not yet reviewed/merged but the packages will be available soon.)

In the process of that I had to patch podman/buildah/skopeo (well https://github.com/containers/common and https://github.com/containers/image to be exact) to support the peculiarities of installing software in Conda environments:

• for podman: https://github.com/conda-forge/staged-recipes/blob/59eebb34af86760a7a6276a6348ba8e72b0bd621/recipes/podman/0003-Allow-relative-paths-for-system-config-files.patch
• for buildah: https://github.com/conda-forge/staged-recipes/blob/86087166cbacbbb4d1253a4a75198c93e332ea6f/recipes/buildah/0004-Use-config-files-in-prefix-as-fallback.patch
• for skopeo: https://github.com/conda-forge/skopeo-feedstock/blob/e9481a1459920b2aad8a8b3e026f5bf9f2bb2ef2/recipe/0003-Allow-relative-paths-for-system-config-files.patch

These patches work and give me everything to be able to, e.g., conda install podman and have a user installed podman available on any (glibc-based) Linux distribution.
I wanted us (conda-forge and Bioconda) to have functional packages first (so we can start to make use of the very useful tools you provide :D) and then see to refine/replace all of that.
Because, frankly, those patches aren't high quality at all -- I went for minimal changeset, only tested them manually, and, well, I'm no Go developer and haven't dug terribly much into your code.
And this is why I'm here: To see what your expert thoughts are and to (hopefully) work out what could be done upstream in your projects to allow our use cases (with less fiddly patches).

---

Let me explain what those "peculiarities" we have with Conda installed software are:
The conda package manager lets users install software into "environments" created at arbitrary prefixes, e.g.,

conda create --prefix=/path/to/podman-env podman

will create a directory /path/to/podman-env (separate from the system /usr installation) where you'd find podman at /path/to/podman-env/bin/podman. You can then "activate" that environment (which is really just setting PATH="/path/to/podman-env:${PATH}") with conda activate /path/to/podman-env and have podman readily available.
From the user perspective this is sometimes compared to containers because you have "one installation installed on top another". Technically, they aren't that similar, of course, because you get (by design) no kind of isolation from the host system. You can, however, compare it to have statically linked binaries installed at /path/to/podman-env/bin. But instead of just the executables, we also have /path/to/podman-env/lib, /path/to/podman-env/share, etc. in that prefix. However, we don't (necessarily) use static linking but we do work with a binary package manager.

Which leads to this: Conda allows the user (without root privileges and without touching /etc, /usr, etc.) to install precompiled software packages into, e.g., their home folders. The packages thus not only carry relocatable binaries but all other files are also relocatable. Since we don't know the installation prefix at compile time, but do want to install fully functional packages, we have to infer paths to supplementary files at run time (or at least at installation time).

---

Concrete example for the containers tools: We want to ship default configuration files like etc/containers/containers.conf or etc/containers/policy.json, install them into the prefix at, e.g., /path/to/podman-env/etc/containers/ and have the tools read them for default values.
The package manager of the system would install default configuration files into /etc or /usr/share but these paths are not accessible for us since we are installing rootless.

The make-shift patches above still honor the configs from /etc/containers if they are available, but fallback to using the default configurations we have in our install prefix. For podman and skopeo this simply determines the path of the current executable (which we know is installed at PREFIX/bin/) and uses relative paths, e.g., ../etc/containers/containers.conf then. For buildah, which re-execs itself (re-exec in podman seems to be confined to the runtimes themselves) I had to make it even more hacky and embed a C string* into the binary which will be replaced at install time with the path to the installation prefix (forgive me for the ugliness of the patch; I have no experience with Go).
(* NB: The patch carries an empty strings but I replace that in a build script with a ~255 character string to make room for the replacement at install time.)

---

I look forward to hearing your thought on this and if/in which way this Conda use case could be incorporated upstream.

Cheers,
Marcel

[mbargull]

One other issue related to the embedded C string in buildah: We make heavy use of relative RPATHs, e.g., @ORIGIN/../lib so we can avoid having to patch binaries at install time. However, with executable that move themselves for re-execution, like buildah or the container runtimes (crun/runc), this doesn't work. I'd be interested if this topic came up before and what your thoughts are one this.

[rhatdan]

@containers/podman-maintainers WDYT?

[rhatdan]

Please open a PR for this, if common needs to be modified.

[rhatdan]

Since no action was ever taken on this, I am going to assume reporter is no longer interested.